Real-time rendering system and method

Abstract

Provided in the present disclosure are a real-time rendering system and method, which are used for implementing the cross-platform deployment of a rendering service on clients in different network environments by means of a serving end generating a rendering stream and pushing the rendering stream to the clients. The system comprises a serving end and a client, wherein the serving end generates an application file and an access address of an application, and configures a transmission parameter for the application, which application file comprises a three-dimensional scene model and a logic code of the application; the serving end generates a rendering stream on the basis of the application file, which rendering stream is associated with the access address and is used for displaying, in the form of a video stream, the three-dimensional scene model of the application; and the client acquires the access address of the application, establishes a communication connection with the serving end on the basis of the access address, receives the rendering stream that is sent by the serving end on the basis of the transmission parameter, and plays and displays the received rendering stream by means of a client playing medium.

Description

Real-time rendering system and method Technical Field

[0001] The present disclosure relates to the field of three-dimensional rendering technology, and in particular to a real-time rendering system and method. Background Art

[0002] With the increasing maturity of 5G technology, the continuous upgrading of technologies such as AI (Artificial Intelligence), IoT (Internet of Things), and cloud computing, as well as the various possibilities of the metaverse era, 3D rendering scenarios are widely used in related fields such as digital twins and visualization applications. The development of industries and enterprises is also placing increasingly higher demands on real-time 3D rendering technology, and more application scenarios require the simultaneous use of multiple capabilities.

[0003] 3D rendering scenarios can use 3D rendering engines to facilitate business application development. However, due to the inherent virtualization nature of 3D rendering scenarios, actual development and construction can be extremely cumbersome. Current 3D rendering services are typically deployed on devices, requiring development of specific 3D rendering services based on the network environments and platforms of each device, making cross-platform deployment impossible.

[0004] Summary of the Invention

[0005] The present disclosure provides a real-time rendering system and method for realizing cross-platform deployment of rendering services on clients in different network environments by generating a rendering stream on a server and pushing it to a client.

[0006] In a first aspect, an embodiment of the present disclosure provides a real-time rendering system, including a server and a client, wherein:

[0007] The server generates an application file and access address for the application, and configures transmission parameters for the application. The application file includes a three-dimensional scene model and logic code for the application. The transmission parameters are used to represent parameters required for the application to be transmitted from the server to the client for display.

[0008] The server generates a rendering stream according to the application file, the rendering stream is associated with the access address, and the rendering stream is used to display the three-dimensional scene model of the application in the form of a video stream;

[0009] The client obtains the access address of the application, establishes a communication connection with the server according to the access address, receives the rendering stream sent by the server based on the transmission parameters, and plays and displays the received rendering stream through the client playback medium.

[0010] As an optional implementation, the server includes a rendering system and a real-time communication server, wherein the real-time communication server is used to establish a communication connection between the rendering system and the client based on real-time communication technology; the server is specifically used to:

[0011] generating an application file of the application using the rendering system, configuring transmission parameters of the application, and uploading the transmission parameters to the real-time communication server;

[0012] The real-time communication server is used to generate an access address for the application.

[0013] As an optional implementation manner, after the server generates the access address of the application using the real-time communication server, the server further includes:

[0014] The server uploads the access address to the service platform, generates a rendering stream according to the application file, and sends the rendering stream to the real-time communication server;

[0015] The client obtains the access address of the application from the service platform, establishes a communication connection with the real-time communication server according to the access address, and receives a rendering stream sent by the real-time communication server.

[0016] As an optional implementation, the rendering system includes a 3D rendering engine and a pixel stream plug-in; the server is specifically configured to:

[0017] generating an application file for an application using the three-dimensional rendering engine;

[0018] The three-dimensional rendering engine is used to generate a pixel stream of the application based on the application file, and the pixel stream plug-in is used to convert the pixel stream into a rendering stream.

[0019] As an optional implementation, the rendering system includes a configuration interface; the server is specifically configured to:

[0020] Displaying configuration options of the application on the configuration interface; configuring transmission parameters of the application in response to transmission parameters input by the user in the configuration options on the configuration interface; or,

[0021] Based on the customized transmission parameters, the transmission parameters of the application are configured.

[0022] As an optional implementation,

[0023] The real-time communication server includes a signaling service, and the signaling service is used to implement the interaction of multiple types of signaling between the server and the client; and / or,

[0024] The client includes a client engine, and the client engine is used to connect to the signaling service and interact with the signaling service for multiple types of signaling.

[0025] As an optional implementation, the real-time communication server includes a signaling service;

[0026] The signaling service is used to establish a communication connection between the rendering system and the client playback medium; or,

[0027] The signaling service is used to establish a communication connection between the real-time communication server and the client playing the medium; or

[0028] The signaling service is used to establish a communication service between the client and the client playing media.

[0029] As an optional implementation, the client includes a client engine, the client playback medium includes a player, the player is encapsulated based on the Flutter language, supports multiple transmission protocols, and the player is applied to clients in different network environments;

[0030] The client responds to an interaction instruction executed by a user on the player, sending the interaction instruction to the signaling service via the client engine;

[0031] The signaling service sends the interaction instruction to the rendering system, and the rendering system responds to the interaction instruction and sends the response result to the player through the signaling service.

[0032] As an optional implementation, the client playback medium includes a browser; and the server is specifically configured to:

[0033] The client sends the interaction instruction to the signaling service in response to the interaction instruction executed by the user on the browser;

[0034] The interaction instruction is sent to the rendering system by using the signaling service. The rendering system responds to the interaction instruction and sends the response result to the browser through the signaling service.

[0035] As an optional implementation manner, the signaling service includes a playback service and a streaming service;

[0036] The client playback medium includes a player, and the server is specifically configured to:

[0037] Using the playback service to send the interaction instruction to the client engine, and then sending it to the signaling service through the client engine; using the streaming service to send the response result to the player; or,

[0038] The client playback medium includes a browser, and the server is specifically configured to:

[0039] The interactive instruction is sent to the signaling service by using the playback service; and the response result is sent to the browser by using the streaming service.

[0040] As an optional implementation, the application file further includes interactive logic code, the rendering flow further includes a UI interactive interface, and the interactive logic code is used to generate the UI interactive interface;

[0041] After playing and displaying the received rendering stream through the client playback medium, it also includes:

[0042] The client plays the media in response to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server;

[0043] The server performs the corresponding operation according to the operation instruction, and sends the UI interaction result after the operation to the client to play the media.

[0044] As an optional implementation, the server further includes a material warehouse; the server is further configured to:

[0045] The application file is uploaded to a material warehouse, and a corresponding rendering flow is generated according to the application file stored in the material warehouse.

[0046] As an optional implementation, the transmission parameter includes a startup mode, where the startup mode is used to indicate the maximum number of clients that can access the same 3D rendering scene of the application; the server is specifically configured to:

[0047] The startup mode of the application is configured so that the same 3D rendering scene of the application can be accessed by multiple clients, but multiple clients cannot operate the same element in the 3D rendering scene at the same time.

[0048] As an optional implementation manner, the client is further configured to:

[0049] Acquire a resource parameter set of the client device, the resource parameter set including at least one resource parameter, the resource parameter being used to represent an operating state of the client device;

[0050] If the resource parameter set meets the first set condition, performing image enhancement processing on the rendering stream to obtain an enhanced video stream;

[0051] The enhanced video stream is played and displayed through the client playback medium.

[0052] As an optional implementation, the resource parameter set includes a streaming mode and a network speed for the rendering stream; and the first setting condition includes:

[0053] The current network speed of the client device is less than or equal to the set network speed threshold; and / or,

[0054] The streaming mode of the rendering stream received by the client device does not meet the preset requirements.

[0055] As an optional implementation manner, the streaming mode includes a resolution of the rendering stream; and the first setting condition includes:

[0056] The resolution of the rendering stream received this time by the client device is lower than or equal to the resolution of the rendering stream received last time.

[0057] In an optional implementation manner, the resource parameter set includes a network card configuration. If the resource parameter set meets the first setting condition, the client is specifically configured to:

[0058] Determining hardware resources used for image enhancement processing based on whether the network card configuration includes a discrete graphics card and an integrated graphics card;

[0059] The hardware resources are used to perform image enhancement processing on the rendering stream to obtain an enhanced video stream.

[0060] As an optional implementation manner, the resource parameter set further includes CPU resources; and the client is specifically configured to:

[0061] If the network card configuration includes an independent graphics card and an integrated graphics card, it is determined that the hardware resources used for the image enhancement process include the integrated graphics card and CPU resources.

[0062] As an optional implementation manner, the resource parameter set includes GPU resources and video memory resources; and the client is specifically configured to:

[0063] If the network card configuration does not meet the conditions for performing image enhancement processing in the integrated graphics card, and the GPU resources are greater than or equal to the first threshold, and the video memory resources are greater than or equal to the second threshold, it is determined that the hardware resources used for image enhancement processing include GPU resources.

[0064] As an optional implementation manner, the resource parameter set further includes GPU resources; if the resource parameter set meets the first setting condition, the client is further configured to:

[0065] The enhancement level of the image enhancement algorithm is adjusted according to GPU resources, and the adjusted image enhancement algorithm is used to perform image enhancement processing on the rendering stream; the enhancement level is used to represent the intensity of image enhancement.

[0066] As an optional implementation manner, the client is further configured to:

[0067] If it is detected that the resource parameter set meets the first setting condition, the server is notified to adjust the transmission parameters of the application; or,

[0068] If it is monitored that the resource parameter set meets the first setting condition and the resource parameter set meets the second setting condition, the server is notified to adjust the transmission parameters of the application; the second setting condition includes: the resource parameters in the resource parameter set reach a peak value, and one resource parameter corresponds to a peak value; or, the network speed in the resource parameter set is less than or equal to the network speed threshold.

[0069] As an optional implementation, the transmission parameter includes a resolution of the rendering stream.

[0070] As an optional implementation manner, if the resource parameter set meets the first setting condition, the client is specifically configured to:

[0071] The rendering stream is stream-decoded to obtain a decoded video stream, and image enhancement processing is performed on the decoded video stream to obtain an enhanced video stream.

[0072] As an optional implementation, the client is deployed on a client device in a different network environment, where the network environment is used to represent an operating system or network service of the client device; and / or,

[0073] The server is deployed on a cloud server and / or a local area network service.

[0074] In a second aspect, an embodiment of the present disclosure provides a real-time rendering method, comprising:

[0075] In response to a first operation by a user on an application in the service platform displayed on the client device, the client obtains an access address of the application;

[0076] In response to a second operation in which the user inputs the access address into the client to play the media, establishing a communication connection with the server according to the access address;

[0077] The client receives the rendering stream associated with the access address sent by the server, and plays and displays the received rendering stream through the client playback medium. The rendering stream is used to display the three-dimensional scene model of the application in the form of a video stream.

[0078] As an optional implementation, the client includes a client engine, the client playback medium includes a player, the player is encapsulated based on the Flutter language, supports multiple transmission protocols, and the player is applied to clients in different network environments; the method also includes:

[0079] The client responds to the interactive instruction executed by the user on the player, and sends the interactive instruction to the server through the client engine, so that the server responds to the interactive instruction and sends the response result to the player;

[0080] The player displays the received response result.

[0081] As an optional implementation manner, the client playback medium includes a browser; and the method further includes:

[0082] The client sends the interactive instruction to the server in response to the interactive instruction executed by the user on the browser, so that the server responds to the interactive instruction and sends the response result to the browser;

[0083] The browser displays the received response result.

[0084] As an optional implementation, the rendering flow further includes a UI interaction interface, and the method further includes:

[0085] The client playback medium responds to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server, so that the server performs the corresponding operation according to the operation instruction and sends the UI interaction result after the operation to the client playback medium;

[0086] The client plays the media to display the received UI interaction results.

[0087] In a third aspect, an embodiment of the present disclosure provides a real-time rendering method, comprising:

[0088] Using the server to generate an application file and access address for the application, and configuring transmission parameters for the application, wherein the application file includes a three-dimensional scene model and logic code of the application, and the transmission parameters are used to represent parameters required for the application to be transmitted from the server to the client for display;

[0089] Using a server to generate a rendering stream according to the application file, the rendering stream is associated with the access address, and the rendering stream is used to display the three-dimensional scene model of the application in a video stream;

[0090] The client obtains the access address of the application, establishes a communication connection with the server according to the access address, receives the rendering stream of the server according to the transmission parameters, and plays and displays the received rendering stream through the client playback medium.

[0091] As an optional implementation, the server includes a rendering system and a real-time communication server, and the real-time communication server is used to establish a communication connection between the rendering system and the client based on RTC technology;

[0092] generating an application file of the application using the rendering system, configuring transmission parameters of the application, and uploading the transmission parameters to the real-time communication server;

[0093] The real-time communication server is used to generate an access address for the application.

[0094] As an optional implementation manner, after the server generates the access address of the application using the real-time communication server, the server further includes:

[0095] The server uploads the access address to the service platform, generates a rendering stream according to the application file, and sends the rendering stream to the real-time communication server;

[0096] The client obtains the access address of the application from the service platform, establishes a communication connection with the real-time communication server according to the access address, and receives a rendering stream sent by the real-time communication server.

[0097] As an optional implementation, the rendering system includes a three-dimensional rendering engine and a pixel stream plug-in;

[0098] generating an application file for an application using the three-dimensional rendering engine;

[0099] The three-dimensional rendering engine is used to generate a pixel stream of the application based on the application file, and the pixel stream plug-in is used to convert the pixel stream into a rendering stream.

[0100] As an optional implementation, the rendering system includes a configuration interface;

[0101] Displaying configuration options of the application on the configuration interface; configuring transmission parameters of the application in response to transmission parameters input by the user in the configuration options on the configuration interface; or,

[0102] Based on the customized transmission parameters, the transmission parameters of the application are configured.

[0103] As an optional implementation,

[0104] The real-time communication server includes a signaling service, and the signaling service is used to implement the interaction of multiple types of signaling between the server and the client; and / or,

[0105] The client includes a client engine, and the client engine is used to connect to the signaling service and interact with the signaling service for multiple types of signaling.

[0106] As an optional implementation, the real-time communication server includes a signaling service;

[0107] The signaling service is used to establish a communication connection between the rendering system and the client playback medium; or,

[0108] The signaling service is used to establish a communication connection between the real-time communication server and the client playing the medium; or

[0109] The signaling service is used to establish a communication service between the client and the client playing media.

[0110] As an optional implementation, the client includes a client engine, the client playback medium includes a player, the player is encapsulated based on the Flutter language, supports multiple transmission protocols, and the player is applied to clients in different network environments;

[0111] The client responds to an interaction instruction executed by a user on the player, sending the interaction instruction to the signaling service via the client engine;

[0112] The signaling service sends the interaction instruction to the rendering system, and the rendering system responds to the interaction instruction and sends the response result to the player through the signaling service.

[0113] As an optional implementation, the client playback medium includes a browser;

[0114] The client sends the interaction instruction to the signaling service in response to the interaction instruction executed by the user on the browser;

[0115] The interaction instruction is sent to the rendering system by using the signaling service. The rendering system responds to the interaction instruction and sends the response result to the browser through the signaling service.

[0116] As an optional implementation, the signaling service includes a playback service and a streaming service; the client playback medium includes a player:

[0117] Using the playback service to send the interaction instruction to the client engine, and then sending it to the signaling service through the client engine; using the streaming service to send the response result to the player; or,

[0118] The client playback medium includes a browser:

[0119] The interactive instruction is sent to the signaling service by using the playback service; and the response result is sent to the browser by using the streaming service.

[0120] As an optional implementation, the application file further includes interactive logic code, the rendering flow further includes a UI interactive interface, and the interactive logic code is used to generate the UI interactive interface;

[0121] After playing and displaying the received rendering stream through the client playback medium, it also includes:

[0122] The client plays the media in response to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server;

[0123] The server performs the corresponding operation according to the operation instruction, and sends the UI interaction result after the operation to the client to play the media.

[0124] As an optional implementation, the server further includes a material warehouse;

[0125] The application file is uploaded to a material warehouse, and a corresponding rendering flow is generated according to the application file stored in the material warehouse.

[0126] As an optional implementation, the transmission parameter includes a startup mode, where the startup mode is used to indicate the maximum number of clients accessing the same 3D rendering scene of the application;

[0127] The startup mode of the application is configured so that the same 3D rendering scene of the application can be accessed by multiple clients, but multiple clients cannot operate the same element in the 3D rendering scene at the same time.

[0128] As an optional implementation, the method further includes:

[0129] Acquire a resource parameter set of the client device, the resource parameter set including at least one resource parameter, the resource parameter being used to represent an operating state of the client device;

[0130] If the resource parameter set meets the first set condition, performing image enhancement processing on the rendering stream to obtain an enhanced video stream;

[0131] The enhanced video stream is played and displayed through the client playback medium.

[0132] As an optional implementation, the resource parameter set includes a streaming mode and a network speed for the rendering stream; and the first setting condition includes:

[0133] The current network speed of the client device is less than or equal to the set network speed threshold; and / or,

[0134] The streaming mode of the rendering stream received by the client device does not meet the preset requirements.

[0135] As an optional implementation manner, the streaming mode includes a resolution of the rendering stream; and the first setting condition includes:

[0136] The resolution of the rendering stream received this time by the client device is lower than or equal to the resolution of the rendering stream received last time.

[0137] As an optional implementation, the resource parameter set includes a network card configuration. If the resource parameter set meets the first setting condition, the hardware resources used for image enhancement processing are determined based on whether the network card configuration includes an independent graphics card and an integrated graphics card; the rendering stream is subjected to image enhancement processing using the hardware resources to obtain an enhanced video stream.

[0138] As an optional implementation, the resource parameter set further includes CPU resources;

[0139] If the network card configuration includes an independent graphics card and an integrated graphics card, it is determined that the hardware resources used for the image enhancement process include the integrated graphics card and CPU resources.

[0140] As an optional implementation, the resource parameter set includes GPU resources and video memory resources;

[0141] If the network card configuration does not meet the conditions for performing image enhancement processing in the integrated graphics card, and the GPU resources are greater than or equal to the first threshold, and the video memory resources are greater than or equal to the second threshold, it is determined that the hardware resources used for image enhancement processing include GPU resources.

[0142] As an optional implementation manner, the resource parameter set further includes GPU resources; if the resource parameter set meets the first setting condition, it further includes:

[0143] The client adjusts the enhancement level of the image enhancement algorithm according to GPU resources, and performs image enhancement processing on the rendering stream using the adjusted image enhancement algorithm; the enhancement level is used to represent the intensity of image enhancement.

[0144] As an optional implementation, it also includes:

[0145] If the client detects that the resource parameter set meets the first set condition, the client notifies the server to adjust the transmission parameters of the application; or

[0146] If the client monitors that the resource parameter set meets the first setting condition and the resource parameter set meets the second setting condition, the server is notified to adjust the transmission parameters of the application; the second setting condition includes: the resource parameters in the resource parameter set reach a peak value, and one resource parameter corresponds to a peak value; or, the network speed in the resource parameter set is less than or equal to the network speed threshold.

[0147] As an optional implementation, the transmission parameter includes a resolution of the rendering stream.

[0148] As an optional implementation, if the resource parameter set meets the first set condition, the rendering stream is stream-decoded to obtain a decoded video stream, and image enhancement processing is performed on the decoded video stream to obtain an enhanced video stream.

[0149] As an optional implementation, the client is deployed on a client device in a different network environment, where the network environment is used to represent an operating system or network service of the client device; and / or,

[0150] The server is deployed on a cloud server and / or a local area network server.

[0151] In a fourth aspect, an embodiment of the present disclosure provides a server device, comprising a processor and a memory, wherein the memory is configured to store a program executable by the processor, and the processor is configured to read the program in the memory and perform the following steps:

[0152] Generate an application file and access address for the application, and configure transmission parameters for the application, wherein the application file includes a three-dimensional scene model and logic code of the application, and the transmission parameters are used to represent parameters required for transmitting the application to a client device for display;

[0153] generating a rendering stream according to the application file, wherein the rendering stream is associated with the access address, and the rendering stream is used to display a three-dimensional scene model of the application in a video stream;

[0154] The rendering stream is sent to a client device based on the transmission parameters, wherein the client device includes a device that establishes a communication connection with a server device based on the access address.

[0155] As an optional implementation, the server device includes a rendering system and a real-time communication server, and the real-time communication server is used to establish a communication connection between the rendering system and the client device based on real-time communication technology; the processor is specifically used to:

[0156] generating an application file of the application using the rendering system, configuring transmission parameters of the application, and uploading the transmission parameters to the real-time communication server;

[0157] The real-time communication server is used to generate an access address for the application.

[0158] As an optional implementation manner, after the server device generates the access address of the application using the real-time communication server, the processor is further configured to:

[0159] Uploading the access address to a service platform, generating a rendering stream based on the application file, and sending the rendering stream to a real-time communication server, so that the client device can obtain the access address of the application from the service platform and establish a communication connection with the real-time communication server based on the access address;

[0160] The rendering stream is sent to the client device through the real-time communication server.

[0161] As an optional implementation, the rendering system includes a 3D rendering engine and a pixel stream plug-in; the processor is specifically configured to:

[0162] generating an application file for an application using the three-dimensional rendering engine;

[0163] The three-dimensional rendering engine is used to generate a pixel stream of the application based on the application file, and the pixel stream plug-in is used to convert the pixel stream into a rendering stream.

[0164] As an optional implementation, the rendering system includes a configuration interface; and the processor is specifically configured to:

[0165] Displaying configuration options of the application on the configuration interface; configuring transmission parameters of the application in response to transmission parameters input by the user in the configuration options on the configuration interface; or,

[0166] Based on the customized transmission parameters, the transmission parameters of the application are configured.

[0167] As an optional implementation, the real-time communication server includes a signaling service, and the signaling service is used to implement the interaction of multiple types of signaling between the server device and the client device.

[0168] As an optional implementation, the real-time communication server includes a signaling service;

[0169] The signaling service is used to establish a communication connection between the rendering system and the client playback medium; or,

[0170] The signaling service is used to establish a communication connection between the real-time communication server and the client playing the medium; or,

[0171] The signaling service is used to establish a communication service between the client and the client playing media.

[0172] As an optional implementation manner, the processor is specifically configured to:

[0173] receiving an interaction instruction sent by the client engine through the signaling service, and sending the interaction instruction to the rendering system;

[0174] The rendering system responds to the interaction instruction, and sends the response result to the client playback medium through the signaling service. The client playback medium includes a player. The player is encapsulated based on the Flutter language and supports multiple transmission protocols. The player is applied to clients in different network environments.

[0175] As an optional implementation manner, the processor is specifically configured to:

[0176] receiving, via the signaling service, an interaction instruction sent by the client playing the medium, and sending the interaction instruction to the rendering system;

[0177] The signaling service is used to send the interaction instruction to the rendering system. The rendering system responds to the interaction instruction and sends the response result to the client playback medium through the signaling service. The client playback medium includes a browser.

[0178] As an optional implementation manner, the signaling service includes a playback service and a streaming service; and the processor is specifically configured to:

[0179] Using the playback service to send the interaction instruction to the client engine, and then to the signaling service through the client engine; using the streaming service to send the response result to the client playback medium, wherein the client playback medium includes a player; or

[0180] The interactive instruction is sent to the signaling service by using the playback service; and the response result is sent to the client playback medium by using the streaming service, where the client playback medium includes a browser.

[0181] As an optional implementation, the application file further includes interactive logic code, the rendering flow further includes a UI interactive interface, and the interactive logic code is used to generate the UI interactive interface;

[0182] After playing and displaying the received rendering stream through the client playback medium, the processor is further configured to:

[0183] Receive operation instructions sent by the client to play the media;

[0184] Execute the corresponding operation according to the operation instruction, and send the UI interaction result after the operation to the client to play the medium.

[0185] As an optional implementation manner, the server device further includes a material warehouse; and the processor is further specifically configured to:

[0186] The application file is uploaded to a material warehouse, and a corresponding rendering flow is generated according to the application file stored in the material warehouse.

[0187] As an optional implementation, the transmission parameter includes a startup mode, where the startup mode is used to indicate a maximum number of clients accessing the same 3D rendering scene of the application; and the processor is specifically configured to:

[0188] The startup mode of the application is configured so that the same 3D rendering scene of the application can be accessed by multiple clients, but multiple clients cannot operate the same element in the 3D rendering scene at the same time.

[0189] In a fifth aspect, an embodiment of the present disclosure provides a client device, comprising a processor and a memory, wherein the memory is configured to store a program executable by the processor, and the processor is configured to read the program in the memory and perform the following steps:

[0190] Obtaining an access address for the application, and establishing a communication connection with a server based on the access address; associating the access address with a rendering stream for the application, wherein the rendering stream is used to display a three-dimensional scene model of the application via a video stream;

[0191] Receive the rendering stream associated with the access address sent by the server device, and play and display the received rendering stream through the client playback medium.

[0192] As an optional implementation manner, the processor is further configured to:

[0193] Acquire a resource parameter set, the resource parameter set including at least one resource parameter, the resource parameter being used to represent an operating state of the client device;

[0194] If the resource parameter set meets the first set condition, performing image enhancement processing on the rendering stream to obtain an enhanced video stream;

[0195] The enhanced video stream is played and displayed through the client playback medium.

[0196] As an optional implementation, the resource parameter set includes a streaming mode and a network speed for the rendering stream; and the first setting condition includes:

[0197] The current network speed of the client device is less than or equal to the set network speed threshold; and / or,

[0198] The streaming mode of the rendering stream received by the client device does not meet the preset requirements.

[0199] As an optional implementation manner, the streaming mode includes a resolution of the rendering stream; and the first setting condition includes:

[0200] The resolution of the rendering stream received this time by the client device is lower than or equal to the resolution of the rendering stream received last time.

[0201] As an optional implementation manner, the resource parameter set includes a network card configuration. If the resource parameter set meets the first setting condition, the processor is specifically configured to:

[0202] Determining hardware resources used for image enhancement processing based on whether the network card configuration includes a discrete graphics card and an integrated graphics card;

[0203] The hardware resources are used to perform image enhancement processing on the rendering stream to obtain an enhanced video stream.

[0204] As an optional implementation manner, the resource parameter set further includes CPU resources; and the processor is specifically configured to:

[0205] If the network card configuration includes an independent graphics card and an integrated graphics card, it is determined that the hardware resources used for the image enhancement process include the integrated graphics card and CPU resources.

[0206] As an optional implementation manner, the resource parameter set includes GPU resources and video memory resources; and the processor is specifically configured to:

[0207] If the network card configuration does not meet the conditions for performing image enhancement processing in the integrated graphics card, and the GPU resources are greater than or equal to the first threshold, and the video memory resources are greater than or equal to the second threshold, it is determined that the hardware resources used for image enhancement processing include GPU resources.

[0208] As an optional implementation manner, the resource parameter set further includes GPU resources; if the resource parameter set meets the first setting condition, the processor is further configured to:

[0209] The enhancement level of the image enhancement algorithm is adjusted according to GPU resources, and the adjusted image enhancement algorithm is used to perform image enhancement processing on the rendering stream; the enhancement level is used to represent the intensity of image enhancement.

[0210] As an optional implementation manner, the processor is further configured to:

[0211] If it is detected that the resource parameter set meets the first setting condition, the server is notified to adjust the transmission parameters of the application; or,

[0212] If it is monitored that the resource parameter set meets the first setting condition and the resource parameter set meets the second setting condition, the server is notified to adjust the transmission parameters of the application; the second setting condition includes: the resource parameters in the resource parameter set reach a peak value, and one resource parameter corresponds to a peak value; or, the network speed in the resource parameter set is less than or equal to the network speed threshold.

[0213] As an optional implementation, the transmission parameter includes a resolution of the rendering stream.

[0214] As an optional implementation manner, if the resource parameter set meets the first setting condition, the processor is specifically configured to:

[0215] The rendering stream is stream-decoded to obtain a decoded video stream, and image enhancement processing is performed on the decoded video stream to obtain an enhanced video stream.

[0216] As an optional implementation, the client is deployed on a client device in a different network environment, where the network environment is used to represent an operating system or network service of the client device; and / or,

[0217] The server is deployed on a cloud server and / or a local area network server.

[0218] In a sixth aspect, an embodiment of the present disclosure further provides a computer storage medium on which a computer program is stored, which, when executed by a processor, is used to implement the steps of the method described in the second or third aspect above.

[0219] These and other aspects of the present disclosure will become more readily apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0220] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0221] FIG1 is a schematic diagram of a real-time rendering system provided by an embodiment of the present disclosure;

[0222] FIG2 is a diagram of a deployment package setting interface of a UE provided by an embodiment of the present disclosure;

[0223] 3A-3C are schematic diagrams of an operation interface for accessing a rendering system provided by an embodiment of the present disclosure;

[0224] 4A-4B are schematic diagrams of a configuration interface provided by an embodiment of the present disclosure;

[0225] FIG5 is a schematic diagram showing the effect of displaying a rendering flow of an application provided by an embodiment of the present disclosure on a client;

[0226] FIG6 is a schematic diagram of an interaction between a client and a real-time rendering system provided by an embodiment of the present disclosure;

[0227] FIG7 is a flowchart of an implementation of dynamic client allocation provided by an embodiment of the present disclosure;

[0228] FIG8 is a flowchart of an implementation of an image enhancement service provided by an embodiment of the present disclosure;

[0229] FIG9 is a flowchart of a real-time rendering method according to an embodiment of the present disclosure;

[0230] 10A-10B are an operation interface of a client playing a media display rendering stream provided by an embodiment of the present disclosure;

[0231] FIG11 is a flowchart of a real-time rendering method according to an embodiment of the present disclosure;

[0232] FIG12 is a schematic diagram of a server device provided in an embodiment of the present disclosure;

[0233] FIG13 is a schematic diagram of a client device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0234] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the present disclosure will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present disclosure, rather than all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present disclosure without creative effort are intended to fall within the scope of protection of the present disclosure.

[0235] In the embodiments of the present disclosure, the term "and / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0236] The application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Persons skilled in the art will appreciate that, as new application scenarios emerge, the technical solutions provided by the embodiments of the present disclosure will also be applicable to similar technical problems. In the description of the present disclosure, unless otherwise specified, "multiple" means two or more.

[0237] With the increasing maturity of 5G technology, the continuous upgrading of technologies such as AI (Artificial Intelligence), IoT (Internet of Things), and cloud computing, as well as the various possibilities of the metaverse era, 3D rendering scenes are widely used in related fields such as digital twins and visualization applications. The development of industries and enterprises also places higher and higher demands on real-time 3D rendering technology, and more application scenarios require the simultaneous use of multiple capability combinations. 3D rendering scenes can use 3D rendering engines to assist in the development of business applications, but due to the virtualization properties of the 3D rendering scene itself, the actual development and construction process will be extremely cumbersome. Current 3D rendering services are usually deployed on terminals, and corresponding 3D rendering services need to be developed based on the network environment platforms of different terminals, making cross-platform deployment impossible.

[0238] To address the aforementioned technical issues, this embodiment provides a real-time rendering system that executes 3D rendering tasks on a server, such as through a cloud server or a local area network server. The rendering results are then transmitted to the client in the form of a video stream, i.e., a rendering stream is transmitted to the client. The client is responsible for receiving and displaying the rendering stream. The rendering and processing of the 3D scene no longer relies on the client device, but is instead handled by the server on the server side. The client device only needs to receive the rendering results transmitted by the server on the server side and display them in real time on a browser or other playback medium. By distributing the rendering tasks to a high-performance server, insufficient client device performance is addressed, allowing complex 3D scenes to run smoothly on a variety of devices and enabling real-time multi-user collaboration. Due to the powerful computing power of the server on the server side, it can support more complex lighting calculations, shadow generation, material details, and post-processing effects, making the 3D scene more realistic and providing an immersive experience for users. The server on the server side can process large amounts of scene data and models, supporting high-performance rendering of scenes such as large-scale cities, terrain, and natural environments, helping users better understand and analyze complex 3D environments. Because rendering tasks are completed in the cloud, client devices do not require powerful computing power, enabling cross-platform compatibility, including computers, mobile terminals, and virtual reality devices. Distributing rendering tasks to server-side servers reduces the performance requirements of client devices, allowing complex 3D scenes to run smoothly on a variety of devices, thereby reducing client performance requirements. Server-side rendering can significantly reduce the computing, memory, and storage resource requirements of client devices, extending device lifespan and thus conserving client resources. The server-based rendering architecture allows for rapid application deployment and upgrades, making it easier for users to access the latest features and optimizations. Servers can be elastically scaled based on actual needs, effectively responding to fluctuations in the number of users and computing tasks.

[0239] The real-time rendering system of this embodiment places the three-dimensional rendering task on the server side, and then transmits the rendering results to the client in the form of a video stream. This approach allows high-performance three-dimensional rendering to be performed on the server side, and transmits the rendering results (rendering stream) to the client in real time, providing users with a high-quality visual experience and reducing the requirements for client device performance. Since the actual rendering process is completed on the server side, the client device only needs to receive and display the rendering stream, so that even devices with lower performance can view high-quality three-dimensional rendering results, and is suitable for various devices, including low-performance smartphones and tablets. In short, the server-based real-time rendering system architecture has powerful functions and significant advantages, and can provide customers with high-performance, high-compatibility and high-security three-dimensional rendering services. This allows complex three-dimensional scenes to run smoothly on various devices, while also enabling multi-user real-time collaborative operations.

[0240] As shown in FIG1 , this embodiment provides a real-time rendering system, including a server 100 and a client 101 , wherein:

[0241] Step 1: The server generates an application file and access address for the application, and configures transmission parameters for the application. The application file includes a 3D scene model and logic code for the application. The transmission parameters represent the parameters required for the application to be transmitted from the server to the client for display.

[0242] Optionally, the transmission parameters in this embodiment include but are not limited to: application name, application description, executable program path, streaming method, startup mode, maximum number of concurrency, timeout period, and at least one or more of the application cover image.

[0243] The streaming method includes but is not limited to: the resolution of the rendering stream, the transmission protocol and other information;

[0244] The startup mode is used to indicate the maximum number of clients that can access the same 3D rendering scene of the application. For example, the same 3D rendering scene of the application can be accessed by one or more clients. When the same 3D rendering scene of the application can be accessed by multiple clients, it can be realized that users corresponding to multiple clients exist within the same 3D rendering scene, and users of multiple clients can operate the same 3D rendering scene, achieving the effect of multiple people accessing one person's operation.

[0245] The maximum concurrency indicates the maximum number of windows that can be simultaneously opened for display for the same application. This can mean displaying the application's rendering stream in one or more windows on the same client, or in windows on different clients. The number of windows is configured based on the maximum concurrency. A window can be understood as a window on the client playing media.

[0246] The timeout value is used to indicate the maximum time required for the client to establish a connection with the RTC (Real-Time Communication) server.

[0247] In step 2, the server generates a rendering stream based on the application file, the rendering stream is associated with the access address, and the rendering stream is used to display the three-dimensional scene model of the application in the form of a video stream;

[0248] In step 3, the client obtains the access address of the application, establishes a communication connection with the server according to the access address, receives the rendering stream sent by the server based on the transmission parameters, and plays and displays the received rendering stream through the client playback medium.

[0249] In some embodiments, the server of this embodiment is deployed on a cloud server and / or a local area network server, and the client of this embodiment is deployed on a client device in a different network environment, where the network environment is used to represent the operating system or network service of the client device.

[0250] Optionally, the network environment includes but is not limited to one or more of Android, iOS, Window, and Web, which are not limited in this embodiment.

[0251] In practice, the server-side device of this embodiment includes a cloud server / LAN server. The real-time rendering system of this embodiment implements the application's 3D rendering tasks via the cloud server / LAN server. The client in this embodiment resides on a client device, which includes but is not limited to a computer, mobile phone, iPad, or other terminal. This embodiment can support client devices in different network environments (cross-platform) to receive and display the rendering stream.

[0252] In some embodiments, the server in this embodiment includes a rendering system and an RTC server. The RTC server is used to establish a communication connection between the rendering system and the client based on RTC technology. The server is specifically used to:

[0253] The rendering system is used to generate an application file of the application, and transmission parameters of the application are configured, and the transmission parameters are uploaded to the RTC server; and the RTC server is used to generate an access address of the application.

[0254] In some embodiments, after the server generates the access address of the application using the RTC server, the server further performs the following process for the client to obtain the access address:

[0255] The server uploads the access address to the service platform, generates a rendering stream according to the application file, and sends the rendering stream to the RTC server;

[0256] The client obtains the access address of the application from the service platform, establishes a communication connection with the RTC server according to the access address, and receives the rendering stream sent by the RTC server.

[0257] During implementation, the rendering system first generates the application file, then configures the application's transmission parameters and uploads them to the RTC server. After receiving the transmission parameters, the RTC server generates the application's access address and feeds it back to the rendering system. The rendering system then uploads the access address fed back by the RTC server to the service platform, making it visible to the user. This allows the client to obtain the access address from the service platform, thereby establishing a communication connection between the client and the RTC server, and then establishing a communication connection between the client and the rendering system through the RTC server. It should be noted that since the RTC server is used to transmit the rendering stream generated by the rendering system to the client for playback and display, the RTC server needs to configure the transmission information of the rendering stream based on the transmission parameters uploaded by the rendering system.

[0258] It should be noted that RTC (Real-Time Communication) is a real-time communication technology, which is generally defined as a technology for transmitting voice, video and other media streams in real time over the network. This technology is mainly used in the field of real-time communication. The RTC server in this embodiment is a server built based on the RTC architecture. The functions of the RTC server include: (1) Media transmission: The RTC server is responsible for the transmission and forwarding of audio and video media to ensure the efficient transmission of real-time data. (2) Signaling processing: The RTC server processes signaling interactions between clients, including establishing connections, controlling call status, sending and receiving call-related instructions, etc. (3) Media processing: The RTC server may perform processing operations such as media encoding and decoding, mixing media streams, and adjusting audio and video quality to provide a better communication experience. (4) Resource management: The RTC server manages the resource allocation and scheduling of call participants, including bandwidth management, network routing optimization, etc., to provide stable and high-quality real-time communication.

[0259] In some embodiments, the rendering system in this embodiment includes a 3D rendering engine and a pixel stream plug-in; the server is specifically configured to:

[0260] generating an application file for an application using the three-dimensional rendering engine;

[0261] The three-dimensional rendering engine is used to generate a pixel stream of the application based on the application file, and the pixel stream plug-in is used to convert the pixel stream into a rendering stream.

[0262] During implementation, the 3D rendering engine UE (Unreal Engine) and the pixel stream plug-in (Pixel Streaming) can be used to generate the pixel stream of the application, realize the modification and conversion of the pixel stream to the rendering stream, and load the deployment package (application file), wherein the pixel stream is understood as the two-dimensional pixel data representation of the three-dimensional scene model, and the rendering stream is used to display the three-dimensional scene model of the application through a video stream, and transmit the three-dimensional scene model to the client playback medium for display in the form of a video stream; the deployment package represents the application file in this embodiment. Taking the three-dimensional rendering engine UE as an example, as shown in Figure 2, this embodiment provides a UE deployment package packaging setting interface diagram, and the specific deployment package packaging process is as follows:

[0263] First, import the project through the project directory. The project contains the written and debugged UE blueprint code and resource map by default. The blueprint code represents the logic code executed by the application; the resource map represents the three-dimensional scene model of the application.

[0264] Then, run the project to test whether the various functional modules of the project are running normally, and set the compilation configuration according to the project situation. If it is a debugging project, set it to "Development", and if it is an online project, set it to "Release". Then check whether the platform is supported, set the map and mode, check whether the GameMode is set correctly, check whether the main level is set correctly, and check whether the GameInstance is set correctly. Then package the project and integrate the compiled products into a zip. At this point, a deployment package with a unified format and an access address URL is generated.

[0265] It should be noted that the 3D rendering engine will not be used to render the application files before the application is launched. After the application is launched, the 3D rendering engine and pixel stream plug-in can be used to generate a rendering stream, encode the final result of each frame rendering, and use H.264 video compression to package the video frames into a media stream and send it to the RTC server. The RTC server will transmit the generated media stream to one or more clients through a direct peer-to-peer connection (two-way connection), which can achieve the interactive effect of multiple people sharing the perspective and one person operating.

[0266] In some embodiments, the rendering system of this embodiment includes a configuration interface; the server is specifically configured to:

[0267] Displaying configuration options of the application on the configuration interface; configuring transmission parameters of the application in response to transmission parameters input by the user in the configuration options on the configuration interface; or,

[0268] Based on the customized transmission parameters, the transmission parameters of the application are configured.

[0269] This embodiment provides two configuration methods: one allows users to customize transmission parameters on the configuration interface, and the other is automatic configuration based on customized transmission parameters, such as using the default configuration method. Users can choose the configuration method based on their actual needs, and this embodiment does not impose too many restrictions on this.

[0270] During implementation, you can import the generated application file into the rendering system, and then configure the transmission parameters for the application corresponding to the application file. The specific implementation process is as follows:

[0271] Process a) First access the deployed rendering system (such as the local address http: / / 127.0.0.1:8080) and access the rendering system through a browser.

[0272] As shown in Figures 3A to 3C, this embodiment provides a schematic diagram of an operation interface for accessing a rendering system. Click the "Unpublished Applications" tab on the page of Figure 3A to enter the application upload interface as shown in Figure 3B, click the "Upload Application" button in the upper right corner to enter the file selection prompt box; select the application file to be uploaded, and click the OK button to upload the application file, wherein the application file is required to be packaged in zip format, and all application files are contained in the first-level directory. As shown in Figure 3C, you can also click the search box in the upper left corner to search for uploaded application files, click the delete button below the application operation bar to delete the application file, and the "Publish" function button is below the application operation bar. Click the publish button to publish the application. It should be noted that unpublished applications refer to applications visible to developers, and the application's transmission parameters can be configured, modified, deleted, and other editing operations can be performed, and the application files of the application can also be edited. Published applications, namely "My Applications", refer to applications that developers cannot edit, but the published applications are still visible to developers at this time and invisible to users.

[0273] Process b) configuring transmission parameters through the rendering system;

[0274] During implementation, the uploaded applications are displayed in the "Unpublished Applications" list. Find the unpublished application and click the publish button to enter the configuration interface. The user can enter the configured transmission parameters in the configuration options of the configuration interface. As shown in Figure 4A-Figure 4B, this embodiment provides a schematic diagram of a configuration interface; the user enters the transmission parameters in the configuration interface, for example, the first step is to enter the application name, 20 strings; the second step is to enter the application description, 50 strings; the third step is to select the path of the executable program; the fourth step is to select the streaming method, the default is 2K; the fifth step is to select the startup mode, the default is one-to-one; the sixth step is to fill in the maximum number of concurrency; the seventh step is to select the timeout period, the optional options are "unlimited" and "5 minutes"; the eighth step is to upload the application cover image. After filling in the options, click the confirmation button to submit the application's transmission parameters to the RTC server. After the user confirms the transmission parameters configured, click "My Application" on the configuration interface, and the corresponding application's transmission parameters and other information will be displayed. Click the "Link Button" in the action bar below the app to generate the app's access address. Clicking the Link Button again will copy the corresponding access address. After copying the access address, you need to "list" the app before you can access it. Click the "List" button in the action bar to list the app. The access address copied in the previous step can be used in a browser or a custom playback control plug-in. Listing the app means uploading the app's access address to a service platform (such as an app store) so that users can obtain the access address through the service platform and establish a communication connection with the server.

[0275] In some embodiments, the transmission parameters include a startup mode, where the startup mode is used to indicate the maximum number of clients that can access the same 3D rendering scene of the application; the server is specifically configured to:

[0276] The startup mode of the application is configured so that the same 3D rendering scene of the application can be accessed by multiple clients, but multiple clients cannot operate the same element in the 3D rendering scene at the same time.

[0277] The rendering stream generated in this embodiment can be run in a browser or a custom-developed player, supporting multiple users sharing a viewpoint while allowing one user to operate the system. There's no need to download or install any additional programs; simply import the application's access URL into your browser or custom client player. This reduces hardware configuration dependency and unnecessary time waste while retaining the necessary interactive systems.

[0278] In some embodiments, the server in this embodiment includes a rendering system and an RTC server, the RTC server includes a signaling service, and the signaling service is used to realize the interaction of multiple types of signals between the server and the client; and / or, the client includes a client engine, and the client engine is used to connect to the signaling service and interact with the signaling service for multiple types of signals.

[0279] Since the RTC server on the server side of this embodiment has developed a signaling service, the signaling service can realize the transmission and interaction of various types of signals, thereby not restricting the network environment and signaling, and can provide cross-platform rendering services for clients on different platforms. Through the connection between the signaling service on the server side and the client engine on the client side, it is compatible with multiple types of signaling and realizes the interaction of different types of signaling, so that the rendering system in this embodiment can achieve cross-platform and compatibility effects.

[0280] It should be noted that the signaling service in this embodiment is mainly used for signaling interaction, and the transmission of the rendering stream is not performed through the signaling service, but is performed through the proxy service of the RTC server. This embodiment uses different services to execute the rendering stream and signaling transmission, which can not affect the user's normal viewing experience during the execution of signaling interaction, thereby improving the response speed of the signaling interaction and improving the user's viewing experience.

[0281] In some embodiments, the RTC server includes a signaling service; the signaling service is configured to perform any one or more of the following:

[0282] (1) The signaling service is used to establish a communication connection between the rendering system and the client playback medium;

[0283] During implementation, the signaling service can provide a connection between the rendering system and the client playback media. Optionally, when the client playback media is a browser, the signaling service establishes a connection between the browser and the RTC server, and establishes a connection between the browser and the rendering system through the RTC server; when the client playback media is a player, the signaling service first establishes a connection between the player and the client engine, and then establishes a connection between the player and the RTC server, and establishes a connection between the player and the rendering system through the RTC server.

[0284] (2) The signaling service is used to establish a communication connection between the RTC server and the client playing media;

[0285] During implementation, the signaling service can provide a connection between the RTC server and the client playback media. Optionally, when the client playback media is a browser, the signaling service establishes a connection between the browser and the RTC server; when the client playback media is a player, the signaling service first establishes a connection between the player and the client engine, and then establishes a connection between the player and the RTC server.

[0286] (3) The signaling service is used to establish a communication service between the client and the client playback medium.

[0287] In implementation, the signaling service can provide a connection between the client and the client playback medium. When the client playback medium is a player, the signaling service first establishes a connection between the player and the client engine, and then establishes a connection between the client and the player through the client engine.

[0288] Optionally, the player in this embodiment is encapsulated based on the Flutter language, supports multiple transmission protocols, and can be applied to clients in different network environments. Because it is encapsulated in the Flutter language, it can run in different network environments, and with the help of Flutter's cross-platform features, it has good cross-platform (Android, iOS, Window, Web, etc.) capabilities.

[0289] In practice, when the rendering system publishes and launches an application, specifically uploading the transmission parameters to the RTC server and the access address to the service platform, it then activates the 3D rendering engine and pixel stream plugin, generates the application's rendering stream, establishes a communication connection with the RTC server, and begins sending the rendering stream to the RTC server. The RTC server then connects to the signaling service, indicating that it is ready to accept new connections. Upon startup, the client program imports the access address and automatically connects to the signaling service, establishing a connection between the client and the signaling service.

[0290] Optionally, the signaling service module uses the Node.js development language and the Express framework to implement related functions. The Pixel Stream plugin and the player communicate with the signaling service via a web socket. The signaling service implements information exchange and business logic. The signaling service provides the client with an HTML page containing the player widget and / or control code written in JavaScript.

[0291] In some embodiments, the client playback medium includes a browser; and the server is specifically configured to:

[0292] The client responds to the interaction instruction executed by the user on the browser by sending the interaction instruction to the signaling service; uses the signaling service to send the interaction instruction to the rendering system, and the rendering system responds to the interaction instruction and sends the response result to the browser through the signaling service.

[0293] During implementation, a communication connection is established between the browser and the RTC server through the signaling service, so that the interactive instructions received by the browser are forwarded to the RTC server through the signaling service, and then sent by the RTC server to the rendering system for response and return the response result. The RTC server sends the response result to the browser through the signaling service.

[0294] In some embodiments, the signaling service includes a playback service and a streaming service; the client playback medium includes a browser, and the server is specifically configured to:

[0295] The interactive instruction is sent to the signaling service by using the playback service; and the response result is sent to the browser by using the streaming service.

[0296] In practice, the overall integration process of real-time rendering in the browser is as follows:

[0297] When a user starts the rendering streaming service, that is, when the user obtains the access address, the signaling service will establish a direct link between the client browser and the RTC proxy server. Once the connection is successfully established, the RTC server will directly transmit the rendering stream to the browser, or the client's web playback container will send it directly to the WebRTC proxy through the page's JavaScript environment, and then it will be relayed back to the 3D rendering engine; even after the rendering stream starts playing, the signaling service still maintains the connection with the browser and RTC server so that it can kick the user out of the streaming service when necessary and handle browser-initiated disconnections.

[0298] In some embodiments, the client includes a client engine, the client playback medium includes a player, the player is encapsulated based on the Flutter language, supports multiple transmission protocols, and the player is applied to clients in different network environments;

[0299] The client responds to the interaction instruction executed by the user on the player and sends the interaction instruction to the signaling service through the client engine; the signaling service sends the interaction instruction to the rendering system, and the rendering system responds to the interaction instruction and sends the response result to the player through the signaling service.

[0300] During implementation, a communication connection is established between the player and the client engine through the signaling service, and the interactive instructions received by the player are sent to the signaling service through the client engine. The signaling service forwards the interactive instructions to the rendering system through the RTC server for response. The rendering system returns the response result, and the RTC server sends the response result to the player through the signaling service.

[0301] In some embodiments, the signaling service includes a playback service and a streaming service; the client playback medium includes a player, and the server is specifically configured to:

[0302] The interactive instruction is sent to the client engine by using the playback service, and then sent to the signaling service by the client engine; and the response result is sent to the player by using the streaming service.

[0303] The signaling service in this embodiment is an independent service module of the RTC server, which is mainly responsible for the communication between the pixel stream plug-in and the player, as well as the setting of application transmission parameters and the control of business logic.

[0304] During implementation, the player in this embodiment is a cross-platform playback control plug-in encapsulated in the client Flutter language. This player not only leverages Flutter's cross-platform features to obtain good cross-platform (Android, iOS, Window, Web, etc.) capabilities, but also because the lower layer is connected to a custom rendering stream player and signaling docking component, it realizes docking and display in multiple rendering stream protocols (x264 / RTSPReal (Time Streaming Protocol, Real-time Streaming Protocol) and other formats. At the same time, through the self-developed signaling component (client engine) and signaling service and RTC server docking, it is compatible with various types of signaling in the signaling processing service and RTC service, achieving the same cross-platform and compatibility effect as the browser, while improving performance and reliability, especially in offline scenarios.

[0305] As shown in Figure 6, this embodiment provides a schematic diagram of the interaction between a client and a real-time rendering system. The signaling service includes a playback service (player service) and a streaming service (streamer service). After the signaling service is started, it listens on ports 8888 and 80, as well as HTTP port 80. The player opens the access address of the rendering stream and enters it. The player loads the access address. After receiving the client's request, the player service performs a business verification on the request data. After passing the verification, it calls the process service to open the client engine. After the client engine is successfully started, it sends ICE (Internet Communications Engine) information to the WS listening port 8888. The streamer service processes the received information according to the relevant business logic and sends it to the player. Based on the WEBRTC protocol, a P2P link is established between the player and the client. The signaling related to the user operation received by the player is sent to the client via the player service, and the client's response is sent to the player via the streamer service. At this point, the client and server achieve information sharing.

[0306] In some embodiments, the application file further includes interactive logic code, the rendering stream further includes a UI interactive interface, and the interactive logic code is used to generate the UI interactive interface; after playing and displaying the received rendering stream through the client playback medium, this embodiment further performs the following steps:

[0307] The client playback medium responds to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server; the server performs the corresponding operation according to the operation instruction and sends the UI interaction result after the operation to the client playback medium.

[0308] During implementation, the client itself will not respond to the operation instructions, but will send the operation instructions to the server, and the server will respond and then feed back the response results to the client.

[0309] Optionally, the interaction logic code includes an interaction tool API (Application Programming Interface).

[0310] It should be noted that the rendering flow in this embodiment is the rendering flow of the twin application, that is, the underlying application, and the UI interaction interface is the interaction interface of the upper layer of the application. The design of the UI interaction interface and the generation of the rendering flow can be executed independently, and this embodiment does not impose too many restrictions on this.

[0311] In some embodiments, the server in this embodiment further includes a material warehouse; the server is further configured to:

[0312] The application file is uploaded to a material warehouse, and a corresponding rendering flow is generated according to the application file stored in the material warehouse.

[0313] During implementation, the application files are uploaded to the material warehouse, and developers can directly extract the application files from the material warehouse, thereby eliminating the step of generating application files and directly reusing the application files to generate the corresponding rendering stream, thereby improving rendering efficiency.

[0314] During implementation, when the transmission parameters of the application are configured, the application scenario model and interactive tool API contained in the application file can be uploaded to the associated material warehouse for unified management and distribution. It should be noted that the interactive tool API package in this embodiment is obtained based on the UI interactive interface involved in the application scenario model. When it is necessary to use the configured application access address and interactive tool API, since the application access address has been configured in the rendering system, by entering or importing the access address into the browser and the customized broadcast control plug-in (i.e., player), each platform client can display the rendering stream on the multi-platform client by accessing the access address URL of the rendering stream. As shown in Figure 5, this embodiment provides a schematic diagram of the effect of displaying the rendering stream of an application on the client.

[0315] In some embodiments, the client of this embodiment can not only receive and play the rendering stream transmitted by the server through a playback medium, but also perform image enhancement processing on the rendering stream and further provide feedback to the server on resource parameters, thereby dynamically adjusting the transmission parameters of the application corresponding to the rendering stream. The client of this embodiment can dynamically obtain the operating status of the client device, dynamically adjust various resource parameters, and synchronize feedback with the server, thereby gradually saving server resources, improving client utilization efficiency, reducing costs, and enhancing output flexibility.

[0316] After the server packages and publishes the application files, the client obtains the access address of the application and interacts with the rendering system through the client media. When the client starts the rendering process, that is, when the client receives the rendering stream and plays it, the scheduling and monitoring service can be started synchronously to monitor the resource parameters of the client device in real time to determine whether to perform image enhancement processing on the rendering stream.

[0317] In some embodiments, the client is further configured to:

[0318] Acquire a resource parameter set of the client device, the resource parameter set including at least one resource parameter, the resource parameter being used to represent an operating state of the client device;

[0319] If the resource parameter set meets the first set condition, performing image enhancement processing on the rendering stream to obtain an enhanced video stream;

[0320] The enhanced video stream is played through the client playback medium.

[0321] Optionally, the resource parameter set in this embodiment includes but is not limited to: at least one or more of resource parameters such as streaming mode, network card configuration, CPU resources, GPU resources, video memory resources and network speed.

[0322] Optionally, the first setting condition in this embodiment includes but is not limited to any one or more of the following: the current network speed of the client device is less than or equal to the set network speed threshold; the push streaming mode of the rendering stream received by the client device this time does not meet the preset requirements.

[0323] In implementation, the resource parameter set includes the streaming mode and network speed of the rendering stream; the client is specifically used to:

[0324] Method (1) If it is monitored that the current network speed is less than or equal to the set network speed threshold, image enhancement processing is performed on the rendering stream to obtain an enhanced video stream.

[0325] Method (2) If it is monitored that the push streaming mode of the rendering stream received this time does not meet the preset requirements, image enhancement processing is performed on the rendering stream to obtain an enhanced video stream.

[0326] Optionally, the streaming method includes resolution, transmission protocol of the rendering stream (media stream), etc.

[0327] In an implementation, the streaming method includes the resolution of the rendered stream; and the first set condition includes: the resolution of the rendered stream currently received by the client device is lower than or equal to the resolution of the rendered stream previously received. If the resolution of the rendered stream currently received by the client device is lower than or equal to the resolution of the rendered stream previously received, image enhancement processing is performed on the rendered stream to obtain an enhanced video stream.

[0328] In implementation, the above-mentioned method (1) and method (2) can be implemented in combination, and the specific implementation methods will not be repeated here.

[0329] During implementation, the client will start the scheduling and monitoring service to monitor whether the resource parameters in the client's resource parameter set meet the first set condition. When the first set condition is met, the client's enhanced service is started. Optionally, in this embodiment, the enhanced service can be composed of a super-resolution algorithm (Gan classification) and an API for allocating resources and calling. When it is detected that an independent graphics card and an integrated graphics card coexist, the client's image quality resolution is improved by the computing power of the integrated graphics card and the CPU. Similarly, when it is detected that the independent graphics card has redundant computing power, the resolution of the server's rendering stream can be further reduced through signaling interaction with the server, and the image quality can be improved through the redundant computing power of the client device to complete dynamic allocation.

[0330] During implementation, the client first starts the scheduling and monitoring service, which calls the Nvidia GPU nvmlDeviceGetUtilizationRates(device,&utilization) interface, the network card interface GetAdaptersInfo, and the performance collection interface PdhOpenQuery to obtain resource parameters such as the current network card configuration, resource usage, GPU configuration, and CPU usage. Secondly, the current network speed is judged to determine whether the network speed is ≤X (X is the pre-set network speed threshold). If not, it proves that the network bandwidth is sufficient and the client can choose whether to start the enhancement service for image enhancement processing. If so, the rendering system is notified to reduce the resolution of the rendering stream. Then, the resolution is judged based on the rendering stream obtained by the client to determine whether the resolution of the rendering stream is ≤Y (Y is the resolution before adjustment). If not, the process returns to the network speed judgment stage. If so, the process enters the GPU and other resource parameter judgment stage.

[0331] In some embodiments, the resource parameter set includes a network card configuration. If the resource parameter set satisfies a first setting condition, the client is specifically configured to:

[0332] Step a: determining hardware resources used for image enhancement processing based on whether the network card configuration includes an independent graphics card and an integrated graphics card;

[0333] Optionally, the resource parameter set further includes CPU resources; if the network card configuration includes an independent graphics card and an integrated graphics card, then it is determined that the hardware resources used for image enhancement processing include the integrated graphics card and CPU resources.

[0334] During implementation, it is first determined whether the client device has both an independent graphics card and an integrated graphics card. If both are available, the client's image enhancement processing is performed using the integrated graphics card + CPU mode.

[0335] Optionally, the resource parameter set includes GPU resources and video memory resources; and the client is specifically configured to:

[0336] If the network card configuration does not meet the conditions for performing image enhancement processing in the integrated graphics card, and the GPU resources are greater than or equal to the first threshold, and the video memory resources are greater than or equal to the second threshold, it is determined that the hardware resources used for image enhancement processing include GPU resources.

[0337] During implementation, if the conditions for performing image enhancement processing in the integrated graphics card are not met, it is determined whether the GPU remaining is ≥U (U is the first threshold that can be dynamically set according to the application volume), and at the same time, it is determined whether the video memory remaining is ≥T (T is the second threshold that can be dynamically set according to the application volume). If either is no, the image enhancement processing is turned off. If both are yes, the enhancement level of the image enhancement processing is adjusted according to the GPU resources. After the adjustment, it is determined whether a loop detection is required. If not, the current process is terminated. If required, the loop detection stage is entered.

[0338] In some embodiments, the resource parameter set further includes GPU resources; if the resource parameter set satisfies a first setting condition, the client is further configured to:

[0339] The enhancement level of the image enhancement algorithm is adjusted according to GPU resources, and the adjusted image enhancement algorithm is used to perform image enhancement processing on the rendering stream; the enhancement level is used to represent the intensity of image enhancement.

[0340] During implementation, the enhancement level of the client's image enhancement algorithm can be adjusted according to the GPU resources of the client device monitored by the client. After the GPU resources are monitored, the monitoring results are sent to the rendering system through the RTC server. The rendering system adjusts the resolution of the rendering stream based on the monitoring results and sends the adjusted rendering stream to the client through the RTC server for playback media display.

[0341] Optionally, the image enhancement algorithm includes resolution enhancement, and / or image detail enhancement such as a super-resolution algorithm.

[0342] Step b: using the hardware resources to perform image enhancement processing on the rendering stream to obtain an enhanced video stream.

[0343] In some embodiments, if the resource parameter set satisfies a first setting condition, the client is specifically configured to:

[0344] The rendering stream is stream-decoded to obtain a decoded video stream, and image enhancement processing is performed on the decoded video stream to obtain an enhanced video stream.

[0345] During implementation, when it is determined that the client needs to perform image enhancement processing, it is necessary to first perform stream decoding on the rendering stream to obtain the decoded video stream, and then perform image enhancement processing to obtain the enhanced video stream.

[0346] In some embodiments, the client is further configured to:

[0347] If it is detected that the resource parameter set meets the first setting condition, the server is notified to adjust the transmission parameters of the application; or,

[0348] If it is monitored that the resource parameter set meets the first setting condition and the resource parameter set meets the second setting condition, the server is notified to adjust the transmission parameters of the application; optionally, the transmission parameters include the resolution of the rendering stream.

[0349] The second setting condition includes: the resource parameters in the resource parameter set reach a peak value, and one resource parameter corresponds to one peak value; or the network speed in the resource parameter set is less than or equal to the network speed threshold.

[0350] During implementation, the client starts the scheduling and monitoring service, and when the resource parameter set meets the first set condition (such as determining that the current network speed is less than or equal to the set network speed threshold), it first decodes the rendering stream and determines whether to start image enhancement processing. If the decoding is successful and the scheduling and monitoring service feedback indicates that the conditions for starting image enhancement processing are met (such as monitoring that the resolution of the rendering stream received this time is lower than the resolution of the rendering stream received last time), then image enhancement processing is started. If it is determined that image enhancement processing does not need to be started, the received rendering stream is directly sent to the player for display.

[0351] This embodiment can determine the enhancement level of the image enhancement algorithm based on the current hardware occupancy when it detects that the network conditions of the current client device are insufficient to support the rendering flow of the application, or that the server resources are tight and the client needs to perform collaborative rendering, while releasing some server resources. For example, the enhancement level of resolution enhancement, taking into account the actual usage and dynamic management of resources, the enhancement level of the client's image enhancement algorithm is usually enhanced to the target resolution rather than the highest resolution. After enhancing to the corresponding resolution, the occupancy of each hardware will be synchronously checked in a cyclic manner. When the resource parameters reach the peak value, timely feedback will be sent to the server for information exchange, and the resolution of the rendering flow output by the server will be adjusted in the reverse direction. At the same time, the client's image enhancement algorithm will be temporarily reduced or turned off, so that it can achieve the goal of both positive and negative control.

[0352] As shown in FIG7 , this embodiment provides an implementation process of dynamic client allocation, which is specifically as follows:

[0353] Step 700: Enable environmental monitoring and obtain a resource parameter set of the client device;

[0354] Step 701: Determine whether the current network speed of the client device is less than or equal to the set network speed threshold. If so, execute step 702; otherwise, execute step 701.

[0355] Step 702: Notify the server to reduce the resolution of the rendering stream of the application;

[0356] Step 703: Determine whether the resolution of the rendering stream received this time by the client is lower than or equal to the resolution of the rendering stream received last time. If so, execute step 704; otherwise, execute step 701.

[0357] Step 704: Determine whether the network card configuration includes a discrete graphics card and an integrated graphics card. If yes, proceed to step 705; otherwise, proceed to step 706.

[0358] Step 705: Perform image enhancement processing on the rendering stream using the integrated graphics card and CPU resources to obtain an enhanced video stream;

[0359] Step 706: Determine whether the GPU resources are greater than or equal to the first threshold, and whether the video memory resources are greater than or equal to the second threshold. If both are true, execute step 707; otherwise, execute step 710.

[0360] Step 707: Start the image enhancement processing service, adjust the enhancement level of the image enhancement algorithm according to the GPU resources, and use the adjusted image enhancement algorithm to perform image enhancement processing on the rendering stream.

[0361] Step 708: Determine whether to exit the client. If yes, execute step 709; otherwise, execute step 701.

[0362] Step 709, end the process.

[0363] Step 710: Close the image enhancement processing service.

[0364] During implementation, when the scheduling service starts the image enhancement service of the client device, the image enhancement service of the client device will run a process by itself, as shown in FIG8 . This embodiment provides an implementation process of the image enhancement service, which is specifically as follows:

[0365] Step 800: Start the image enhancement service;

[0366] Step 801: Decode the rendered stream to obtain a decoded video stream.

[0367] Step 802: Determine whether to enable the image enhancement algorithm. If yes, proceed to step 803; otherwise, proceed to step 804.

[0368] For example, determine whether to enable the super-resolution algorithm.

[0369] Step 803: Perform image enhancement processing on the rendering stream using an image enhancement algorithm;

[0370] For example, the rendering stream can be format converted and super-resolution processed.

[0371] Step 804: Use a player to play and display the rendered stream.

[0372] This embodiment, based on a 3D rendering engine, generates a 3D deployment package (application file) for the application, imports it into the rendering system, configures transmission parameters through the rendering system's configuration interface, generates a URL for the rendering stream, and quickly distributes the URL to the playback media (container) of various client devices (terminals). On the client side, the scheduling and monitoring module dynamically obtains the client device status, activates the client-side image enhancement service in real time, and dynamically adjusts various transmission parameters and synchronizes feedback with the server. This can gradually save server resources, improve client utilization efficiency, reduce costs, and enhance output flexibility.

[0373] Based on the same inventive concept, the embodiment of the present disclosure further provides a real-time rendering method, as shown in FIG9 . The specific implementation process of the method is as follows:

[0374] Step 900: In response to a first operation by a user on an application in a service platform displayed on a client device, the client obtains an access address of the application;

[0375] Optionally, the first operation includes but is not limited to click, long press, and other copy operations, which are used to copy the access address to the clipboard.

[0376] Step 901: In response to a second operation in which a user inputs the access address to a client to play a medium, a communication connection is established with a server according to the access address.

[0377] Optionally, the second operation includes but is not limited to the user pasting or importing the access address into the address bar of the client playing the media to access the server.

[0378] Step 902: The client receives the rendering stream associated with the access address sent by the server, and plays and displays the received rendering stream through the client playback medium. The rendering stream is used to display the three-dimensional scene model of the application in the form of a video stream.

[0379] As an optional implementation, the client includes a client engine, the client playback medium includes a player, the player is encapsulated based on the Flutter language, supports multiple transmission protocols, and the player is applied to clients in different network environments; the method also includes:

[0380] The client responds to the interactive instruction executed by the user on the player, and sends the interactive instruction to the server through the client engine, so that the server responds to the interactive instruction and sends the response result to the player;

[0381] The player displays the received response result.

[0382] As an optional implementation manner, the client playback medium includes a browser; and the method further includes:

[0383] The client sends the interactive instruction to the server in response to the interactive instruction executed by the user on the browser, so that the server responds to the interactive instruction and sends the response result to the browser;

[0384] The browser displays the received response result.

[0385] As an optional implementation, the rendering flow further includes a UI interaction interface, and the method further includes:

[0386] The client playback medium responds to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server, so that the server performs the corresponding operation according to the operation instruction and sends the UI interaction result after the operation to the client playback medium;

[0387] The client plays the media to display the received UI interaction results.

[0388] As shown in Figures 10A and 10B, this embodiment also provides an operation interface for displaying a rendered stream on a client-side media player. Figure 10A shows the operation interface for displaying a rendered stream on a browser. After a user enters an access address in the address bar, the browser establishes a connection with the server and receives the rendering stream associated with the access address sent by the server, thereby displaying the rendering stream on the browser. Figure 10B shows the operation interface for displaying a rendered stream on a player. After a user enters an access address in the player's address bar, the player establishes a connection with the server and receives the rendering stream associated with the access address sent by the server, thereby displaying the rendering stream on the player.

[0389] Based on the same inventive concept, the embodiment of the present disclosure further provides a real-time rendering method, as shown in FIG11 . The specific implementation process of the method is as follows:

[0390] Step 1100: Generate an application file and access address for the application using the server, and configure transmission parameters for the application. The application file includes a 3D scene model and logic code for the application. The transmission parameters represent parameters required for transmitting the application from the server to the client for display.

[0391] Step 1101: Generate a rendering stream based on the application file using a server, wherein the rendering stream is associated with the access address, and the rendering stream is used to display a three-dimensional scene model of the application in a video stream manner;

[0392] Step 1102: Use the client to obtain the access address of the application, establish a communication connection with the server according to the access address, receive the rendering stream of the server according to the transmission parameters, and play and display the received rendering stream through the client playback medium.

[0393] As an optional implementation, the server includes a rendering system and a real-time communication RTC server, and the RTC server is used to establish a communication connection between the rendering system and the client based on RTC technology;

[0394] Generate an application file of the application using the rendering system, configure transmission parameters of the application, and upload the transmission parameters to the RTC server;

[0395] The RTC server is used to generate an access address for the application.

[0396] As an optional implementation manner, after the server generates the access address of the application using the RTC server, the server further includes:

[0397] The server uploads the access address to the service platform, generates a rendering stream according to the application file, and sends the rendering stream to the RTC server;

[0398] The client obtains the access address of the application from the service platform, establishes a communication connection with the RTC server according to the access address, and receives the rendering stream sent by the RTC server.

[0399] As an optional implementation, the rendering system includes a three-dimensional rendering engine and a pixel stream plug-in;

[0400] generating an application file for an application using the three-dimensional rendering engine;

[0401] The three-dimensional rendering engine is used to generate a pixel stream of the application based on the application file, and the pixel stream plug-in is used to convert the pixel stream into a rendering stream.

[0402] As an optional implementation, the rendering system includes a configuration interface;

[0403] Displaying configuration options of the application on the configuration interface; configuring transmission parameters of the application in response to transmission parameters input by the user in the configuration options on the configuration interface; or,

[0404] Based on the customized transmission parameters, the transmission parameters of the application are configured.

[0405] As an optional implementation,

[0406] The RTC server includes a signaling service, and the signaling service is used to implement the interaction of multiple types of signaling between the server and the client; and / or,

[0407] The client includes a client engine, and the client engine is used to connect to the signaling service and interact with the signaling service for multiple types of signaling.

[0408] As an optional implementation, the RTC server includes a signaling service;

[0409] The signaling service is used to establish a communication connection between the rendering system and the client playback medium; or,

[0410] The signaling service is used to establish a communication connection between the RTC server and the client playing media; or,

[0411] The signaling service is used to establish a communication service between the client and the client playing media.

[0412] As an optional implementation, the client includes a client engine, the client playback medium includes a player, the player is encapsulated based on the Flutter language, supports multiple transmission protocols, and the player is applied to clients in different network environments;

[0413] The client responds to an interaction instruction executed by a user on the player, sending the interaction instruction to the signaling service via the client engine;

[0414] The signaling service sends the interaction instruction to the rendering system, and the rendering system responds to the interaction instruction and sends the response result to the player through the signaling service.

[0415] As an optional implementation, the client playback medium includes a browser;

[0416] The client sends the interaction instruction to the signaling service in response to the interaction instruction executed by the user on the browser;

[0417] The interaction instruction is sent to the rendering system by using the signaling service. The rendering system responds to the interaction instruction and sends the response result to the browser through the signaling service.

[0418] As an optional implementation, the signaling service includes a playback service and a streaming service; the client playback medium includes a player:

[0419] Using the playback service to send the interaction instruction to the client engine, and then sending it to the signaling service through the client engine; using the streaming service to send the response result to the player; or,

[0420] The client playback medium includes a browser:

[0421] The interactive instruction is sent to the signaling service by using the playback service; and the response result is sent to the browser by using the streaming service.

[0422] As an optional implementation, the application file further includes interactive logic code, the rendering flow further includes a UI interactive interface, and the interactive logic code is used to generate the UI interactive interface;

[0423] After playing and displaying the received rendering stream through the client playback medium, it also includes:

[0424] The client plays the media in response to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server;

[0425] The server performs the corresponding operation according to the operation instruction, and sends the UI interaction result after the operation to the client to play the media.

[0426] As an optional implementation, the server further includes a material warehouse;

[0427] The application file is uploaded to a material warehouse, and a corresponding rendering flow is generated according to the application file stored in the material warehouse.

[0428] As an optional implementation, the transmission parameter includes a startup mode, where the startup mode is used to indicate the maximum number of clients accessing the same 3D rendering scene of the application;

[0429] The startup mode of the application is configured so that the same 3D rendering scene of the application can be accessed by multiple clients, but multiple clients cannot operate the same element in the 3D rendering scene at the same time.

[0430] As an optional implementation, the method further includes:

[0431] Acquire a resource parameter set of the client device, the resource parameter set including at least one resource parameter, the resource parameter being used to represent an operating state of the client device;

[0432] If the resource parameter set meets the first set condition, performing image enhancement processing on the rendering stream to obtain an enhanced video stream;

[0433] The enhanced video stream is played and displayed through the client playback medium.

[0434] As an optional implementation, the resource parameter set includes a streaming mode and a network speed for the rendering stream; and the first setting condition includes:

[0435] The current network speed of the client device is less than or equal to the set network speed threshold; and / or,

[0436] The streaming mode of the rendering stream received by the client device does not meet the preset requirements.

[0437] As an optional implementation manner, the streaming mode includes a resolution of the rendering stream; and the first setting condition includes:

[0438] The resolution of the rendering stream received this time by the client device is lower than or equal to the resolution of the rendering stream received last time.

[0439] As an optional implementation, the resource parameter set includes a network card configuration. If the resource parameter set meets the first setting condition, the hardware resources used for image enhancement processing are determined based on whether the network card configuration includes an independent graphics card and an integrated graphics card; the rendering stream is subjected to image enhancement processing using the hardware resources to obtain an enhanced video stream.

[0440] As an optional implementation, the resource parameter set further includes CPU resources;

[0441] If the network card configuration includes an independent graphics card and an integrated graphics card, it is determined that the hardware resources used for the image enhancement process include the integrated graphics card and CPU resources.

[0442] As an optional implementation, the resource parameter set includes GPU resources and video memory resources;

[0443] If the network card configuration does not meet the conditions for performing image enhancement processing in the integrated graphics card, and the GPU resources are greater than or equal to the first threshold, and the video memory resources are greater than or equal to the second threshold, it is determined that the hardware resources used for image enhancement processing include GPU resources.

[0444] As an optional implementation manner, the resource parameter set further includes GPU resources; if the resource parameter set meets the first setting condition, it further includes:

[0445] The client adjusts the enhancement level of the image enhancement algorithm according to GPU resources, and performs image enhancement processing on the rendering stream using the adjusted image enhancement algorithm; the enhancement level is used to represent the intensity of image enhancement.

[0446] As an optional implementation, it also includes:

[0447] If the client detects that the resource parameter set meets the first set condition, the client notifies the server to adjust the transmission parameters of the application; or

[0448] If the client monitors that the resource parameter set meets the first setting condition and the resource parameter set meets the second setting condition, the server is notified to adjust the transmission parameters of the application; the second setting condition includes: the resource parameters in the resource parameter set reach a peak value, and one resource parameter corresponds to a peak value; or, the network speed in the resource parameter set is less than or equal to the network speed threshold.

[0449] As an optional implementation, the transmission parameter includes a resolution of the rendering stream.

[0450] As an optional implementation, if the resource parameter set meets the first set condition, the rendering stream is stream-decoded to obtain a decoded video stream, and image enhancement processing is performed on the decoded video stream to obtain an enhanced video stream.

[0451] As an optional implementation, the client is deployed on a client device in a different network environment, where the network environment is used to represent an operating system or network service of the client device; and / or,

[0452] The server is deployed on a cloud server and / or a local area network server.

[0453] Based on the same inventive concept, the embodiment of the present disclosure also provides a server-side device. Since the server-side device is the server in the system in the embodiment of the present disclosure, and the principle of solving the problem by the server-side device is similar to that of the system, the implementation of the server-side device can refer to the implementation of the system, and the repeated parts will not be repeated.

[0454] As shown in FIG12 , the server device includes a processor 1200 and a memory 1201 . The memory 1201 is used to store programs executable by the processor 1200 . The processor 1200 is used to read the programs in the memory 1201 and perform the following steps:

[0455] Generate an application file and access address for the application, and configure transmission parameters for the application, wherein the application file includes a three-dimensional scene model and logic code of the application, and the transmission parameters are used to represent parameters required for transmitting the application to a client device for display;

[0456] generating a rendering stream according to the application file, wherein the rendering stream is associated with the access address, and the rendering stream is used to display a three-dimensional scene model of the application in a video stream;

[0457] The rendering stream is sent to a client device based on the transmission parameters, wherein the client device includes a device that establishes a communication connection with a server device based on the access address.

[0458] As an optional implementation, the server device includes a rendering system and a real-time communication RTC server, and the RTC server is used to establish a communication connection between the rendering system and the client device based on RTC technology; the processor 1200 is specifically used to:

[0459] Generate an application file of the application using the rendering system, configure transmission parameters of the application, and upload the transmission parameters to the RTC server;

[0460] The RTC server is used to generate an access address for the application.

[0461] As an optional implementation manner, after the server device generates the access address of the application using the RTC server, the processor 1200 is further configured to:

[0462] Uploading the access address to the service platform, generating a rendering stream based on the application file, and sending the rendering stream to the RTC server so that the client device can obtain the access address of the application from the service platform and establish a communication connection with the RTC server based on the access address;

[0463] The rendering stream is sent to the client device through the RTC server.

[0464] As an optional implementation, the rendering system includes a 3D rendering engine and a pixel stream plug-in; the processor 1200 is specifically configured to:

[0465] generating an application file for an application using the three-dimensional rendering engine;

[0466] The three-dimensional rendering engine is used to generate a pixel stream of the application based on the application file, and the pixel stream plug-in is used to convert the pixel stream into a rendering stream.

[0467] As an optional implementation, the rendering system includes a configuration interface; the processor 1200 is specifically configured to:

[0468] Displaying configuration options of the application on the configuration interface; configuring transmission parameters of the application in response to transmission parameters input by the user in the configuration options on the configuration interface; or,

[0469] Based on the customized transmission parameters, the transmission parameters of the application are configured.

[0470] As an optional implementation, the RTC server includes a signaling service, and the signaling service is used to implement the interaction of multiple types of signaling between the server device and the client device.

[0471] As an optional implementation, the RTC server includes a signaling service;

[0472] The signaling service is used to establish a communication connection between the rendering system and the client playback medium; or,

[0473] The signaling service is used to establish a communication connection between the RTC server and the client playing media; or,

[0474] The signaling service is used to establish a communication service between the client and the client playing media.

[0475] As an optional implementation manner, the processor 1200 is specifically configured to:

[0476] receiving an interaction instruction sent by the client engine through the signaling service, and sending the interaction instruction to the rendering system;

[0477] The rendering system responds to the interaction instruction, and sends the response result to the client playback medium through the signaling service. The client playback medium includes a player. The player is encapsulated based on the Flutter language and supports multiple transmission protocols. The player is applied to clients in different network environments.

[0478] As an optional implementation manner, the processor 1200 is specifically configured to:

[0479] receiving, via the signaling service, an interaction instruction sent by the client playing the medium, and sending the interaction instruction to the rendering system;

[0480] The signaling service is used to send the interaction instruction to the rendering system. The rendering system responds to the interaction instruction and sends the response result to the client playback medium through the signaling service. The client playback medium includes a browser.

[0481] As an optional implementation manner, the signaling service includes a playback service and a streaming service; the processor 1200 is specifically configured to:

[0482] Using the playback service to send the interaction instruction to the client engine, and then to the signaling service through the client engine; using the streaming service to send the response result to the client playback medium, wherein the client playback medium includes a player; or

[0483] The interactive instruction is sent to the signaling service by using the playback service; and the response result is sent to the client playback medium by using the streaming service, where the client playback medium includes a browser.

[0484] As an optional implementation, the application file further includes interactive logic code, the rendering flow further includes a UI interactive interface, and the interactive logic code is used to generate the UI interactive interface;

[0485] After playing and displaying the received rendering stream through the client playback medium, the processor 1200 is further configured to:

[0486] Receive operation instructions sent by the client to play the media;

[0487] Execute the corresponding operation according to the operation instruction, and send the UI interaction result after the operation to the client to play the medium.

[0488] As an optional implementation manner, the server device further includes a material warehouse; and the processor 1200 is further specifically configured to:

[0489] The application file is uploaded to a material warehouse, and a corresponding rendering flow is generated according to the application file stored in the material warehouse.

[0490] As an optional implementation, the transmission parameter includes a startup mode, where the startup mode is used to indicate the maximum number of clients accessing the same 3D rendering scene of the application; the processor 1200 is specifically configured to:

[0491] The startup mode of the application is configured so that the same 3D rendering scene of the application can be accessed by multiple clients, but multiple clients cannot operate the same element in the 3D rendering scene at the same time.

[0492] Based on the same inventive concept, the embodiment of the present disclosure also provides a client device. Since the client device is the server in the system in the embodiment of the present disclosure, and the principle of solving the problem by the client device is similar to that of the system, the implementation of the client device can refer to the implementation of the system, and the repeated parts will not be repeated.

[0493] As shown in FIG13 , the client device includes a processor 1300 and a memory 1301 . The memory 1301 is used to store programs executable by the processor 1300 . The processor 1300 is used to read the programs in the memory 1301 and perform the following steps:

[0494] Obtaining an access address for the application, and establishing a communication connection with a server based on the access address; associating the access address with a rendering stream for the application, wherein the rendering stream is used to display a three-dimensional scene model of the application via a video stream;

[0495] Receive the rendering stream associated with the access address sent by the server device, and play and display the received rendering stream through the client playback medium.

[0496] As an optional implementation manner, the processor 1300 is further configured to:

[0497] Acquire a resource parameter set, the resource parameter set including at least one resource parameter, the resource parameter being used to represent an operating state of the client device;

[0498] If the resource parameter set meets the first set condition, performing image enhancement processing on the rendering stream to obtain an enhanced video stream;

[0499] The enhanced video stream is played and displayed through the client playback medium.

[0500] As an optional implementation, the resource parameter set includes a streaming mode and a network speed for the rendering stream; and the first setting condition includes:

[0501] The current network speed of the client device is less than or equal to the set network speed threshold; and / or,

[0502] The streaming mode of the rendering stream received by the client device does not meet the preset requirements.

[0503] As an optional implementation manner, the streaming mode includes a resolution of the rendering stream; and the first setting condition includes:

[0504] The resolution of the rendering stream received this time by the client device is lower than or equal to the resolution of the rendering stream received last time.

[0505] As an optional implementation manner, the resource parameter set includes a network card configuration. If the resource parameter set meets the first setting condition, the processor 1300 is specifically configured to:

[0506] Determining hardware resources used for image enhancement processing based on whether the network card configuration includes a discrete graphics card and an integrated graphics card;

[0507] The hardware resources are used to perform image enhancement processing on the rendering stream to obtain an enhanced video stream.

[0508] As an optional implementation manner, the resource parameter set further includes CPU resources; and the processor 1300 is specifically configured to:

[0509] If the network card configuration includes an independent graphics card and an integrated graphics card, it is determined that the hardware resources used for the image enhancement process include the integrated graphics card and CPU resources.

[0510] As an optional implementation manner, the resource parameter set also includes GPU resources and video memory resources; the processor 1300 is specifically configured to:

[0511] If the network card configuration does not meet the conditions for performing image enhancement processing in the integrated graphics card, and the GPU resources are greater than or equal to the first threshold, and the video memory resources are greater than or equal to the second threshold, it is determined that the hardware resources used for image enhancement processing include GPU resources.

[0512] As an optional implementation manner, the resource parameter set further includes GPU resources; if the resource parameter set meets the first setting condition, the processor 1300 is further configured to:

[0513] The enhancement level of the image enhancement algorithm is adjusted according to GPU resources, and the adjusted image enhancement algorithm is used to perform image enhancement processing on the rendering stream; the enhancement level is used to represent the intensity of image enhancement.

[0514] As an optional implementation manner, the processor 1300 is further configured to:

[0515] If it is detected that the resource parameter set meets the first setting condition, the server is notified to adjust the transmission parameters of the application; or,

[0516] If it is monitored that the resource parameter set meets the first setting condition and the resource parameter set meets the second setting condition, the server is notified to adjust the transmission parameters of the application; the second setting condition includes: the resource parameters in the resource parameter set reach a peak value, and one resource parameter corresponds to a peak value; or, the network speed in the resource parameter set is less than or equal to the network speed threshold.

[0517] As an optional implementation, the transmission parameter includes a resolution of the rendering stream.

[0518] As an optional implementation manner, if the resource parameter set meets the first setting condition, the processor 1300 is specifically configured to:

[0519] The rendering stream is stream-decoded to obtain a decoded video stream, and image enhancement processing is performed on the decoded video stream to obtain an enhanced video stream.

[0520] As an optional implementation, the client is deployed on a client device in a different network environment, where the network environment is used to represent an operating system or network service of the client device; and / or,

[0521] The server is deployed on a cloud server and / or a local area network server.

[0522] Based on the same inventive concept, embodiments of the present disclosure provide a computer storage medium comprising computer program code. When executed on a computer, the computer program code causes the computer to execute any of the real-time rendering methods discussed above. Because the principles underlying the problems solved by the computer storage medium are similar to those of the real-time rendering method, the implementation of the computer storage medium can be referenced to the implementation of the method, and any repetitions will be omitted.

[0523] In a specific implementation process, computer storage media may include: Universal Serial Bus Flash Drive (USB), mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0524] Based on the same inventive concept, embodiments of the present disclosure further provide a computer program product comprising computer program code that, when executed on a computer, causes the computer to perform any of the real-time rendering methods discussed above. Because the principles underlying the problems solved by the computer program products are similar to those of the real-time rendering methods, the implementation of the computer program products can be referenced to the implementation of the methods, and any repetitions will not be repeated.

[0525] The computer program product can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0526] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0527] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0528] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0529] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0530] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A real-time rendering system, wherein: The system includes a server and a client, where: The server generates an application file and access address for the application, and configures transmission parameters for the application. The application file includes a three-dimensional scene model and logic code for the application. The transmission parameters are used to represent parameters required for the application to be transmitted from the server to the client for display. The server generates a rendering stream according to the application file, the rendering stream is associated with the access address, and the rendering stream is used to display the three-dimensional scene model of the application in a video stream; The client obtains the access address of the application, establishes a communication connection with the server according to the access address, receives the rendering stream sent by the server based on the transmission parameters, and plays and displays the received rendering stream through the client playback medium.

2. The system according to claim 1, wherein: The server includes a rendering system and a real-time communication server, and the real-time communication server is used to establish a communication connection between the rendering system and the client based on real-time communication technology; the server is specifically used to: generating an application file of the application using the rendering system, configuring transmission parameters of the application, and uploading the transmission parameters to the real-time communication server; The real-time communication server is used to generate an access address for the application.

3. The system according to claim 2, wherein: After the server generates the access address of the application using the real-time communication server, the server further includes: The server uploads the access address to the service platform, generates a rendering stream according to the application file, and sends the rendering stream to the real-time communication server; The client obtains the access address of the application from the service platform, establishes a communication connection with the real-time communication server according to the access address, and receives a rendering stream sent by the real-time communication server.

4. The system according to claim 2, wherein: The rendering system includes a 3D rendering engine and a pixel stream plug-in; the server is specifically used to: generating an application file for an application using the three-dimensional rendering engine; The three-dimensional rendering engine is used to generate a pixel stream of the application based on the application file, and the pixel stream plug-in is used to convert the pixel stream into a rendering stream.

5. The system according to claim 2, wherein: The rendering system includes a configuration interface; the server is specifically used to: Displaying configuration options of the application on the configuration interface; configuring transmission parameters of the application in response to transmission parameters input by the user in the configuration options on the configuration interface; or, Based on the customized transmission parameters, the transmission parameters of the application are configured.

6. The system according to claim 2, wherein: The real-time communication server includes a signaling service, and the signaling service is used to implement the interaction of multiple types of signaling between the server and the client; and / or, The client includes a client engine, and the client engine is used to connect to the signaling service and interact with the signaling service for multiple types of signaling.

7. The system according to claim 6, wherein: The real-time communication server includes a signaling service; The signaling service is used to establish a communication connection between the rendering system and the client playback medium; or, The signaling service is used to establish a communication connection between the real-time communication server and the client playing the medium; or The signaling service is used to establish a communication service between the client and the client playing media.

8. The system according to claim 6, wherein: The client includes a client engine, and the client playback medium includes a player. The player is encapsulated based on the Flutter language and supports multiple transmission protocols. The player is applied to clients in different network environments; The client responds to an interaction instruction executed by a user on the player, sending the interaction instruction to the signaling service via the client engine; The signaling service sends the interaction instruction to the rendering system, and the rendering system responds to the interaction instruction and sends the response result to the player through the signaling service.

9. The system according to claim 6, wherein: The client playback medium includes a browser; the server is specifically used for: The client sends the interaction instruction to the signaling service in response to the interaction instruction executed by the user on the browser; The interaction instruction is sent to the rendering system by using the signaling service. The rendering system responds to the interaction instruction and sends the response result to the browser through the signaling service.

10. The system according to claim 8 or 9, wherein: The signaling service includes a playback service and a streaming service; The client playback medium includes a player, and the server is specifically configured to: Using the playback service to send the interaction instruction to the client engine, and then sending it to the signaling service through the client engine; using the streaming service to send the response result to the player; or, The client playback medium includes a browser, and the server is specifically configured to: The interactive instruction is sent to the signaling service by using the playback service; and the response result is sent to the browser by using the streaming service.

11. The system according to claim 1, wherein: The application file further includes an interactive logic code, the rendering flow further includes a UI interactive interface, and the interactive logic code is used to generate the UI interactive interface; After playing and displaying the received rendering stream through the client playback medium, it also includes: The client plays the media in response to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server; The server performs the corresponding operation according to the operation instruction, and sends the UI interaction result after the operation to the client to play the media.

12. The system according to claim 1, wherein: The server also includes a material warehouse; the server is also used to: The application file is uploaded to a material warehouse, and a corresponding rendering flow is generated according to the application file stored in the material warehouse.

13. The system according to claim 1, wherein: The transmission parameters include a startup mode, which is used to indicate the maximum number of clients that can access the same 3D rendering scene of the application; the server is specifically used to: The startup mode of the application is configured so that the same 3D rendering scene of the application can be accessed by multiple clients, but multiple clients cannot operate the same element in the 3D rendering scene at the same time.

14. The system according to claim 1, wherein: The client is further specifically used for: Acquire a resource parameter set of the client device, the resource parameter set including at least one resource parameter, the resource parameter being used to represent an operating state of the client device; If the resource parameter set meets the first set condition, performing image enhancement processing on the rendering stream to obtain an enhanced video stream; The enhanced video stream is played and displayed through the client playback medium.

15. The system according to claim 14, wherein: The resource parameter set includes the streaming mode and network speed of the rendering stream; the first setting condition includes: The current network speed of the client device is less than or equal to the set network speed threshold; and / or, The streaming mode of the rendering stream received by the client device does not meet the preset requirements.

16. The system according to claim 15, wherein: The streaming mode includes the resolution of the rendering stream; The first setting condition includes: The resolution of the rendering stream received this time by the client device is lower than or equal to the resolution of the rendering stream received last time.

17. The system of claim 14, wherein: The resource parameter set includes a network card configuration. If the resource parameter set satisfies a first setting condition, the client is specifically configured to: Determining hardware resources used for image enhancement processing based on whether the network card configuration includes a discrete graphics card and an integrated graphics card; The hardware resources are used to perform image enhancement processing on the rendering stream to obtain an enhanced video stream.

18. The system according to claim 17, wherein: The resource parameter set also includes CPU resources; the client is specifically configured to: If the network card configuration includes an independent graphics card and an integrated graphics card, it is determined that the hardware resources used for the image enhancement process include the integrated graphics card and CPU resources.

19. The system according to claim 17, wherein: The resource parameter set includes GPU resources and video memory resources; the client is specifically used to: If the network card configuration does not meet the conditions for performing image enhancement processing in the integrated graphics card, and the GPU resources are greater than or equal to the first threshold, and the video memory resources are greater than or equal to the second threshold, it is determined that the hardware resources used for image enhancement processing include GPU resources.

20. The system of claim 14, wherein: The resource parameter set also includes GPU resources; if the resource parameter set meets the first setting condition, the client is further configured to: The enhancement level of the image enhancement algorithm is adjusted according to GPU resources, and the adjusted image enhancement algorithm is used to perform image enhancement processing on the rendering stream; the enhancement level is used to represent the intensity of image enhancement.

21. The system of claim 15, wherein: The client is further specifically used for: If it is detected that the resource parameter set meets the first setting condition, the server is notified to adjust the transmission parameters of the application; or, If it is monitored that the resource parameter set meets the first setting condition and the resource parameter set meets the second setting condition, notifying the server to adjust the transmission parameters of the application; The second setting condition includes: a resource parameter in the resource parameter set reaches a peak value, and one resource parameter corresponds to one peak value; Or, the network speed in the resource parameter set is less than or equal to the network speed threshold.

22. The system of claim 21, wherein: The transmission parameters include the resolution of the rendering stream.

23. The system of claim 14, wherein: If the resource parameter set satisfies the first setting condition, the client is specifically configured to: The rendering stream is stream-decoded to obtain a decoded video stream, and image enhancement processing is performed on the decoded video stream to obtain an enhanced video stream.

24. The system according to any one of claims 1 to 9 and 11 to 23, wherein: The client is deployed on a client device in a different network environment, where the network environment is used to represent an operating system or network service of the client device; and / or, The server is deployed on a cloud server and / or a local area network server.

25. A real-time rendering method, wherein: The method includes: In response to a first operation by a user on an application in the service platform displayed on the client device, the client obtains an access address of the application; In response to a second operation in which the user inputs the access address into the client to play the media, establishing a communication connection with the server according to the access address; The client receives the rendering stream associated with the access address sent by the server, and plays and displays the received rendering stream through the client playback medium. The rendering stream is used to display the three-dimensional scene model of the application in the form of a video stream.

26. The method according to claim 25, wherein The client includes a client engine, and the client playback medium includes a player. The player is encapsulated based on the Flutter language and supports multiple transmission protocols. The player is applied to clients in different network environments; The method further includes: The client responds to the interactive instruction executed by the user on the player, and sends the interactive instruction to the server through the client engine, so that the server responds to the interactive instruction and sends the response result to the player; The player displays the received response result.

27. The method according to claim 25, wherein The client playback medium includes a browser; the method further includes: The client sends the interactive instruction to the server in response to the interactive instruction executed by the user on the browser, so that the server responds to the interactive instruction and sends the response result to the browser; The browser displays the received response result.

28. The method according to claim 25, wherein The rendering flow also includes a UI interaction interface, and the method further includes: The client playback medium responds to the user's operation instruction on the UI interaction interface and sends the operation instruction to the server, so that the server performs the corresponding operation according to the operation instruction and sends the UI interaction result after the operation to the client playback medium; The client plays the media to display the received UI interaction results.

29. A real-time rendering method, wherein: The method includes: Using the server to generate an application file and access address for the application, and configuring transmission parameters for the application, wherein the application file includes a three-dimensional scene model and logic code of the application, and the transmission parameters are used to represent parameters required for the application to be transmitted from the server to the client for display; Using a server to generate a rendering stream according to the application file, the rendering stream is associated with the access address, and the rendering stream is used to display the three-dimensional scene model of the application in a video stream; The client obtains the access address of the application, establishes a communication connection with the server according to the access address, receives the rendering stream of the server according to the transmission parameters, and plays and displays the received rendering stream through the client playback medium.

30. A server device, wherein: The system comprises a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps: Generate an application file and access address for the application, and configure transmission parameters for the application, wherein the application file includes a three-dimensional scene model and logic code of the application, and the transmission parameters are used to represent parameters required for transmitting the application to a client device for display; generating a rendering stream according to the application file, wherein the rendering stream is associated with the access address, and the rendering stream is used to display the three-dimensional scene model of the application in a video stream; The rendering stream is sent to a client device based on the transmission parameters, wherein the client device includes a device that establishes a communication connection with a server device based on the access address.

31. A client device, wherein: The system comprises a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps: Obtaining an access address for the application, and establishing a communication connection with a server based on the access address; associating the access address with a rendering stream for the application, wherein the rendering stream is used to display a three-dimensional scene model of the application via a video stream; Receive the rendering stream associated with the access address sent by the server device, and play and display the received rendering stream through the client playback medium.

32. A computer storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method according to any one of claims 25 to 29 are implemented.

Images