Cloud rendering connection acceleration method and system based on WebSocket

Abstract

The invention provides a cloud rendering connection acceleration method and system based on WebSocket, and the method comprises the steps: a rear-end server provides a WebSocket connection address, and a front-end webpage and a three-dimensional application are connected with the WebSocket service by carrying the same pair of unique identifiers; the front-end webpage starts the three-dimensional application to initiate connection and sends SDP information to the rear-end server through WebSocket; when any one of the three-dimensional application or the front-end webpage receives the SDP information, corresponding answer data is generated and sent to a back-end server through WebSocket, the back-end server forwards the answer data to the other end after receiving the answer data, and the two parties successfully establish a connection; the front-end webpage and the three-dimensional application continuously send interaction event data to the back-end service, the back-end service continuously forwards the interaction event data to the other end until any one side sends a delete instruction, and the back-end server terminates connection and cleans related resources. According to the invention, the cloud rendering connection speed and the link stability can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud rendering, and in particular, to a method and system for accelerating cloud rendering connection based on WebSocket. Background Art

[0002] Cloud rendering technology mainly consists of a front-end web page, a back-end program, and a 3D application program. The front-end web page and the 3D application program establish a connection through the back-end program, and then push the 3D picture to the front-end page in the form of a video stream for display. At the same time, the front-end web page passes back a series of interaction events such as mouse and keyboard to the 3D application program, and the 3D application program makes a response. A common usage scenario is that the front-end web page pulls up the 3D application program by clicking a button, and then the two exchange data through the back-end service.

[0003] Currently, the cloud rendering technology based on the WebRTC protocol in the prior art constructs an efficient long connection communication mechanism by integrating the Render Streaming component to achieve real-time transmission of audio and video data and two-way event interaction. The process of establishing a connection by WebRTC is similar to the three-way handshake of TCP and requires at least four communications. WebRTC communication data includes various types, such as offer, candidate, answer, delete, etc. The four communications are: sending offer / answer, receiving answer / offer, sending network information, and receiving the other party's network information, as Figure 1 shown. After the connection is successfully established, to maintain the stability of the connection, the front-end web page and the 3D application need to continuously perform data interaction. A common data interaction method is: the back-end service provides an interface, and the front-end web page and the 3D application respectively call the interface to transmit data to each other. The complete implementation logic is as follows: before the front-end web page and the 3D application receive data from the other party, they continuously send data containing offer and candidate headers to the other party through the back-end service. Among them, offer indicates an application to establish a connection, and candidate contains network information (such as IP address, port, etc.). When one party receives the offer request from the other party, it will send an answer message to the other party through the back-end service until one party receives the answer message, and then the connection is officially established. After the connection is established, both parties can send data such as event interaction to each other until one party disconnects and sends a delete message, at which time the connection is closed.

[0004] When the back-end service uses the HTTP protocol to provide an interface for forwarding data, a polling mechanism is usually used for data forwarding, as Figure 2 、 Figure 3As shown. Although this method has relatively high security and is easy to implement, it will waste bandwidth and server resources, and may even lead to the transmission of outdated or invalid data, resulting in the failure of establishing a cloud rendering connection.

[0005] Patent document CN117742997A discloses a method for interactive computing power of three-dimensional cloud rendering push stream based on WebRTC. The process of establishing a streaming media channel and a data channel from the client of the method to the cloud rendering server includes the following steps: S1: Establish a long connection with the signaling service through the client, start the rendering engine program of the cloud server by the signaling, and return a successful creation status; S2: After the cloud server rendering engine program is started, the client creates a streaming media channel and a data channel locally, and sends an offer containing this information to the signaling, which forwards it to the push stream program to agree on the parameters of the streaming media transmission and establish peer-to-peer communication; S3: After receiving the client offer, the push stream program creates a shared memory, and notifies the rendering engine program to start rendering and establish a shared memory rendering transmission channel.

[0006] However, although patent document CN117742997A establishes an interactive link between the front and back ends through WebRTC, which can reduce a certain amount of performance resource consumption, the WebRTC link by default adopts the HTTP polling mechanism. When sending offers and answers, due to the inherent characteristics of the polling mechanism, there is a time difference in the communication between the front and back ends, resulting in uncontrollable connection speed or high latency. And this patent document mainly focuses on the optimization of human-computer interaction and coordinate transformation and other performance improvements when implementing interface interaction based on WebRTC cloud rendering, while the present application focuses on optimizing the connection speed and stability of cloud rendering through WebSocket.

[0007] Therefore, there is a need in the market for a WebSocket-based cloud rendering connection acceleration method and system that can improve the cloud rendering connection speed and link stability. Summary of the Invention

[0008] Aiming at the deficiencies in the prior art, the purpose of the present invention is to provide a WebSocket-based cloud rendering connection acceleration method and system.

[0009] According to a WebSocket-based cloud rendering connection acceleration method provided by the present invention, it includes:

[0010] Step S1: The backend server provides a WebSocket connection address, and the front-end web page and the three-dimensional application connect to the WebSocket service by carrying the same pair of unique identifiers;

[0011] Step S2: The front-end web page starts the three-dimensional application to initiate a connection and sends SDP information to the backend server through WebSocket;

[0012] Step S3: When either the 3D application or the front-end web page receives the SDP information, corresponding answer data will be generated and sent to the back-end server via WebSocket. After receiving the answer data, the back-end service forwards it to the other end, and the two parties successfully establish a connection.

[0013] Step S4: The front-end web page and the 3D application will continuously send interactive event data to the back-end service, and the back-end service will continuously forward the interactive event data to the other end until either party sends a delete instruction, at which point the back-end server will terminate the connection and clean up relevant resources.

[0014] Preferably, the unique identifier is used to pair the front-end web page and the 3D application to ensure correct data forwarding.

[0015] Preferably, after receiving the connection request, the back-end server stores the unique identifier, the Session information between the front-end web page and the 3D application, and the SDP information.

[0016] The SDP information includes offer and candidate data.

[0017] Preferably, when the 3D application is starting up, the front-end web page continuously sends SDP information, and the back-end service continuously stores it.

[0018] After the 3D application is successfully started and paired successfully with the unique identifier, the back-end server forwards the previously stored SDP of the front-end web page to the 3D application.

[0019] Preferably, an application pre-start method is adopted, that is, the 3D application program is preferentially started and connection information is sent to the back-end service with the unique identifier. When the front-end web page connects with the unique identifier, the back-end service forwards the connection data of the 3D application to the front-end web page.

[0020] A cloud rendering connection acceleration system based on WebSocket according to the present invention includes:

[0021] Module M1: The back-end server provides a WebSocket connection address, and the front-end web page and the 3D application connect to the WebSocket service by carrying the same pair of unique identifiers.

[0022] Module M2: The front-end web page starts the 3D application to initiate a connection and sends SDP information to the back-end server via WebSocket.

[0023] Module M3: When either the 3D application or the front-end web page receives the SDP information, it will generate corresponding answer data and send it to the back-end server via WebSocket. After receiving the answer data, the back-end service forwards it to the other end, and the two parties successfully establish a connection;

[0024] Module M4: The front-end web page and the 3D application will continuously send interactive event data to the back-end service, and the back-end service will continuously forward the interactive event data to the other end until either party sends a delete instruction, at which point the back-end server will terminate the connection and clean up relevant resources.

[0025] Preferably, the unique identifier is used to pair the front-end web page and the 3D application to ensure correct data forwarding.

[0026] Preferably, after receiving the connection request, the back-end server stores the unique identifier, the Session information between the front-end web page and the 3D application, and the SDP information;

[0027] The SDP information includes offer and candidate data.

[0028] Preferably, when the 3D application is starting up, the front-end web page will continuously send SDP information, and the back-end service will continuously store it;

[0029] After the 3D application is successfully started and paired successfully with the unique identifier, the back-end server forwards the previously stored SDP of the front-end web page to the 3D application.

[0030] Preferably, the application pre-start method is adopted, that is, the 3D application program is started first and sends connection information to the back-end service with the unique identifier. When the front-end web page connects with the unique identifier, the back-end service forwards the connection data of the 3D application to the front-end web page.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention provides an interface to forward data through the WebSocket protocol, abandoning the polling mechanism, which can improve the connection speed and stability.

[0033] 2. The present invention adopts the application pre-start method to achieve almost instant connection for cloud rendering, significantly reducing the waiting time. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects and advantages of the present invention will become more apparent:

[0035] Figure 1 It is a schematic diagram of the WebRTC connection establishment process;

[0036] Figure 2 It is a schematic diagram of the interaction of the short polling system of the HTTP protocol;

[0037] Figure 3 It is a schematic diagram of the interaction of the long polling system of the HTTP protocol;

[0038] Figure 4 It is a schematic flow diagram of the method for accelerating cloud rendering connection based on WebSocket of the present invention. Specific implementation manner

[0039] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several changes and improvements can still be made. These all belong to the protection scope of the present invention.

[0040] The present invention focuses on accelerating the connection process of cloud rendering and realizing fast push-streaming picture loading. Replacing the HTTP interface of the backend server with the WebSocket protocol and abandoning the polling mechanism can significantly improve the connection speed of cloud rendering based on WebRTC. After actual testing, the present invention can increase the connection speed by 80%-120%.

[0041] According to a method for accelerating cloud rendering connection based on WebSocket provided by the present invention, as Figure 4 shown, it includes:

[0042] Step S1: Pair the WebSocket connection with a unique identifier. The backend server provides a WebSocket connection address, and the front-end web page and the 3D application connect to the WebSocket service by carrying the same pair of unique identifiers. The unique identifier is used to pair the front-end web page and the 3D application to ensure correct data forwarding. After receiving the connection request, the backend server stores the unique identifier and the Session information between the front-end web page and the 3D application.

[0043] Step S2: The front-end web page initiates a connection and sends SDP (Session Description Protocol) information. When the front-end web page starts the 3D application by the user clicking a button, it sends SDP information, including data such as offer and candidate, to the back-end service through WebSocket, and the back-end service stores the SDP information. If the 3D application is in the startup process, the front-end web page will continuously send data such as offer and candidate, and the back-end service will continuously store them. After the 3D application is successfully started and paired successfully with the unique identifier, the back-end server forwards the previously stored offer and candidate data of the front-end web page to the 3D application.

[0044] Step S3: SDP response and connection establishment. When either the 3D application or the front-end web page receives the SDP information, it will generate corresponding answer data and send it to the back-end server through WebSocket. After receiving the answer data, the back-end service forwards it to the other end. The two parties successfully establish a connection.

[0045] Step S4: Interactive event data transmission and connection termination. After the two parties successfully establish a connection, the front-end web page and the 3D application will continuously send interactive event data such as keyboard and mouse to the back-end service, and the back-end service will continuously forward the interactive event data to the other end until either party sends a delete instruction, and the back-end server will terminate the connection and clean up relevant resources.

[0046] In addition, in order to further optimize the connection speed, the present invention adopts the method of pre-starting the application, that is, preferentially starting the 3D application program and sending connection information to the back-end service with a unique identifier. When the front-end web page connects with a unique identifier, the back-end service forwards the connection data of the 3D application to the front-end web page. In this way, the connection waiting time can be greatly reduced, and a near-second-level cloud rendering connection experience can be achieved.

[0047] The present invention also provides a cloud rendering connection acceleration system based on WebSocket. The cloud rendering connection acceleration system based on WebSocket can be implemented by executing the process steps of the cloud rendering connection acceleration method based on WebSocket, that is, those skilled in the art can understand the cloud rendering connection acceleration method based on WebSocket as a preferred implementation manner of the cloud rendering connection acceleration system based on WebSocket.

[0048] A WebSocket-based cloud rendering connection acceleration system provided by the present invention includes: Module M1: The backend server provides a WebSocket connection address, and the front-end web page and the 3D application connect to the WebSocket service by carrying the same pair of unique identifiers. Module M2: The front-end web page starts the 3D application to initiate a connection and sends SDP information to the backend server through WebSocket. The unique identifier is used to pair the front-end web page and the 3D application to ensure correct data forwarding. After receiving the connection request, the backend server stores the unique identifier and the Session information between the front-end web page and the 3D application, and stores the SDP information. The SDP information includes offer and candidate data. When the 3D application is starting up, the front-end web page will continuously send SDP information, and the backend service will continuously store it. After the 3D application is successfully started and paired successfully with the unique identifier, the backend server forwards the previously stored SDP of the front-end web page to the 3D application. Module M3: When either the 3D application or the front-end web page receives the SDP information, it will generate corresponding answer data and send it to the backend server through WebSocket. After receiving the answer data, the backend service forwards it to the other end, and the two parties successfully establish a connection. Module M4: The front-end web page and the 3D application will continuously send interaction event data to the backend service, and the backend service will continuously forward the interaction time data to the other end until either party sends a delete instruction, and the backend server will terminate the connection and clean up relevant resources. The application pre-start method is adopted, that is, the 3D application program is preferentially started, and connection information is sent to the backend service by carrying a unique identifier. When the front-end web page connects by carrying the unique identifier, the backend service forwards the connection data of the 3D application to the front-end web page.

[0049] Those skilled in the art know that in addition to implementing the system and its various devices, modules, and units provided by the present invention in the form of pure computer-readable program code, the method steps can be logically programmed to enable the system and its various devices, modules, and units provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers to achieve the same function. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be regarded as the structure within the hardware component; the devices, modules, and units for implementing various functions can also be regarded as both software modules for implementing the method and the structure within the hardware component.

[0050] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined arbitrarily with each other.

Claims

1. A cloud rendering connection acceleration method based on WebSocket, characterized in that Including: Step S1: The backend server provides a WebSocket connection address, and the front-end web page and the 3D application connect to the WebSocket service by carrying the same pair of unique identifiers; Step S2: The front-end web page starts the 3D application to initiate a connection and sends SDP information to the backend server through WebSocket; Step S3: When either the 3D application or the front-end web page receives the SDP information, corresponding answer data will be generated and sent to the backend server through WebSocket. After receiving the answer data, the backend service forwards it to the other end, and the two parties successfully establish a connection; Step S4: The front-end web page and the 3D application will continuously send interactive event data to the backend service, and the backend service will continuously forward the interactive time data to the other end until either party sends a delete instruction, and the backend server will terminate the connection and clean up relevant resources.

2. The WebSocket-based cloud rendering connection acceleration method according to claim 1, wherein The unique identifier is used to pair the front-end web page and the 3D application to ensure correct data forwarding.

3. The WebSocket-based cloud rendering connection acceleration method according to claim 1, wherein After receiving the connection request, the backend server stores the unique identifier and the Session information between the front-end web page and the 3D application, and stores the SDP information; The SDP information includes offer and candidate data.

4. The WebSocket-based cloud rendering connection acceleration method according to claim 1, characterized in that When the 3D application is starting up, the front-end web page will continuously send SDP information, and the backend service will continuously store it; After the 3D application is successfully started and paired successfully with the unique identifier, the backend server forwards the previously stored SDP of the front-end web page to the 3D application.

5. The WebSocket-based cloud rendering connection acceleration method according to claim 1, wherein Adopt the method of pre-starting the application, that is, give priority to starting the 3D application program and send connection information to the backend service by carrying a unique identifier. When the front-end web page connects by carrying the unique identifier, the backend service forwards the connection data of the 3D application to the front-end web page.

6. A cloud rendering connection acceleration system based on WebSocket, characterized in that, Including: Module M1: The backend server provides a WebSocket connection address, and the front-end web page and the 3D application connect to the WebSocket service by carrying the same pair of unique identifiers; Module M2: The front-end web page starts the 3D application to initiate a connection and sends SDP information to the backend server through WebSocket; Module M3: When either the 3D application or the front-end web page receives the SDP information, corresponding answer data will be generated and sent to the backend server through WebSocket. After receiving the answer data, the backend service forwards it to the other end, and the two parties successfully establish a connection; Module M4: The front-end web page and the 3D application will continuously send interactive event data to the backend service, and the backend service will continuously forward the interactive time data to the other end until either party sends a delete instruction, and the backend server will terminate the connection and clean up relevant resources.

7. The WebSocket-based cloud rendering connection acceleration system according to claim 6, wherein, The unique identifier is used to pair the front-end web page and the 3D application to ensure correct data forwarding.

8. The WebSocket-based cloud rendering connection acceleration system according to claim 6, characterized in that After receiving the connection request, the backend server stores the unique identifier and the Session information between the front-end web page and the 3D application, and stores the SDP information; The SDP information includes offer and candidate data.

9. The WebSocket-based cloud rendering connection acceleration system according to claim 6, wherein When the 3D application is starting up, the front-end web page will continuously send SDP information, and the back-end service will continuously store it; When the 3D application is successfully started and paired successfully with the unique identifier, the back-end server will forward the previously stored SDP of the front-end web page to the 3D application.

10. The WebSocket-based cloud rendering connection acceleration system according to claim 6, wherein, Adopt the application pre-start method, that is, give priority to starting the 3D application program, and send connection information to the back-end service with the unique identifier. When the front-end web page connects with the unique identifier, the back-end service will forward the connection data of the 3D application to the front-end web page.

Images