Desktop virtualization method based on WebRTC

Abstract

The invention relates to a WebRTC-based desktop virtualization method. The method comprises the following steps: (1) providing a virtual desktop host; (2) setting a server; (3) accessing a Web interface or an API (Application Program Interface) provided by the server side through a Web browser of the user side; (4) using an RDP or VNC protocol to connect the server to the virtual desktop host, and adopting intranet transmission to ensure transmission stability; (5) enabling a user to log in and select a virtual desktop through a Web interface; (6) the signaling server processes the signaling request of the user side; (7) the WebRTC server establishes and maintains a WebRTC session; and (8) the behavior prediction server learns user behaviors by using machine learning. The WebRTC-based desktop virtualization method has the advantages that a WebRTC-based desktop virtualization scheme has remarkable advantages in the aspects of improving user experience, simplifying management, optimizing resource utilization, enhancing safety, reducing cost and the like, and a modern, efficient and reliable desktop virtualization solution is provided for enterprises and organizations.

Description

Technical Field

[0001] The invention relates to a desktop virtualization method based on WebRTC, and belongs to the field of desktop virtualization. Background Art

[0002] Desktop virtualization is a technology that virtualizes the computer desktop environment, allowing the user's desktop environment to run on a data center server instead of on a local computer. Users can access their virtual desktops over the network and use them on a variety of devices, no matter where they are.

[0003] Traditional desktop virtualization technology generally uses RDP / VNC and other protocols for transmission. For ordinary users, it is highly professional and complex to configure. Due to its heavy reliance on the client, traditional desktop virtualization is difficult to use across platforms and has very limited support for mobile office scenarios.

[0004] WebRTC (Web Real-Time Communication) is a real-time communication technology that allows web pages or applications to communicate audio, video, and data directly between users without going through a server. WebRTC was open sourced by Google after it acquired Global IP Solutions in 2011, and has now become an official standard of W3C and IETF.

[0005] The emergence of WebRTC technology has greatly simplified the development process of real-time communication, allowing developers to easily integrate real-time communication functions in web pages and applications, providing users with a richer and more convenient communication experience. As the technology continues to mature and become more popular, WebRTC is being used more and more widely in various fields.

[0006] WebRTC technology is very suitable for desktop virtualization and mobile office scenarios due to its low latency, low bandwidth usage and point-to-point communication.

[0007] However, existing desktop virtualization technologies have the following problems and shortcomings:

[0008] Highly professional and complex configuration: Traditional desktop virtualization technology usually relies on protocols such as RDP (Remote Desktop Protocol) or VNC (Virtual Network Computing) for transmission. These protocols are highly professional and complex to configure for ordinary users, making them difficult to understand and operate.

[0009] High client dependency: Due to its heavy reliance on client software, traditional desktop virtualization technology is difficult to achieve cross-platform use. Users need to install and configure the corresponding client software on different devices, which limits the flexibility and convenience of the technology.

[0010] Limited support for mobile office: Existing desktop virtualization technology has very limited support for mobile office scenarios. Due to the limitations of client software and configuration complexity, users face many inconveniences when using virtual desktops on mobile devices.

[0011] Network dependence and latency issues: Traditional technologies usually require data to be transferred through servers, which not only increases the load on the server, but may also cause higher network latency and affect the user experience. Summary of the invention

[0012] In order to overcome the defects of the prior art, the present invention provides a desktop virtualization method based on WebRTC. The technical solution of the present invention is:

[0013] A desktop virtualization method based on WebRTC includes the following steps:

[0014] (1) Provide a virtual desktop host that runs an operating system such as Windows or Linux to provide a desktop environment;

[0015] (2) Setting up a server, including RDP / VNC server, Web server, signaling server, WebRTC server and behavior prediction server;

[0016] (3) Access the web interface or API provided by the server through the web browser of the user;

[0017] (4) Use RDP or VNC protocol to connect the server to the virtual desktop host, and use intranet transmission to ensure transmission stability;

[0018] (5) Allow users to log in and select virtual desktops through a web interface, or allow third-party system integration through an API;

[0019] (6) The signaling server processes the signaling requests from the user, including establishing and disconnecting connections, exchanging WebRTC SDP information and ICE candidates;

[0020] (7) The WebRTC server establishes and maintains WebRTC sessions, converts RDP / VNC protocols into WebRTC video streams, and handles the transmission of audio, video, and data channels;

[0021] (8) The behavior prediction server uses machine learning to learn user behavior, build a user behavior model, analyze user input, predict their next action, and generate user event instructions in advance.

[0022] The server encodes the screen image of the virtual desktop host into a video stream by decoding the RDP / VNC protocol; transmits the video stream to the user end through the Video channel of WebRTC; and the user end browser decodes and displays the video stream.

[0023] Supports mainstream video encoding such as H264 / VP8 / VP9 / AV1, and selects the encoding algorithm with the highest compression rate based on user-side support.

[0024] The web browser on the user side monitors mouse and keyboard events, as well as operation events of smart devices; the behavior prediction server analyzes and predicts the user's next operation, intervenes when there is a network delay, and assists the user in operation;

[0025] The mouse and keyboard events and operation events are sent to the server through the WebRTC DataChannel; after receiving the user input event, the server encapsulates it into an RDP or VNC protocol command and sends the encapsulated command to the virtual desktop host for operation.

[0026] When transferring files, create a WebRTC DataChannel for the file transfer to ensure that the file transfer does not interfere with the transmission of desktop images and user input; set up a reserved interface to access the blockchain or online scanning service to ensure the security and traceability of the file; set up a transfer mechanism to implement file selection, transfer progress display and transfer completion notification, and support file transfer, including block transfer and breakpoint resumption; before starting and after the transfer, calculate and compare the HASH value to ensure the complete transfer of the file; call the file transfer capability of the RDP protocol, and finally upload the file to the virtual desktop host.

[0027] It also includes using DTLS and SRTP to ensure encryption of WebRTC communications and encrypting RDP / VNC connections; performing authentication when users log in to ensure that only authorized users can access the virtual desktop.

[0028] It also includes steps for performance optimization: selecting video codecs to reduce latency and improve image quality; dynamically adjusting encoding parameters based on network conditions; using machine learning models to adjust encoding and parameters to reduce latency and bandwidth usage; using the ICE framework and TURN server to ensure network penetration; and extending the WebRTC protocol to adapt to network scenarios.

[0029] The advantages of the present invention are:

[0030] 1. Real-time and low latency: Through WebRTC's point-to-point communication, a desktop operation response speed close to that of the local one is achieved, greatly reducing the latency caused by network transmission.

[0031] 2. Simplified user-side deployment: Users do not need to install or configure any special client software locally. They can access the virtual desktop through a web browser, which simplifies the user-side deployment process.

[0032] 3. Efficient file transfer: The file transfer mechanism of the dedicated DataChannel ensures fast and stable file transfer and will not be affected by desktop flow and user input events.

[0033] 4. Powerful network adaptability: Using WebRTC's ICE, STUN, and TURN mechanisms, the solution can automatically adapt to complex network environments, including NAT and firewall traversal, to ensure connection stability.

[0034] 5. High security: The entire communication process, including the WebRTC channel and RDP / VNC connection, is encrypted to ensure the security of data transmission.

[0035] 6. Improved user experience: The real-time and low-latency features allow users to experience the same smoothness as traditional desktops when using virtual desktops, improving the user experience.

[0036] 7. Easy to manage and maintain: The server centrally manages all virtual desktops and sessions, which simplifies the maintenance work of IT administrators and reduces operation and maintenance costs.

[0037] 8. Cross-platform compatibility: Users can access the virtual desktop on any device that supports WebRTC, whether it is a PC, tablet or smartphone, which increases the flexibility and convenience of the solution.

[0038] 9. Network resource optimization: Point-to-point communication reduces the data transmission burden on the server and optimizes the utilization of network resources, especially in large-scale deployments.

[0039] 10. Enhanced security: Encrypted communications and multi-level security measures protect the data security of enterprises and users, reducing the risk of data leakage and malicious attacks.

[0040] 11. Cost-effectiveness: No need to purchase and configure additional client software, reducing hardware and software costs, while also reducing long-term costs caused by client maintenance.

[0041] 12. Scalability: The web-based architecture is easy to expand, and more virtual desktops and users can be easily added to adapt to the expansion of the enterprise.

[0042] 13. Flexible integration: Providing a web interface and API, the solution can be easily integrated into existing enterprise applications and services, providing higher customization.

[0043] In summary, this WebRTC-based desktop virtualization solution has shown significant advantages in improving user experience, simplifying management, optimizing resource utilization, enhancing security, and reducing costs, providing enterprises and organizations with a modern, efficient, and reliable desktop virtualization solution. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with specific embodiments, and the advantages and features of the present invention will become clearer as the description proceeds. However, these embodiments are exemplary only and do not constitute any limitation to the scope of the present invention. It should be understood by those skilled in the art that the details and forms of the technical solution of the present invention may be modified or replaced without departing from the spirit and scope of the present invention, but these modifications and replacements all fall within the scope of protection of the present invention.

[0045] The present invention relates to a desktop virtualization method based on WebRTC, comprising the following steps:

[0046] (1) Provide a virtual desktop host that runs an operating system such as Windows or Linux to provide a desktop environment;

[0047] (2) Setting up a server, including RDP / VNC server, Web server, signaling server, WebRTC server and behavior prediction server;

[0048] (3) Access the web interface or API provided by the server through the web browser of the user;

[0049] (4) Use RDP or VNC protocol to connect the server to the virtual desktop host, and use intranet transmission to ensure transmission stability;

[0050] (5) Allow users to log in and select virtual desktops through a web interface, or allow third-party system integration through an API;

[0051] (6) The signaling server processes the signaling requests from the user, including establishing and disconnecting connections, exchanging WebRTC SDP information and ICE candidates;

[0052] (7) The WebRTC server establishes and maintains WebRTC sessions, converts RDP / VNC protocols into WebRTC video streams, and handles the transmission of audio, video, and data channels;

[0053] (8) The behavior prediction server uses machine learning to learn user behavior, build a user behavior model, analyze user input, predict their next action, and generate user event instructions in advance.

[0054] Advantages of this step:

[0055] Cross-platform compatibility: By accessing the web interface or API provided by the server through a web browser, users can access the virtual desktop on any device that supports WebRTC, whether it is a PC, tablet or smartphone. This improves the flexibility and convenience of the solution.

[0056] Simplified user-side deployment: Users do not need to install or configure any special client software locally. They can access the virtual desktop through a web browser, which simplifies the user-side deployment process.

[0057] Real-time and low latency: WebRTC's peer-to-peer communication feature enables a near-local desktop operation response speed, greatly reducing the latency caused by network transmission.

[0058] Efficient file transfer: The dedicated DataChannel file transfer mechanism ensures fast and stable file transfer without being affected by desktop streams and user input events.

[0059] Powerful network adaptability: By leveraging WebRTC's ICE, STUN, and TURN mechanisms, the solution can automatically adapt to complex network environments, including NAT and firewall traversal, to ensure connection stability.

[0060] High security: The entire communication process, including the WebRTC channel and RDP / VNC connection, is encrypted to ensure the security of data transmission.

[0061] Improved user experience: The real-time and low-latency features allow users to experience the same smoothness as traditional desktops when using virtual desktops, improving the user experience.

[0062] Easy to manage and maintain: The server centrally manages all virtual desktops and sessions, simplifying the maintenance work of IT administrators and reducing operation and maintenance costs.

[0063] The server encodes the screen image of the virtual desktop host into a video stream by decoding the RDP / VNC protocol; transmits the video stream to the user end through the Video channel of WebRTC; the user end browser decodes and displays the video stream; supports mainstream video encoding such as H264 / VP8 / VP9 / AV1, and selects the encoding algorithm with the highest compression rate according to the support of the user end.

[0064] The Web browser on the user side monitors mouse and keyboard events, as well as operation events of smart devices (such as the perception of user gestures by smart cameras, sensor values ​​of VR devices, etc.); the behavior prediction server analyzes and predicts the user's next operation, intervenes when there is a network delay (such as 300ms), and assists the user in the operation; the mouse and keyboard events and operation events are sent to the server side through the DataChannel of WebRTC; after receiving the user input event, the server side encapsulates it into an RDP or VNC protocol command, and sends the encapsulated command to the virtual desktop host for operation.

[0065] Based on the above scheme, the following advantages are achieved:

[0066] Efficient image transmission: By decoding the RDP / VNC protocol and encoding the screen image into a video stream, and transmitting it through the Video channel of WebRTC, the efficiency and real-time performance of image transmission are ensured.

[0067] Video encoding optimization: Supports mainstream video encoding such as H264 / VP8 / VP9 / AV1, and selects the encoding algorithm with the highest compression rate based on the user-side support to optimize bandwidth usage and image quality.

[0068] Cross-platform compatibility: The user end accesses through a Web browser without the need to install special client software, which improves the cross-platform compatibility and user convenience of the solution.

[0069] Intelligent user input processing: The user-side browser monitors mouse, keyboard and smart device operation events, and the behavior prediction server analyzes and predicts the user's next operation, reducing input delays and improving user experience.

[0070] Intelligent compensation for network delay: When the network delay is large (such as 300ms), the behavior prediction server intervenes to assist the user in operation, ensuring smooth operation and responsiveness.

[0071] Efficient use of data channels: User input events are sent to the server through WebRTC's DataChannel, ensuring efficient and stable data transmission.

[0072] Accurate encapsulation and execution of operations: After receiving user input events, the server encapsulates them into RDP or VNC protocol commands and sends them to the virtual desktop host for operation, ensuring the accuracy and reliability of the operation.

[0073] Enhanced security: WebRTC's peer-to-peer communication and encryption mechanisms enhance the security of data transmission and protect user data from leakage and attacks.

[0074] Improved user experience: The real-time and low-latency features allow users to experience a similar level of smoothness when using virtual desktops as local operations, significantly improving the user experience.

[0075] Easy to maintain and manage: Centralized server management simplifies the maintenance work of IT administrators and reduces operation and maintenance costs.

[0076] When transferring files, create a WebRTC DataChannel for the file transfer to ensure that the file transfer does not interfere with the transmission of desktop images and user input; set up a reserved interface to access the blockchain or online scanning service to ensure the security and traceability of the file; set up a transfer mechanism to implement file selection, transfer progress display and transfer completion notification, and support file transfer, including block transfer and breakpoint resumption; before starting and after the transfer, calculate the HASH value (such as sha-256) and compare it to ensure the complete transmission of the file; call the file transfer capability of the RDP protocol, and finally upload the file to the virtual desktop host.

[0077] It also includes the use of DTLS and SRTP to ensure the encryption of WebRTC communications, and can also support new encryption algorithms (such as SM or quantum encryption, etc.). Encrypt RDP / VNC connections; authenticate users when they log in to ensure that only authorized users can access the virtual desktop.

[0078] It also includes steps for performance optimization: selecting video codecs to reduce latency and improve image quality; dynamically adjusting encoding parameters based on network conditions; using machine learning models to adjust encoding and parameters to reduce latency and bandwidth usage; using the ICE framework and TURN server to ensure network penetration; and extending the WebRTC protocol to adapt to network scenarios.

[0079] In the present invention, the main innovations are:

[0080] WebRTC integration: Apply WebRTC technology to desktop virtualization, taking advantage of its peer-to-peer communication and real-time features to reduce server load and network latency.

[0081] Difference: Compared with traditional server-based desktop virtualization solutions, WebRTC provides a more direct communication path.

[0082] RDP / VNC protocol encapsulation without client configuration: The server automatically encapsulates WebRTC user input events into RDP or VNC protocol, and the user does not need to install or configure the RDP / VNC client.

[0083] Difference: It simplifies the deployment process on the user side and improves usability. Users can access the virtual desktop directly through a Web browser.

[0084] Web interface / API driven signaling: Provide a web interface or API to receive and process user-side signaling, establish and manage WebRTC sessions.

[0085] Difference: Compared with signaling mechanisms that require specialized client software, web interfaces / APIs are more flexible and easier to integrate and extend.

[0086] File transfer with dedicated DataChannel: Create an independent WebRTC DataChannel for file transfer to ensure the efficiency and stability of file transfer without sharing the channel with desktop images and user input.

[0087] Difference: In traditional desktop virtualization solutions, file transfers are usually mixed with desktop streams or input events in the same channel, which may cause performance issues.

[0088] Network penetration and adaptive streaming: Utilize WebRTC's ICE, STUN, and TURN mechanisms to ensure connection stability in different network environments and dynamically adjust the encoding parameters of the video stream according to network conditions.

[0089] Difference: Existing technologies may require complex network configuration and manual adjustments to ensure connection quality, while WebRTC provides automatic network penetration and adaptive streaming processing.

[0090] Security enhancement: Based on WebRTC communication, encryption of RDP / VNC connections is added to ensure the security of the entire communication process.

[0091] Difference: Compared with traditional solutions that rely only on server-side encryption, this solution strengthens security at multiple levels, including DTLS / SRTP encryption of WebRTC and encryption of RDP / VNC, and can also use more secure encryption algorithms to ensure communication security.

[0092] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A desktop virtualization method based on WebRTC, characterized in that: The following steps are involved: (1) Provide a virtual desktop host that runs an operating system such as Windows or Linux to provide a desktop environment; (2) Setting up a server, including RDP / VNC server, Web server, signaling server, WebRTC server and behavior prediction server; (3) Access the web interface or API provided by the server through the web browser of the user; (4) Use RDP or VNC protocol to connect the server to the virtual desktop host, and use intranet transmission to ensure transmission stability; (5) Allow users to log in and select virtual desktops through a web interface, or allow third-party system integration through an API; (6) The signaling server processes the signaling requests from the user, including establishing and disconnecting connections, exchanging WebRTC SDP information and ICE candidates; (7) The WebRTC server establishes and maintains WebRTC sessions, converts RDP / VNC protocols into WebRTC video streams, and handles the transmission of audio, video, and data channels; (8) The behavior prediction server uses machine learning to learn user behavior, build a user behavior model, analyze user input, predict their next action, and generate user event instructions in advance.

2. The WebRTC-based desktop virtualization method according to claim 1, characterized in that: The server encodes the screen image of the virtual desktop host into a video stream by decoding the RDP / VNC protocol; transmits the video stream to the user end through the Video channel of WebRTC; and the user end browser decodes and displays the video stream.

3. The WebRTC-based desktop virtualization method according to claim 1, characterized in that: The web browser on the user side monitors mouse and keyboard events, as well as operation events of smart devices; the behavior prediction server analyzes and predicts the user's next operation, intervenes when there is a network delay, and assists the user in operation; The mouse and keyboard events and operation events are sent to the server through the WebRTC DataChannel; after receiving the user input event, the server encapsulates it into an RDP or VNC protocol command and sends the encapsulated command to the virtual desktop host for operation.

4. The WebRTC-based desktop virtualization method according to claim 3, characterized in that: When transferring files, create a WebRTC DataChannel for the file transfer to ensure that the file transfer does not interfere with the transmission of desktop images and user input; set up a reserved interface to access the blockchain or online scanning service to ensure the security and traceability of the file; set up a transfer mechanism to implement file selection, transfer progress display and transfer completion notification, and support file transfer, including block transfer and breakpoint resumption; before starting and after the transfer, calculate and compare the HASH value to ensure the complete transfer of the file; call the file transfer capability of the RDP protocol, and finally upload the file to the virtual desktop host.

5. The WebRTC-based desktop virtualization method according to claim 4, characterized in that: It also includes using DTLS and SRTP to ensure encryption of WebRTC communications and encrypting RDP / VNC connections; performing authentication when users log in to ensure that only authorized users can access the virtual desktop.

6. The WebRTC-based desktop virtualization method according to claim 5, characterized in that: It also includes steps for performance optimization: selecting video codecs to reduce latency and improve image quality; dynamically adjusting encoding parameters based on network conditions; using machine learning models to adjust encoding and parameters to reduce latency and bandwidth usage; using the ICE framework and TURN server to ensure network penetration; and extending the WebRTC protocol to adapt to network scenarios.