Desktop cloud system and method, device, and medium for providing virtual desktops

Abstract

This application provides a desktop cloud system, including: a desktop cloud server and a terminal. The desktop cloud server deploys virtual desktop instances. The desktop cloud server is used to obtain the encoded data output by the virtual desktop instances and send the encoded data to the terminal corresponding to the virtual desktop. The terminal is used to decode the source data from the encoded data and present the virtual desktop according to the source data. Since there is no need for secondary transcoding, additional computing overhead is avoided, and the desktop cloud server can serve more terminals, reducing the hardware cost of the desktop cloud system.

Description

[0001] This application claims the priority of a Chinese patent application with the application number 202011052754.3 and the application title "Method, apparatus, server, and storage medium for providing virtual desktops", which was filed with the China National Intellectual Property Administration on September 29, 2020. The entire content of this application is incorporated herein by reference. Technical Field

[0002] This application relates to the field of cloud computing technology, and in particular, to a desktop cloud system, as well as a method, apparatus, device, and computer-readable storage medium for providing virtual desktops. Background Art

[0003] To meet the need for working anytime and anywhere, the industry has proposed the virtual desktop infrastructure (VDI). The system implemented based on this architecture is called a desktop cloud system. The desktop cloud system includes personal devices such as terminals and a desktop cloud server. Users can access the desktop virtual machine environment deployed on the desktop cloud server through the desktop cloud client installed on the personal device.

[0004] The core technology of the desktop cloud system is the desktop access protocol. The desktop access protocol defines the interaction protocol and communication channel for transmitting data such as desktop images, videos, and peripheral events between the desktop cloud client and the desktop cloud server (such as a virtual machine on the desktop cloud server).

[0005] However, different desktop cloud products have each defined their own private protocol specifications. To interface with different protocol specifications, open-source HTML5 client projects such as Apache Guacamole have deployed an extensible protocol conversion component architecture on the Web server side. By adding independent components (such as guacd + remote desktop protocol components), different protocol specifications are docked and converted into protocol data that conforms to the Guacamole protocol specification. The front-end Web client and the server-side components are docked through the publicly available Guacamole protocol specification.

[0006] This architecture increases flexibility and protocol extensibility, but it brings significant performance problems. Specifically, the protocol conversion component needs to consume a large amount of computing power to decrypt and decode server-side protocol data such as video streams, and recompress the decoded data according to the algorithms of the Guacamole protocol for video, images, audio, etc. This results in a large amount of additional computing overhead, limiting the number of Web clients served by a single Web server and increasing the hardware cost of the desktop cloud system. Summary of the Invention

[0007] The present application provides a desktop cloud system. The desktop cloud system includes a desktop cloud server and a terminal. The desktop cloud server obtains the encoded data output by the virtual desktop instance, sends the encoded data to the terminal, and the terminal directly decodes it on the terminal side and presents the virtual desktop according to the decoded source data, without performing secondary transcoding, avoiding additional computational overhead, enabling the desktop cloud server to serve more terminals, and reducing the hardware cost of the desktop cloud system. The present application also provides a method for providing a virtual desktop based on the desktop cloud system, as well as corresponding apparatuses, devices, computer-readable storage media, and computer program products.

[0008] In a first aspect, the present application provides a desktop cloud system. The desktop cloud system includes a desktop cloud server and a terminal. A virtual desktop instance is deployed on the desktop cloud server. Specifically, the virtual desktop instance is a process or thread for providing a virtual desktop, and the process or thread can directly run on a physical machine such as the desktop cloud server, or run in a virtual machine or container on the physical machine. The virtual desktop instance can be generated by the desktop cloud server executing a code block with the function of providing a virtual desktop.

[0009] Specifically, the desktop cloud server is configured to obtain the encoded data output by the virtual desktop instance, send the encoded data to the corresponding terminal, and then the terminal decodes the source data from the encoded data on the terminal side and presents the virtual desktop according to the source data. Since there is no need for secondary transcoding, the additional computational overhead caused by secondary transcoding is avoided, the number of terminals that the desktop cloud server can serve is increased, and the hardware cost of the desktop cloud system is reduced.

[0010] In some possible implementation manners, the desktop cloud server does not encode or decode the encoded data output by the virtual desktop instance, thereby solving the problem in the related art that a large amount of computing power is required to decrypt and decode server-side protocol data such as video streams, and re-compress the decoded data according to the algorithm of a unified protocol for data such as video, images, and audio, reducing the computational overhead and the hardware cost of the desktop cloud system.

[0011] In some possible implementation manners, the terminal is specifically configured to utilize bytecode specifications, for example, compile the algorithm provided by a private desktop access protocol into bytecode for subsequent direct execution of the bytecode to implement decoding the source data from the encoded data. The bytecode specifications include, but are not limited to, the WebAssembly (wasm) bytecode specification and the Java bytecode specification.

[0012] On the one hand, after the source code of the algorithm provided by the private desktop access protocol is compiled into bytecode, the source code is invisible on the terminal, thus solving the problem of leakage of the private desktop access protocol. On the other hand, the algorithm provided by the private desktop access protocol is compiled into bytecode, such as being compiled into WebAssembly bytecode. Its volume is greatly reduced, and it does not require an interpreter for interpretation. It can be loaded and instantiated relatively quickly, reducing the waiting time before running. It can execute computationally intensive tasks such as encoding and decoding with performance close to that of binary executable programs, achieving encoding and decoding acceleration, further increasing the number of terminals that the desktop cloud server can serve and reducing the hardware cost.

[0013] In some possible implementation manners, the terminal is further configured to receive an event, encode the event to obtain an encoded event, and then send the encoded event to the desktop cloud server. Correspondingly, the desktop cloud server is specifically configured to operate the virtual desktop instance according to the encoded event to obtain encoded data output by the virtual desktop instance.

[0014] Thereby, remote operation of the virtual desktop can be realized, and an effect approximate to operating the desktop locally can be obtained, which provides convenience for users and improves the user experience.

[0015] In some possible implementation manners, the terminal can process any combination of the following multiple steps in parallel:

[0016] Encode the event to obtain an encoded event;

[0017] Send the encoded event to the desktop cloud server;

[0018] Receive the encoded data;

[0019] Decode the source data from the encoded data.

[0020] Specifically, the terminal can create multiple concurrent threads, such as creating multiple concurrent peripheral processing threads, audio processing threads, transmission threads, video processing threads, image processing threads, etc. These multiple concurrent threads can execute in parallel, so as to decouple the desktop protocol operations and avoid the synchronous dependence between desktop protocol operations from affecting the real-time performance of desktop operations.

[0021] In some possible implementation manners, the multiple concurrent threads created by the terminal can share memory. In this way, these threads can interact data through the shared memory, reducing data copying. On the one hand, it can reduce resource occupancy, and on the other hand, it saves the latency caused by data copying.

[0022] In some possible implementations, the event specifically includes at least one of a mouse input event, a keyboard input event, and an audio input event. In this way, different interaction methods such as keyboard and mouse interaction and voice interaction can be provided to meet the personalized needs of users.

[0023] In some possible implementations, the source data includes any one or more of image data, audio data, and video data. Based on this, the desktop cloud system can not only provide a virtual desktop for users, but also remotely play audio or video to meet the personalized needs of users.

[0024] In some possible implementations, the terminal deploys a browser client for accessing the virtual desktop instance. Since the browser client has good compatibility, the desktop cloud system based on the browser client has high availability. Moreover, there is no need to develop corresponding versions of dedicated clients for different personal devices, further reducing the cost of the desktop cloud system.

[0025] In some possible implementations, an application layer long connection can be established between the terminal and the desktop cloud server, such as a WebSocket long connection. In this way, the terminal and the desktop cloud server can directly transmit encoded data through this application layer long connection without having to perform secondary transcoding, reducing the computing overhead and the hardware cost of the desktop cloud system.

[0026] In a second aspect, the present application provides a method for providing a virtual desktop. This method can be executed by a desktop cloud system. Specifically, the desktop cloud system includes a desktop cloud server and a terminal, and the desktop cloud server deploys a virtual desktop instance. The method includes:

[0027] The desktop cloud server obtains the encoded data output by the virtual desktop instance and sends the encoded data to the terminal corresponding to the virtual desktop instance;

[0028] The terminal decodes the source data from the encoded data and presents the virtual desktop according to the source data.

[0029] In some possible implementations, the method further includes:

[0030] The desktop cloud server does not encode or decode the encoded data output by the virtual desktop instance.

[0031] In some possible implementations, the terminal decodes the source data from the encoded data, including:

[0032] The terminal decodes the source data from the encoded data by using the bytecode specification.

[0033] In some possible implementations, the method further includes:

[0034] The terminal receives an event, encodes the event to obtain an encoded event, and then sends the encoded event to the desktop cloud server;

[0035] The desktop cloud server obtains the encoded data output by the virtual desktop instance, including:

[0036] The desktop cloud server operates the virtual desktop instance according to the encoded event, and obtains the encoded data output by the virtual desktop instance.

[0037] In some possible implementation manners, the terminal processes in parallel any combination of the following multiple steps:

[0038] Encode the event to obtain an encoded event;

[0039] Send the encoded event to the desktop cloud server;

[0040] Receive the encoded data;

[0041] Decode the source data from the encoded data.

[0042] In some possible implementation manners, the event includes at least one of a mouse input event, a keyboard input event, and an audio input event.

[0043] In some possible implementation manners, the source data includes any one or more of image data, audio data, and video data.

[0044] In some possible implementation manners, the terminal deploys a browser client for accessing the virtual desktop instance.

[0045] In a third aspect, the present application provides a device for providing a virtual desktop. The device includes a module for executing the method steps performed by the desktop cloud server in the second aspect or any one of the implementation manners of the second aspect.

[0046] In a fourth aspect, the present application provides a device for providing a virtual desktop. The device includes a module for executing the method steps performed by the terminal in the second aspect or any one of the implementation manners of the second aspect.

[0047] In a fifth aspect, the present application provides a desktop cloud server. The desktop cloud server is used to implement the functions of the desktop cloud server in the desktop cloud system described in the first aspect or any one of the implementation manners of the first aspect.

[0048] In a sixth aspect, the present application provides a terminal. The terminal is used to implement the functions of the terminal in the desktop cloud system described in the first aspect or any one of the implementation manners of the first aspect.

[0049] In a seventh aspect, the present application provides a virtual desktop instance. The virtual desktop instance is used to implement the functions of the virtual desktop instance in the desktop cloud system as described in the first aspect or any implementation manner of the first aspect.

[0050] In an eighth aspect, the present application provides a desktop cloud server. The desktop cloud server includes a processor and a memory. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory to cause the device to execute the method in the second aspect or any implementation manner of the second aspect.

[0051] In a ninth aspect, the present application provides a terminal. The terminal includes a processor and a memory. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory to cause the terminal to execute the method in the second aspect or any implementation manner of the second aspect.

[0052] In a tenth aspect, the present application provides a computer-readable storage medium storing instructions that direct a desktop cloud server to execute the method described in the second aspect or any implementation manner of the second aspect.

[0053] In an eleventh aspect, the present application provides a computer-readable storage medium storing instructions that direct a terminal to execute the method described in the second aspect or any implementation manner of the second aspect.

[0054] In a twelfth aspect, the present application provides a computer program product containing instructions. When it runs on a desktop cloud server, it causes the desktop cloud server to execute the method described in the second aspect or any implementation manner of the second aspect.

[0055] In a thirteenth aspect, the present application provides a computer program product containing instructions. When it runs on a terminal, it causes the terminal to execute the method described in the second aspect or any implementation manner of the second aspect.

[0056] Based on the implementation manners provided in the above aspects of the present application, further combinations can be made to provide more implementation manners. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the embodiments will be briefly introduced below.

[0058] Figure 1 It is a system architecture diagram of a desktop cloud system provided by an embodiment of the present application;

[0059] Figure 2 Flowchart of a method for providing a virtual desktop provided by an embodiment of the present application;

[0060] Figure 3 Schematic diagram of the process of transmitting data by the end - side protocol module provided by an embodiment of the present application;

[0061] Figure 4 Schematic diagram of the process of interacting data by the browser client provided by an embodiment of the present application;

[0062] Figure 5 Flowchart of a method for providing a virtual desktop provided by an embodiment of the present application;

[0063] Figure 6 Schematic diagram of the structure of a desktop cloud server provided by an embodiment of the present application;

[0064] Figure 7 Schematic diagram of the structure of a terminal provided by an embodiment of the present application. Detailed implementation manners

[0065] The terms "first" and "second" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0066] First, some technical terms involved in the embodiments of the present application are introduced.

[0067] The virtual desktop infrastructure (VDI) is a system architecture that provides virtual desktops for users. The principle of the virtual desktop solution based on the VDI architecture is to prepare a dedicated virtual machine for each user on the server side and deploy the operating system and various applications required by the user in the virtual machine, and then deliver the complete virtual machine desktop (i.e., the virtual desktop) to remote users for use through the desktop access protocol.

[0068] The system based on the VDI architecture is also called the desktop cloud system. The desktop cloud system includes a desktop cloud server and a terminal. The desktop access protocol defines the interaction protocol and communication channel for transmitting data such as desktop images, videos, and peripheral events between the terminal (for example, the desktop cloud client deployed on the terminal) and the desktop cloud server (for example, the virtual machine deployed on the desktop cloud server).

[0069] Different types of desktop cloud systems each define their own private desktop access protocols. To interface with different protocol specifications, the open-source client project deploys a scalable protocol conversion component architecture on the network (web) server. The scalable protocol conversion component architecture docks with different protocol specifications by adding independent components and converts them into protocol data that conforms to a specified protocol specification, such as the Guacamole protocol specification. The terminal and the desktop cloud server are docked through the Guacamole protocol specification. In this way, the flexibility and protocol scalability of the desktop cloud system can be improved.

[0070] However, the protocol conversion component requires a large amount of computing power to decrypt and decode server-side protocol data such as video streams, and recompresses the decoded data according to the algorithms of the Guacamole protocol for data such as video, images, and audio. This results in a large amount of additional computing overhead, limiting the number of web clients served by the web server and increasing the hardware cost of the desktop cloud system.

[0071] In view of this, the embodiments of the present application provide a desktop cloud system and a method for providing a virtual desktop based on the desktop cloud system. The desktop cloud system includes a desktop cloud server and a terminal. Among them, the desktop cloud server deploys a virtual desktop instance. The desktop cloud server is used to obtain the encoded data output by the virtual desktop instance, send the encoded data to the corresponding terminal, and then the terminal decodes the source data from the encoded data on the terminal side and presents the virtual desktop according to the source data. Since there is no need for secondary transcoding, the additional computing overhead caused by secondary transcoding is avoided, the number of terminals that the desktop cloud server can serve is increased, and the hardware cost of the desktop cloud system is reduced.

[0072] Among them, the terminal can use bytecode specifications, such as using the WebAssembly (wasm) bytecode specification, to precompile the algorithms provided by the private desktop access protocol into bytecodes and directly execute the bytecodes for encoding and decoding. The source code of the algorithms provided by the private desktop access protocol is not visible on the terminal, thus solving the problem of leakage of the private desktop access protocol.

[0073] Moreover, the algorithms provided by the private desktop access protocol are compiled into bytecodes, such as being compiled into WebAssembly bytecodes. Their volume is greatly reduced, and there is no need for an interpreter to interpret them. They can be loaded and instantiated relatively quickly, reducing the waiting time before running. They can execute computationally intensive tasks such as encoding and decoding with performance close to that of binary executable programs, achieving encoding and decoding acceleration, further increasing the number of terminals that the desktop cloud server can serve, and further reducing the hardware cost.

[0074] To make the technical solution of this application clearer and easier to understand, the desktop cloud system provided in the embodiments of this application will be introduced below with reference to the accompanying drawings.

[0075] See Figure 1 The schematic architecture diagram of the desktop cloud system shown. The desktop cloud system 10 includes a terminal 12 and a desktop cloud server 14. The desktop cloud server 14 deploys a virtual desktop instance 142. Among them, the virtual desktop instance 142 is specifically a process or thread for providing a virtual desktop. This process or thread can run directly on a physical machine such as the desktop cloud server 14, or run in a virtual machine or container on the physical machine. The terminal 12 can deploy a client for accessing the virtual desktop instance 142, such as a browser client 122 for accessing the virtual desktop instance. In some embodiments, the browser client 122 (or the terminal 12) corresponds to the virtual desktop instance 142 one by one.

[0076] Specifically, the desktop cloud server 14 is used to obtain the encoded data output by the virtual desktop instance 142. The encoded data refers to the data obtained by encoding the source data, for example, the data obtained by encoding according to the algorithm in the private desktop access protocol. The source data can include any one or more of image data, audio data, and video data. Then the desktop cloud server 14 sends the encoded data to the terminal 12 corresponding to the virtual desktop instance 142. The terminal 12 is used to decode the source data from the encoded data and present the virtual desktop according to the source data. Among them, the terminal 12 decodes the source data and presents the virtual desktop can be specifically implemented by the browser client 122.

[0077] In some possible implementation manners, see Figure 1 , the browser client 122 includes a side protocol module 1222 and a portal module 1224. Among them, the side protocol module 1222 is used to decode the source data from the encoded data, and the portal module 1224 is used to present the virtual desktop according to the source data.

[0078] It should be noted that the end - side protocol module 1222 in the browser client 122 can be obtained in real - time when the virtual desktop is needed. Specifically, the desktop cloud system 10 further includes an access control subsystem, and this access control subsystem includes a network server 16. The portal module 1224 can present a desktop user login page, capture the login information input by the user through the desktop user login page, such as user name, password, verification code and other information, and then send the login information to the network service module 162 in the network server 16. The network service module 162 can perform login verification based on the login information. For example, it compares the login information with the relevant information in the account database to achieve login verification. When the login verification is passed, the browser client 122 can download the code of the end - side protocol module 1222 from the network server 16, for example, from the network service module 162, and then deploy the end - side protocol module 1222 in the browser client 122 according to this code.

[0079] Furthermore, the network server 16 may also include an authorization control module 164. This authorization control module 164 is used to authenticate the logged - in user. For example, it authenticates the access to the virtual desktop instance 142. After the authentication is passed, the logged - in user is allowed to access the virtual desktop instance 142. For this reason, the browser client 122 can download the code of the end - side protocol module 1222 from the network server 16. Otherwise, the logged - in user is refused to access the virtual desktop instance 142, and the browser client 122 fails to download the code of the end - side protocol module 1222.

[0080] Among them, the network service module 162 and the authorization control module 164 can be software modules or hardware modules with corresponding functions. When it is a software module, it can be integrated in the same software or distributed in different software. In addition, Figure 1 taking the deployment of the network service module 162 and the authorization control module 164 in the network server 16 as an example, in some embodiments, the network service module 162 and the authorization control module 164 can also be respectively deployed in different servers. For example, the network service module 162 is deployed in the network server 16, and the authorization control module 164 is deployed in the authentication server.

[0081] In some possible implementation manners, the browser client 122 is used to receive events, such as any one or more of keyboard input events, mouse input events, or audio input events, etc., and then encodes the events to obtain encoded events, and sends the encoded events to the desktop cloud server 14. Correspondingly, the desktop cloud server 14 is used to operate the virtual desktop instance 142 according to the encoded events to obtain the encoded data output by the virtual desktop instance 142.

[0082] Among them, the browser client 122 receives events through the portal module 1224. The portal module 1224 includes a peripheral capture thread, an audio capture and playback thread, and an image rendering thread. Among them, the peripheral capture thread is used to capture peripheral input events, such as keyboard input events, mouse input events, etc., and the audio capture and playback thread is used to capture audio input events.

[0083] The end-side protocol module 1222 can obtain the above events from the portal module 1224, then encode the events to obtain encoded events, and then send the encoded events to the desktop cloud server 14. Specifically, the end-side protocol module 1222 includes a transmission thread, a peripheral processing thread, and an audio processing thread. The transmission thread can be used to obtain events from the portal module 1224, the peripheral processing thread can be used to encode the events to obtain encoded events, and the transmission thread is also used to send the encoded events to the desktop cloud server 14.

[0084] The end-side protocol module 1222 may also include any one or more of an image processing thread and a video processing thread. Among them, the transmission thread is also used to obtain the encoded data output by the virtual desktop instance 142, and the encoded data can be encoded image data, encoded audio data, or encoded video data. The image processing thread is used to decode the encoded image data, and the audio processing thread is also used to decode the encoded audio data, and the video processing thread is used to decode the encoded video data.

[0085] The portal module 1224 can obtain image data from the end-side protocol module 1222 and perform image rendering through the image rendering thread, so as to present the virtual desktop on the desktop display main page. The portal module 1224 can also obtain audio data from the end-side protocol module 1222 and play the audio through the audio capture and playback thread. Similarly, the portal module 1224 can also obtain video data from the end-side protocol module 1222 and then play the video on the desktop display main page.

[0086] In some possible implementation manners, the end-side protocol module 1222 can be implemented through a bytecode specification, such as the WebAssembly bytecode specification. The end-side protocol module 1222 runs in the WebAssembly runtime environment and can natively support multi-threading. Therefore, the browser client 122 can create concurrent threads through the end-side protocol module 1222, such as concurrent video processing threads, image processing threads, audio processing threads, peripheral processing threads, and transmission threads, so as to concurrently execute any combination of the following multiple steps:

[0087] Encode the events to obtain encoded events;

[0088] Send the encoded events to the desktop cloud server 14;

[0089] Receive encoded data;

[0090] Decode the source data from the encoded data.

[0091] Through these multiple parallel threads, the desktop protocol operations can be decoupled, avoiding the impact of synchronization dependencies between desktop protocol operations on the real-time performance of desktop operations.

[0092] Furthermore, the threads can share memory. In this way, when the threads interact with data, they can directly interact with data through the shared memory, thereby reducing the number of data copies, reducing resource occupancy, and improving resource utilization. For example, the peripheral processing thread and the transmission thread can share memory. After the peripheral processing thread encodes the peripheral input event to obtain an encoded event, the transmission thread directly obtains the encoded event through the shared memory and transmits it to the desktop cloud server 14.

[0093] Similarly, the portal module 1224 and the end-side protocol module 1222 can also share memory. In this way, the number of data copies between the portal module 1224 and the end-side protocol module 1222 can be reduced, resource occupancy can be reduced, resource utilization can be improved, and the latency caused by data copying can be reduced.

[0094] Moreover, the end-side protocol module 1222 implemented based on bytecode specifications such as WebAssembly can execute computationally intensive tasks such as protocol encoding and decoding in the browser client 122 with performance close to that of binary executable programs, achieving encoding and decoding acceleration. Compared with end-side programs based on interpreted languages such as asm.js or JavaScript, the performance can be improved by several times or even more than 10 times. In this way, the number of browser clients 122 that the desktop cloud server 14 can serve is further increased, and the hardware cost is reduced.

[0095] In addition, the desktop cloud system 10 can use the efficient encoding and compression algorithms provided by the private protocol specification to compress events or source data to obtain data with a high compression ratio. Only less bandwidth resources are required for transmission between the terminal 12 (such as the browser client 122) and the desktop cloud server 14, improving the utilization efficiency of bandwidth resources. Thereby, the real-time performance of mouse and keyboard interaction can be ensured, and problems such as video image jitter and ghosting can be solved.

[0096] Considering the situation where the desktop cloud server 14 deploys multiple virtual desktop instances to serve multiple browser clients 122, the desktop cloud system 10 may further include a desktop gateway 18. The desktop gateway 18 is specifically used to route the encoded events of different browser clients 122 and route the encoded data of different virtual desktop instances 142.

[0097] An application layer long connection can be established between the browser client 122 and the desktop cloud server 14. This application layer long connection is used to transmit encoded events or encoded data. The desktop cloud server 14 does not perform encoding or decoding on the encoded data output by the virtual desktop instance 142, that is, there is no need for secondary transcoding, which increases the number of browser clients 122 that the desktop cloud server 14 can serve and reduces the hardware cost. Among them, the application layer long connection refers to a long connection established at the application layer, and this application layer long connection can specifically be a network socket (WebSocket) long connection.

[0098] As Figure 1 shown, when the desktop cloud system 10 further includes a desktop gateway 18, the application layer long connection established between the browser client 122 and the desktop cloud server 14 can be an application layer long connection established between the end-side protocol module 1222 and the desktop gateway 18. In this way, the end-side protocol module 1222 can send the encoded event to the desktop gateway 18 through the application layer long connection between the end-side protocol module 1222 and the desktop gateway 18, and then the desktop gateway 18 transmits the encoded event to the corresponding virtual desktop instance 142 through a connection with the corresponding virtual desktop instance 142, such as a user datagram protocol (UDP) connection or a transmission control protocol (TCP) connection. Similarly, the virtual desktop instance 142 can perform operations according to the encoded event, obtain the corresponding source data, and then encode the source data to output encoded data. The desktop cloud server 14 obtains the encoded data output by the virtual desktop instance and transmits the encoded data to the desktop gateway 18 through a connection with the desktop gateway 18, such as a TCP or UDP connection, and then the desktop gateway 18 transmits it to the end-side protocol module 1222 through the application layer long connection with the end-side protocol module 1222.

[0099] It should be noted that Figure 1 only one terminal 12 is shown. In some possible implementation manners, the desktop cloud system 10 may include multiple terminals 12, and the desktop cloud server 14 may provide services for multiple terminals 12. The embodiments of the present application do not limit this.

[0100] Figure 1 The architecture of the desktop cloud system 10 has been described in detail. To make the technical solutions of the present application clearer and easier to understand, the methods provided by the embodiments of the present application will be described in detail from the perspective of the desktop cloud system 10 below.

[0101] See Figure 2 the flowchart of the method for providing a virtual desktop shown, and this method includes:

[0102] S202: The terminal 12 receives an event.

[0103] The event can be a peripheral input event or an audio input event. Among them, the peripheral input events include keyboard input events, mouse input events, touch input events, stylus input events, etc., which will not be enumerated one by one here.

[0104] In specific implementation, the terminal 12 deploys a client for accessing the virtual desktop instance 142, such as the browser client 122. The browser client 122 can capture peripheral input events through the peripheral capture thread in the portal module 1224, and capture audio input events through the audio capture and playback thread. This will not be enumerated one by one here.

[0105] S204: The terminal 12 encodes the event to obtain an encoded event.

[0106] The end-side protocol module 1222 in the browser client 122 deployed by the terminal 12 can be implemented based on bytecode specifications such as WebAssembly. The terminal 12 can use the end-side protocol module 1222 implemented based on the bytecode specification to directly utilize a private desktop access protocol on the end side, specifically the algorithm provided by the desktop access protocol, to encode the event, thereby obtaining an encoded event. The encoded event can specifically be an encoded event with a high compression ratio, so as to reduce the consumption of bandwidth resources. Moreover, the end-side protocol module 1222 implemented based on the bytecode specification does not require an interpreter for interpretation and can encode the event with a performance close to that of a binary executable program, greatly improving the encoding efficiency.

[0107] It should be noted that the end-side protocol module 1222 can share memory with the portal module 1224. Based on this, the end-side protocol module 1222 can directly obtain the event through the shared memory, so as to reduce the number of copy operations and further reduce unnecessary overhead.

[0108] S206: The terminal 12 sends the encoded event to the desktop cloud server 14.

[0109] Specifically, the terminal 12 can send the encoded event to the desktop gateway 18 first through the application layer long connection between the end-side protocol module 1222 and the desktop gateway 18, such as a WebSocket long connection, and then the desktop gateway 18 sends the encoded event to the desktop cloud server 14 through the connection with the desktop cloud server 14, such as a TCP or UDP connection. Among them, the desktop gateway 18 can route the encoded event to the virtual desktop instance 142 corresponding to the terminal 12 deployed in the desktop cloud server 14.

[0110] S208: The desktop cloud server 14 operates the virtual desktop instance 142 according to the encoded event to obtain the encoded data output by the virtual desktop instance 142.

[0111] Specifically, the desktop cloud server 14 provides the encoding event to the corresponding virtual desktop instance 142. The virtual desktop instance 142 can decode the input event from the encoding event, such as a peripheral input event or an audio input event. The virtual desktop instance 142 can perform corresponding operations according to the input event to obtain source data, such as image data, audio data, or video data. The virtual desktop instance 142 can encode the source data to obtain encoded data and output the encoded data. The desktop cloud server 14 obtains the encoded data output by the virtual desktop instance.

[0112] It should be noted that when implementing the method for providing a virtual desktop according to the embodiments of the present application, the above S202 to S208 may not be executed either. For example, the desktop cloud server 14 may also directly obtain the encoded data output by the virtual desktop instance 142, and the embodiments of the present application do not limit this.

[0113] S210: The desktop cloud server 14 sends the encoded data to the terminal 12.

[0114] Specifically, the desktop cloud server 14 can send the encoded data to the terminal 12 through an application layer long connection between the desktop gateway 18 and the end - side protocol module 1222, such as a WebSocket long connection, for example, sending it to the end - side protocol module 1222 in the browser client 122 deployed on the terminal 12.

[0115] S212: The terminal 12 decodes the source data from the encoded data.

[0116] The end - side protocol module 1222 in the browser client 122 deployed on the terminal 12 can be implemented based on bytecode specifications such as WebAssembly. The terminal 12 can directly use a private desktop access protocol on the end - side through the end - side protocol module 1222 implemented based on the bytecode specification. Specifically, it is the algorithm provided by the desktop access protocol to decode the encoded data, thereby obtaining the source data. Moreover, the end - side protocol module 1222 implemented based on the bytecode specification does not require an interpreter for interpretation and can decode the encoded data with performance close to that of a binary executable program, greatly improving the decoding efficiency.

[0117] S214: The terminal 12 presents the virtual desktop according to the source data.

[0118] When the source data includes image data, the terminal can perform image rendering according to the image data to present the virtual desktop. Specifically, the portal module 1224 in the browser client 122 deployed on the terminal 12 can obtain the image data from the end - side protocol module 1222, and then perform image rendering through an image rendering thread, and display the rendered image on the main page of the desktop display, thereby presenting the virtual desktop.

[0119] When the output data includes video data, the terminal 12 can also perform video rendering based on the video data, so as to present a video picture. When the output data includes audio data, the terminal 12 can also play audio according to the audio data.

[0120] In Figure 2 the illustrated embodiment, the end-side protocol module 1222 can perform two-way data transmission. Specifically, the end-side protocol module 1222 can implement two-way data transmission through a transmission thread. Refer to Figure 3 the schematic diagram of the data transmission process shown. The transmission thread can include an uplink data transmission queue, a downlink data transmission queue, and a WebSocket client.

[0121] Specifically, the uplink data transmission queue obtains, through shared memory, the encoded events that need to be transmitted to the desktop cloud server 14 from input processing threads such as a peripheral processing thread and an audio processing thread. The encoded events include encoded keyboard input events, encoded mouse input events (mouse buttons, clicks), or encoded microphone audio input data.

[0122] The WebSocket client converts the underlying TCP / UDP communication requests corresponding to the uplink data (specifically the above-mentioned encoded events) generated by the end-side protocol module 1222 into WebSocket requests, interacts with the WebSocket listening port of the desktop gateway 18, and buffers the encoded data obtained from the desktop cloud server 14 into the downlink data transmission queue.

[0123] The downlink data transmission queue buffers the encoded data obtained from the desktop cloud server 144, including encoded image data, encoded video data, encoded audio data, etc., and then forwards it to output processing threads, such as an image processing thread, an audio processing thread, and a video processing thread, through shared memory. These output processing threads decode the data to obtain source data, such as any one or more of image data, audio data, and video data. Then, this thread interacts with the portal module 1224, and the portal module 1224 presents a virtual desktop on the desktop display main page according to the source data.

[0124] The content displayed on the desktop display main page comes from the end-side protocol module 1222. The desktop display main page of the portal module 1224 and the end-side protocol module 1222 interact with each other through shared memory. Refer to Figure 4 the schematic diagram of the process of the desktop display main page and the end-side protocol module 1222 interacting with each other shown, as Figure 4As shown in the figure, the desktop display main page asynchronously interacts with the end-side protocol module 1222 through shared memory. Specifically, the portal module 1224 and the end-side protocol module 1222 determine the time interval according to the set frame rate, and periodically check the flag bit to determine whether there is data refresh. If there is uplink data, the end-side protocol module 1222 sends the uplink data to the desktop cloud server 14 through the WebSocket client; if there is downlink data, the desktop display main page performs corresponding rendering or playing operations.

[0125] Furthermore, different types of data can have different latency-sensitive characteristics. For example, mouse input events are more sensitive to latency, while video data is less sensitive to latency. Based on this, when the desktop display main page and the end-side protocol module 1222 perform asynchronous interaction based on shared memory, the desktop display main page can update the display at different time intervals according to the latency-sensitive characteristics of various types of data, avoiding blocking the main thread of the browser client 122 and affecting the overall performance of the desktop cloud system 10.

[0126] The above embodiments introduce the method for providing a virtual desktop provided by the embodiments of the present application from the perspective of the desktop cloud system 10. Next, the method provided by the embodiments of the present application will be described in detail from the perspective of the terminal 12.

[0127] See Figure 5 the flowchart of the method for providing a virtual desktop shown in the figure. The method includes:

[0128] S502: The terminal 12 receives an event, encodes the event to obtain an encoded event, and sends the encoded event to the desktop cloud server 14.

[0129] The terminal 12 is deployed with a client for accessing the virtual desktop instance 142, such as the browser client 122. The terminal 12 can receive events through the portal module 1224 of the browser client 122. Specifically, the terminal 12 can capture peripheral input events, such as keyboard input events and mouse input events, through the peripheral capture thread in the portal module 1224, and the terminal 12 can also capture audio input events through the audio capture thread in the portal module 1224.

[0130] The terminal 12 can encode the event through the end-side protocol module 1222 of the browser client 122, such as encoding events such as keyboard input events, mouse input events, and audio input events, to obtain an encoded event.

[0131] In some possible implementation manners, an application layer long connection is established between the terminal 12 and the desktop cloud server 14, such as a WebSocket long connection. When the desktop cloud system 10 includes a desktop gateway 18, an application layer long connection can be established between the terminal 12 (specifically, the end-side protocol module 1222) and the desktop gateway 18, such as a WebSocket long connection. In this way, the end-side protocol module 1222 in the terminal 12 can send the above-mentioned encoded event through the WebSocket long connection with the desktop gateway 18, and then send the encoded event to the desktop cloud server 14 through the desktop gateway 18.

[0132] S504: The terminal 12 receives the encoded data returned by the desktop cloud server 14.

[0133] Among them, the encoded data is obtained by the desktop cloud server 14 operating on the virtual desktop instance 142 corresponding to the terminal 12 (specifically, the browser client 122 deployed on the terminal 12). The encoded data can specifically be encoded image data, audio data, and video data.

[0134] In some possible implementation manners, the encoded data is sent by the desktop cloud server 14 to the desktop gateway 18, and the terminal 12 can receive the above-mentioned encoded data through the WebSocket long connection with the desktop gateway 18.

[0135] S506: The terminal 12 decodes the source data from the encoded data and presents the virtual desktop according to the source data.

[0136] In some possible implementation manners, the terminal 12 can create multiple parallel threads through the end-side protocol module 1222, and then process any combination of the following multiple steps in parallel through the parallel threads:

[0137] Encoding the event to obtain an encoded event;

[0138] Sending the encoded event to the desktop cloud server;

[0139] Receiving the encoded data;

[0140] Decoding the source data from the encoded data.

[0141] Among them, the multiple parallel threads can share memory. In this way, data copying can be reduced and the efficiency of data processing can be improved. Further, the portal module 1224 in the browser client 122 can share memory with the end-side protocol module 1222. Correspondingly, the portal module 1224 can interact with the end-side protocol module 1222 for the source data in a shared memory manner, and after obtaining the source data, perform image rendering according to the source data such as image data, so as to present the virtual desktop.

[0142] In an embodiment of the present application, the browser client 122 based on WebAssembly can run in a sandbox environment controlled by the browser client 122 and cannot directly access device files. Therefore, hardware acceleration can be performed through the hardware acceleration support provided by the browser client 122 itself. For example, 2D (2 dimension) or 3D image rendering by a graphical processing unit (GPU) can be performed through WebGL.

[0143] In some possible implementation manners, acceleration services can also be provided at the operating system level. The service provides acceleration capabilities for WebAssembly programs (such as the end-side protocol module 1222) in the browser client 122 through means such as shared memory or WebSocket services. In this way, the operation of a large amount of data can be accelerated at one time, and the additional overhead caused by interaction with the service can be reduced.

[0144] As described above in conjunction with Figures 1 to 5 the desktop cloud system provided by the embodiments of the present application and the method for providing a virtual desktop based on the desktop cloud system have been introduced in detail. Next, the devices and equipment provided by the embodiments of the present application will be introduced.

[0145] An embodiment of the present application provides a device for providing a virtual desktop. The device includes modules for executing Figure 2 or Figure 5 the method steps executed by the terminal 12 in the illustrated embodiment. Another embodiment of the present application provides another device for providing a virtual desktop. The device includes modules for executing Figure 2 the method steps executed by the desktop cloud server 14 in the illustrated embodiment.

[0146] The device for providing a virtual desktop according to the embodiment of the present application can correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each module / unit of the device for providing a virtual desktop are respectively for implementing Figure 2 or Figure 5 the corresponding processes of each method in the illustrated embodiment. For the sake of brevity, they will not be elaborated here.

[0147] An embodiment of the present application also provides a desktop cloud server 14, which is specifically used to implement Figure 1 the functions of the desktop cloud server 14 in the desktop cloud system 10 shown. Next, the desktop cloud server 14 will be introduced in detail from the perspective of hardware implementation.

[0148] Figure 6 A schematic structural diagram of a desktop cloud server 14 is provided, as shown in Figure 6As shown in the figure, the desktop cloud server 14 includes a bus 601, a processor 602, a communication interface 603, and a memory 604. The processor 602, the memory 604, and the communication interface 603 communicate with each other through the bus 601.

[0149] The bus 601 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 6 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0150] The processor 602 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0151] The communication interface 603 is an input / output (I / O) device. The communication interface 603 is used for external communication. Specifically, the communication interface 603 can receive the encoded events sent by the terminal 12, or send the encoded data output by the virtual desktop instance 142 to the terminal 12 corresponding to the virtual desktop instance 142, and so on.

[0152] The memory 604 can include a volatile memory, such as a random access memory (RAM). The memory 604 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0153] The memory 604 stores executable program codes, and the processor 602 executes the executable program codes to execute the foregoing method for providing a virtual desktop.

[0154] The embodiment of the present application further provides a terminal 12, which is specifically used to implement Figure 1The functions of the terminal 12 in the desktop cloud system 10 shown below. Next, the terminal 12 will be introduced in detail from the perspective of hardware implementation.

[0155] Figure 7 A schematic structural diagram of the terminal 12 is provided, as Figure 7 shown. The terminal 12 includes a bus 701, a processor 702, a communication interface 703, and a memory 704. The processor 702, the memory 704, and the communication interface 703 communicate with each other through the bus 701.

[0156] The bus 701 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 7 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0157] The processor 702 can be any one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Microprocessor (MP), or a Digital Signal Processor (DSP), etc.

[0158] The communication interface 703 is an input / output device. The communication interface 703 is used for external communication. Specifically, the communication interface 703 can send encoded events to the desktop cloud server 14, or receive encoded data sent by the desktop cloud server 14, etc. Further, the communication interface 703 can also include a display. According to different manufacturing materials, the display can be divided into a Liquid Crystal Display (LCD), an Organic Light Emitting Diode (OLED) display, etc. The communication interface 903 can also include a microphone. Among them, the display can present a virtual desktop, and the microphone can receive audio input.

[0159] The memory 704 can include volatile memory, such as Random Access Memory (RAM). The memory 704 can also include non-volatile memory, such as Read-Only Memory (ROM), flash memory, a Hard Disk Drive (HDD), or a Solid State Drive (SSD).

[0160] The memory 704 stores executable program code, and the processor 702 executes the executable program code to execute the aforementioned method for providing a virtual desktop.

[0161] This embodiment of the present application also provides a virtual desktop instance 142, and the virtual desktop instance 142 is used to implement the functions of the virtual desktop instance 142 in the desktop cloud system 10 as Figure 1 shown.

[0162] An embodiment of the present application further provides a computer-readable storage medium, which includes instructions for instructing the desktop cloud server 14 to execute the method for providing a virtual desktop as described above.

[0163] An embodiment of the present application further provides a computer-readable storage medium, which includes instructions for instructing the terminal 12 to execute the method for providing a virtual desktop as described above.

[0164] An embodiment of the present application further provides a computer program product. When the computer program product is executed by a computer, the computer executes any one of the methods for providing a virtual desktop as described above. The computer program product can be a software installation package. In the case where any one of the methods for providing a virtual desktop as described above needs to be used, the computer program product can be downloaded and executed on the computer.

Claims

1. A desktop cloud system, characterized in that, the desktop cloud system includes a desktop cloud server and a terminal, and the desktop cloud server deploys virtual desktop instances; the terminal is used to encode events based on bytecode specifications and using a private desktop access protocol to obtain encoded events; the desktop cloud server is used to operate the virtual desktop instance according to the encoded events, obtain the encoded data output by the virtual desktop instance, and send the encoded data to the terminal corresponding to the virtual desktop instance; the terminal is further used to decode the source data from the encoded data using the bytecode specifications and present the virtual desktop according to the source data.

2. The system according to claim 1, characterized in that, the desktop cloud server does not encode or decode the encoded data output by the virtual desktop instance.

3. The system according to claim 1, characterized in that, the terminal processes in parallel any combination of the following multiple steps: encoding the event to obtain an encoded event; sending the encoded event to the desktop cloud server; receiving the encoded data; decoding the source data from the encoded data.

4. The system according to any one of claims 1 to 3, characterized in that, the event includes at least one of a mouse input event, a keyboard input event, and an audio input event.

5. The system according to any one of claims 1 to 3, characterized in that, the source data includes any one or more of image data, audio data, and video data.

6. The system according to any one of claims 1 to 3, characterized in that, the terminal deploys a browser client for accessing the virtual desktop instance.

7. A method for providing a virtual desktop, characterized in that, applied to a desktop cloud system, the desktop cloud system includes a desktop cloud server and a terminal, and the desktop cloud server deploys virtual desktop instances, and the method includes: the terminal encodes events based on bytecode specifications and using a private desktop access protocol to obtain encoded events; the desktop cloud server operates the virtual desktop instance according to the encoded events, obtains the encoded data output by the virtual desktop instance, and sends the encoded data to the terminal corresponding to the virtual desktop instance; the terminal decodes the source data from the encoded data using the bytecode specifications and presents the virtual desktop according to the source data.

8. The method according to claim 7, characterized in that, the method further includes: the desktop cloud server does not encode or decode the encoded data output by the virtual desktop instance.

9. The method according to claim 7, characterized in that, the terminal processes in parallel any combination of the following multiple steps: encoding the event to obtain an encoded event; sending the encoded event to the desktop cloud server; receiving the encoded data; decoding the source data from the encoded data.

10. The method according to any one of claims 7 to 9, characterized in that, the event includes at least one of a mouse input event, a keyboard input event, and an audio input event.

11. The method according to any one of claims 7 to 9, wherein, the source data includes any one or more of image data, audio data, and video data.

12. The method according to any one of claims 7 to 9, wherein, the terminal deploys a browser client for accessing the virtual desktop instance.

13. A desktop cloud server, wherein, the desktop cloud server is used to implement the functions of the desktop cloud server in the desktop cloud system according to any one of claims 1 to 6.

14. A terminal, wherein, the terminal is used to implement the functions of the terminal in the desktop cloud system according to any one of claims 1 to 6.

15. A desktop cloud server, wherein, the desktop cloud server includes a processor and a memory; the processor is configured to execute instructions stored in the memory, so that the desktop cloud server executes the method according to any one of claims 7 to 12.

16. A terminal, wherein, the terminal includes a processor and a memory; the processor is configured to execute instructions stored in the memory, so that the terminal executes the method according to any one of claims 7 to 12.

17. A computer-readable storage medium, wherein, it includes instructions that direct a computer to execute the method according to any one of claims 7 to 12.

18. A computer program product, wherein, when the computer program product runs on a computer, it causes the computer to execute the method according to any one of claims 7 to 12.

Images