Cloud desktop interaction method, device and equipment and computer readable storage medium

Abstract

The invention discloses a cloud desktop interaction method, device and equipment and a computer readable storage medium, and the method comprises the steps: installing a cloud desktop application integrating a plurality of cloud desktop protocols in an operation system of a cloud server, and carrying out the obtaining and coding of desktop data in the operation system through the cloud desktop application by employing a proper cloud desktop protocol, the cloud desktop protocol is not completed in a virtualization layer of a host machine, participation of bottom layer virtualization is not needed, out-of-band transmission of the cloud desktop protocol is changed into in-band transmission, the whole solution is decoupled from bottom layer virtualization, virtualization density of the host machine can be improved, great advantages are achieved for cloud server transformation of an existing network, the virtualization layer does not need to be changed, and implementation is easy. Only the cloud desktop application needs to be installed, so that the end-to-end technical scheme of the whole cloud desktop is more flexible and more convenient to replace, and the whole scheme is more portable.

Description

Technical Field

[0001] This application belongs to the technical field of cloud desktops, and particularly relates to a cloud desktop interaction method, device, equipment, and computer-readable storage medium. Background Art

[0002] A cloud desktop, also known as desktop virtualization or cloud computer, is a new mode that replaces traditional computers. The operating system of the cloud desktop runs in the cloud. Users use local soft terminals or thin terminals to access the operating system in the cloud through a specific communication protocol, achieving the same experience effect as using a local computer. When using a cloud desktop, all operations of the user locally are transmitted over the network to the operating system in the cloud, and the operating system responds by rendering the corresponding page images, which are then transmitted back to the user's local device over the network for decoding and display, making the user feel as if they are using a local computer. The most core technology of the cloud desktop lies in the network interaction between the local and the cloud, including video codec technology, etc.

[0003] The existing network interaction solutions adopted by cloud desktops usually fixedly use the end-to-end solutions of a certain manufacturer, and each manufacturer has a personalized solution, and even the protocol is closed-source. Therefore, once a solution is implemented, it is very difficult to replace it with the solutions of other manufacturers, or rather, the replacement cost is extremely high, resulting in the overall end-to-end technical solution of the cloud desktop being inflexible and difficult to replace, and the overall solution being relatively cumbersome. Summary of the Invention

[0004] The embodiments of this application provide a cloud desktop interaction method, device, equipment, and computer-readable storage medium, which can virtualize and shield the cloud desktop technology, enabling it to be independently disassembled, making the end-to-end technical solution of the cloud desktop more flexible, and improving the problems that the end-to-end technical solution of the cloud desktop is difficult to replace and the overall solution is relatively cumbersome.

[0005] In a first aspect, the embodiments of this application provide a cloud desktop interaction method, which is applied to a cloud server. A cloud desktop application is installed in the operating system of the cloud server. The cloud server conducts data interaction with a cloud terminal installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The method includes:

[0006] Determine a target cloud desktop protocol from multiple cloud desktop protocols in the cloud desktop application;

[0007] Obtain the desktop data of the cloud server through the cloud desktop application;

[0008] Encode the desktop data based on the codec of the target cloud desktop protocol;

[0009] Send the encoded desktop data to the cloud terminal.

[0010] Second aspect, an embodiment of the present application provides a cloud desktop interaction method, which is applied to a cloud terminal. A cloud desktop application is installed in the cloud terminal, and the cloud terminal interacts with a cloud server installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The method includes:

[0011] Receiving desktop data sent by the cloud server;

[0012] Decoding the desktop data through the codec of the target cloud desktop protocol in the cloud desktop application, where the target cloud desktop protocol is the encoding protocol adopted by the desktop data, and the target cloud desktop protocol belongs to multiple cloud desktop protocols;

[0013] Displaying the decoded desktop data.

[0014] Third aspect, an embodiment of the present application provides a cloud desktop interaction device, which is applied to a cloud server. A cloud desktop application is installed in the operating system of the cloud server, and the cloud server interacts with a cloud terminal installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The device includes:

[0015] A protocol selector module for determining a target cloud desktop protocol from multiple cloud desktop protocols of the cloud desktop application;

[0016] A data acquisition module for acquiring desktop data of the cloud server through the cloud desktop application;

[0017] A codec module for encoding the desktop data based on the codec of the target cloud desktop protocol;

[0018] A sending module for sending the encoded desktop data to the cloud terminal.

[0019] Fourth aspect, an embodiment of the present application provides a cloud desktop interaction device, which is applied to a cloud terminal. A cloud desktop application is installed in the cloud terminal, and the cloud terminal interacts with a cloud server installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The device includes:

[0020] A receiving module for receiving desktop data sent by the cloud server;

[0021] A codec module for decoding the desktop data through the codec of the target cloud desktop protocol in the cloud desktop application, where the target cloud desktop protocol is the encoding protocol adopted by the desktop data, and the target cloud desktop protocol belongs to multiple cloud desktop protocols;

[0022] A display module for displaying the decoded desktop data.

[0023] Fifth aspect, an embodiment of the present application provides an electronic device, which includes: a processor and a memory storing computer program instructions;

[0024] When the processor executes the computer program instructions, the cloud desktop interaction method described in the first aspect or the second aspect is implemented.

[0025] Sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the cloud desktop interaction method described in the first aspect or the second aspect is implemented.

[0026] In the cloud desktop interaction method, device, equipment and computer-readable storage medium of the embodiments of the present application, a cloud desktop application integrating multiple cloud desktop protocols is installed in the operating system of the cloud server. The cloud desktop application uses an appropriate cloud desktop protocol to obtain and encode desktop data, rather than completing it at the virtualization layer of the host computer, without the participation of the underlying virtualization. The out-of-band transmission of the cloud desktop protocol is changed to in-band transmission, so that the entire solution is decoupled from the underlying virtualization, which is beneficial to improving the virtualization density of the host computer. It has great advantages for the transformation of the existing network cloud server. There is no need to modify the virtualization layer, only need to install the cloud desktop application, which makes the overall cloud desktop end-to-end technical solution more flexible, easier to replace, and the overall solution is more lightweight. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0028] Figure 1 It is a schematic flowchart of the cloud desktop interaction method provided by an embodiment of the present application;

[0029] Figure 2 It is a schematic flowchart of the cloud desktop interaction method provided by an embodiment of the present application;

[0030] Figure 3 It is a schematic logical diagram of the cloud desktop interaction method provided by an embodiment of the present application;

[0031] Figure 4 It is a schematic diagram of the cloud desktop interaction device provided by an embodiment of the present application;

[0032] Figure 5 It is a schematic diagram of the cloud desktop interaction device provided by an embodiment of the present application;

[0033] Figure 6It is a schematic structural diagram of an xx device provided by another embodiment of the present application. Detailed implementation manners

[0034] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than limiting the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0035] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, the elements defined by the statement "comprising..." do not exclude the existence of additional identical elements in the process, method, article or device comprising the said elements.

[0036] Before elaborating on the technical solutions provided by the embodiments of the present application, for the convenience of understanding the embodiments of the present application, the present application first specifically describes the problems existing in the prior art:

[0037] In the existing interaction solutions for cloud desktops, basically a certain end-to-end solution is fixedly adopted. Once this solution is implemented, it is very difficult to replace it with the solutions of other manufacturers, or rather, the cost is extremely high, because the entire solution involves virtualization, remote desktop display technology, USB peripheral adaptation technology, end-side application technology, etc. And each manufacturer has a personalized solution, and even the protocol is closed source, the peripheral adaptation technology is uneven, virtualization is not unified, and there is also a strongly coupled part between remote desktop display and virtualization technology, resulting in a relatively high coupling degree between the above-mentioned several major technologies and being unable to be separated independently. Once a cloud desktop solution of a certain manufacturer is implemented, another manufacturer cannot coexist and is completely incompatible, lacking flexibility and having an extremely high replacement cost.

[0038] For example, the commonly used end-to-end solutions in cloud desktops currently mainly include three solutions: spice, VMware, and Citrix. Among them, in the end-to-end cloud desktop solution of spice, the transmission protocol is the open-source spice protocol, the peripheral adaptation is the usbredir technology stack, the virtualization is KVM-QEMU, and the desktop display encoding and decoding are strongly coupled with virtualization and cannot be split independently. In the end-to-end cloud desktop solution of VMware, the virtualization is the self-developed vSphere solution of VMware, the remote desktop display is the Blast closed-source protocol technology, and the entire technology stack, including the USB peripheral adaptation technology, is closed-source and is based on the bare metal architecture and is completely non-splittable. In the end-to-end cloud desktop solution of Citrix, the virtualization is the Xen series technical solution of Citrix, the remote desktop display protocol is HDX-ICA, which is completely closed-source, and the peripheral adaptation is also self-developed and completely closed-source. The overall virtualization technology is also based on the bare metal architecture and is non-splittable.

[0039] From the above cloud desktop technical solutions, it can be seen that the cloud desktop involves a relatively large number of technology stacks, and all of the above solutions belong to highly coupled solutions. All components cannot be reused, manufacturers are completely incompatible with each other, shield each other, and are very cumbersome. The usage and replacement costs are very high.

[0040] This application aims to solve the shortcomings of the above solutions, completely shield virtualization from the entire cloud desktop technology, completely decouple it, make it independently disassemblable, and achieve the purpose of cross-platform.

[0041] The inventor of this application found that the ultimate interaction of the cloud desktop is between the cloud terminal such as the user's local soft terminal or thin terminal and the operating system windows of the cloud server. There are no other communication interaction parties except this. Therefore, independent processes can be deployed in the operating systems of the cloud terminal and the cloud server respectively for communication, and video codec data and USB peripheral data can be interacted, completely decoupled from other technologies such as underlying virtualization.

[0042] Based on this, the embodiments of this application provide a cloud desktop interaction method, device, equipment, and storage medium. Cloud desktop applications including codecs of multiple cloud desktop protocols are respectively installed in the operating systems of the cloud terminal and the cloud server, and the interaction between the cloud terminal and the cloud server is realized based on the cloud desktop application.

[0043] The cloud desktop interaction method provided by the embodiments of this application can be applied to the cloud desktop scenario. In the cloud desktop scenario, the cloud terminal and the cloud server can perform data interaction based on the cloud desktop interaction method provided by the embodiments of this application.

[0044] See Figure 1, which is a schematic flowchart of a cloud desktop interaction method provided by an embodiment of the present application. This method can be applied to a cloud desktop scenario including a cloud terminal and a cloud server. Among them, cloud desktop applications are installed in the operating systems of both the cloud terminal and the cloud server, and the cloud desktop applications include codecs for multiple cloud desktop protocols, such as Figure 1 As shown, the method may include the following steps S11 - S11, which will be specifically described below.

[0045] S11. The cloud server determines a target cloud desktop protocol from multiple cloud desktop protocols of the cloud desktop application.

[0046] In some embodiments of the present application, a cloud desktop application can be developed in advance. The application can integrate cloud desktop protocols commonly used in the cloud desktop scenario. For example, codecs for Blast, HDX - ICA, Spice or other multiple protocols can be pre - encapsulated in the cloud desktop application. Among them, the closed - source Blast and HDX - ICA can be encapsulated by packaging the official SDK (Software Development Kit), and Spice can be implemented by referring to the open - source protocol definition.

[0047] In the cloud desktop scenario, the above - mentioned cloud desktop applications can be installed in the cloud terminal and the cloud server that need to perform data interaction respectively. Among them, the cloud server can pre - install the cloud desktop application when building the image.

[0048] For users, a cloud terminal can be adapted to all cloud desktop protocols on the market. Therefore, the cloud desktop application installed on the cloud terminal completely shields the underlying differences for users and device manufacturers, enabling transparent and seamless access.

[0049] For the operating system of the cloud server, it only needs to pre - install the above - mentioned cloud desktop application in the windows operating system. The cloud desktop application communicates with the standard windows operating system interface to obtain the data that needs to be interacted. Moreover, the cloud desktop application has built - in modules adapted to all cloud desktop protocols on the market, enabling transparent adaptation and seamless access, completely shielding the differences of underlying virtualization, and promoting the adaptation function to be completed within the operating system, achieving standardization. Therefore, for the cloud desktop application installed in the cloud server, it docks with multiple standard cloud desktop protocols on one side and the standard windows operating system on the other side.

[0050] In some embodiments of the present application, the cloud server and the cloud terminal can perform data interaction based on the cloud desktop applications installed on both. For the cloud desktop applications installed on the cloud server and the cloud terminal, when the cloud desktop application is started, the user can select the target cloud desktop protocol for communication between the cloud terminal and the cloud server from various cloud desktop protocols integrated in the cloud desktop application by themselves, or the cloud desktop application can adaptively select the target cloud desktop protocol based on the network conditions between the cloud terminal and the cloud server.

[0051] In some embodiments of the present application, a selector for adaptively selecting a target cloud desktop protocol from various cloud desktop protocols can also be integrated in the cloud desktop application. The user can customize their protocol selection logic in the selector according to their own business needs, so as to use the selector to select the target cloud desktop protocol from various cloud desktop protocols based on the protocol selection logic.

[0052] In some embodiments of the present application, when the cloud server determines the target cloud desktop protocol from various cloud desktop protocols of the cloud desktop application, the following steps may be included:

[0053] Determine the network quality information between the cloud server and the cloud terminal;

[0054] Determine the target cloud desktop protocol from various cloud desktop protocols based on the network quality information.

[0055] In some embodiments of the present application, the network quality between the cloud server and the cloud terminal can be detected through the transmission of PING heartbeat packets, and then the network quality information can be obtained. Specifically, when the cloud server determines the network quality information between the cloud server and the cloud terminal, the following steps may be included:

[0056] Circularly send heartbeat packets to the cloud terminal;

[0057] Determine the number of responses and / or response delay of the cloud terminal to the heartbeat packets within the detection period;

[0058] Based on the number of responses and / or response delay, determine the packet loss rate and / or average network delay within the detection period;

[0059] Take the packet loss rate and / or average network delay as the network quality information between the cloud server and the cloud terminal.

[0060] Wherein, the network delay refers to the RTT delay, that is, the time elapsed from when the sender sends data until the sender receives the confirmation message from the receiver.

[0061] Correspondingly, determining the target cloud desktop protocol from various cloud desktop protocols based on the network quality information may include:

[0062] Determine a target cloud desktop protocol from multiple cloud desktop protocols based on the packet loss rate and / or the average network latency.

[0063] In some embodiments of the present application, the selection logic for determining the target cloud desktop protocol from multiple cloud desktop protocols based on the packet loss rate and / or the average network latency can be set by the user according to actual needs. For example, the selection logic may include:

[0064] When 10% < packet loss rate < 20%, select a cloud desktop protocol that supports UDP at the transport layer, such as HDX, Blast;

[0065] When 0% < packet loss rate < 10%, select a cloud desktop protocol that supports UDP at the transport layer, is transformed based on QUIC, optimizes congestion control, and has FEC forward error correction, such as HDX, Blast;

[0066] When the latency is higher than 100 ms, select a cloud desktop protocol with a bitrate adaptive adjustment function, such as HDX, Blast;

[0067] The above is for reference only. In actual use, continuously accumulate more cloud desktop protocols from various manufacturers according to the network quality to meet various scenarios in a complex public network environment.

[0068] The above is the process of selecting the target desktop protocol through the cloud server. Similarly, the target cloud desktop protocol can also be selected through the cloud terminal. The specific selection method is the same as that used by the cloud server. To avoid repetition, it will not be elaborated here.

[0069] After determining the target cloud desktop protocol, data interaction can be performed between the cloud server and the cloud terminal based on the target cloud desktop protocol.

[0070] S12. The cloud server obtains the desktop data of the cloud server through the cloud desktop application installed in the operating system.

[0071] In some embodiments of the present application, the cloud server can obtain the desktop data that needs to be transmitted to the cloud terminal by means of screen capture or the like through the cloud desktop application installed in the operating system, where the desktop data may include data such as the desktop image and desktop video of the cloud server.

[0072] S13. The cloud server encodes the desktop data based on the codec of the target cloud desktop protocol.

[0073] In some embodiments of the present application, after obtaining the desktop data, in order to reduce the transmission space of the desktop data and improve the reliability and security of the desktop data, the desktop data is encoded using the codec corresponding to the target cloud desktop protocol in the cloud desktop application to obtain the encoded desktop data.

[0074] S14. The cloud server sends the encoded desktop data to the cloud terminal.

[0075] S15. The cloud terminal receives the desktop data sent by the cloud server.

[0076] S16. The cloud terminal decodes the desktop data through the codec of the target cloud desktop protocol in the cloud desktop application.

[0077] In some embodiments of the present application, a cloud desktop application integrating multiple cloud desktop protocols is also installed in the cloud terminal. Based on this, after receiving the desktop data sent by the cloud server, the cloud terminal can use the codec corresponding to the target cloud desktop protocol in the cloud desktop application to decode the received desktop data, so as to obtain the original desktop data.

[0078] S17. The cloud terminal displays the decoded desktop data.

[0079] So far, the interaction of desktop data between the cloud server and the cloud terminal is completed.

[0080] The cloud desktop interaction method provided by the embodiments of the present application installs a cloud desktop application integrating multiple cloud desktop protocols in the operating system of the cloud server. Through the cloud desktop application, the appropriate cloud desktop protocol is used to obtain and encode the desktop data within the operating system, rather than being completed in the virtualization layer (also known as the hypervisor layer) of the host computer. Without the participation of the underlying virtualization, that is, changing the original out-of-band transmission in which the hypervisor layer communicates with the cloud terminal to in-band transmission in which the application layer of the operating system communicates with the cloud terminal, making the entire solution decoupled from the underlying virtualization, which is beneficial to improving the virtualization density of the host computer. It has great advantages for the transformation of the existing network cloud servers. There is no need to modify the virtualization layer, only need to install the cloud desktop application, making the overall cloud desktop end-to-end technical solution more flexible, more convenient to replace, and the overall solution more lightweight.

[0081] In some embodiments of the present application, before determining the network quality information between the cloud server and the cloud terminal, a communication channel between the cloud server and the cloud terminal can be established first, and heartbeat packets are transmitted based on the established communication channel, so as to determine the network quality information between the cloud server and the cloud terminal.

[0082] In some embodiments of the present application, when both the cloud server and the cloud terminal are behind a NAT (Network Address Translation) device, when establishing a communication channel between the cloud server and the cloud terminal, an intranet penetration technology can be used to establish a P2P channel between the cloud server and the cloud terminal, where the P2P channel refers to a communication channel used to implement P2P communication between the cloud server and the cloud terminal, and P2P communication is also known as peer-to-peer communication.

[0083] In some embodiments of the present application, the cloud terminal corresponds to the first intranet address, the cloud server corresponds to the second intranet address, and both the cloud terminal and the cloud server communicate with the index server through corresponding NAT devices, wherein the index server may be an index node of the Linux system. Based on this, the intranet penetration technology is used to establish a P2P channel between the cloud server and the cloud terminal, which may include the following steps:

[0084] The cloud terminal sends the first intranet endpoint information to the index server through the corresponding NAT device, where the first intranet endpoint information includes the first intranet address;

[0085] The cloud server sends the second intranet endpoint information to the index server through the corresponding NAT device, where the second intranet endpoint information includes the second intranet address;

[0086] The index server records the first intranet endpoint information of the cloud terminal, and also records the first public network endpoint information of the cloud terminal observed by itself. Similarly, the index server also records the second intranet endpoint information of the cloud server and the second public network endpoint information of the observed cloud server, wherein the first public network endpoint information and the second public network endpoint information may include the corresponding public network IP address, public network port and other information;

[0087] The cloud terminal or cloud server can send a p2p connection request to the index server to request the establishment of a P2P channel between the cloud terminal and the cloud server;

[0088] In response to the received p2p connection request, the index server sends the second intranet endpoint information and the second public network endpoint information of the cloud server recorded by it to the cloud terminal, and sends the first intranet endpoint information and the first public network endpoint information of the cloud terminal to the cloud server;

[0089] Based on the relevant information received from each other, the cloud terminal and the cloud server send data packets to the endpoint of the other party's NAT device on the public network, thereby opening the connection between the cloud terminal and the cloud server, that is, opening the "hole" between the cloud terminal and the cloud server to achieve intranet penetration. In this way, the cloud terminal and the cloud server send data to each other's public network endpoint, which is equivalent to directly sending UDP data packets to each other's client;

[0090] Once the application confirms that it can send data packets to the other party's public network endpoint and let the data packets reach the destination application behind NAT, the program will automatically stop sending data packets used to "punch holes" and start real P2P data transmission, that is, completing the establishment of the P2P channel between the cloud terminal and the cloud server.

[0091] Through the above method, the cloud operating system side no longer allocates public network to communicate with terminal applications. Instead, the NAT P2P method is adopted to penetrate the intranet, so that the cloud terminal and cloud server operating system applications can communicate directly in the intranet. When the initial communication is carried out through the INDEX public network node in the middle, the link is opened and the network transmission is also directly connected without passing through the intermediate node, avoiding the gateway hardware equipment in front of the cloud desktop system, reducing costs, reducing dependence, and solving the problem of tight public network IP resources.

[0092] In some embodiments of the present application, before using the intranet penetration technology to establish a P2P channel between the cloud terminal and the cloud server, it is also possible to first determine whether the cloud terminal and the cloud server are in a public cloud scenario. When it is determined that they are in a public cloud scenario, the intranet penetration technology is then used to establish a P2P channel between the cloud terminal and the cloud server. When the cloud terminal and the cloud server are in a local area network scenario, a direct connection can be achieved directly through the intranet IP address without the need for a NAT P2P penetration solution. However, for home users and C-end users, in public cloud application scenarios, NAT P2P penetration must be used to achieve direct access to the intranet, thereby improving availability and reducing public network resource costs.

[0093] In some embodiments of the present application, the cloud desktop application also includes a USB / IP protocol. Based on this, the cloud terminal can be used as a USB / IP server and the cloud server can be used as a USB / IP client, so that the cloud terminal and the cloud server can exchange USB peripheral data based on the USB / IP protocol. Figure 2 The USB peripheral data interaction between the cloud server and the cloud terminal may include the following steps S21-S25, which are described in detail below.

[0094] S21. The cloud terminal cyclically monitors the local USB interface insertion status to determine whether a USB device is inserted into the USB interface.

[0095] In some embodiments of the present application, the USB / IP protocol is a technology unrelated to the cloud desktop protocol used for desktop data interaction. The USB / IP protocol is used for USB peripheral data interaction, so that the USB peripheral transmission channel and the cloud desktop channel are independent, which can achieve platform independence and ensure that desktop data interaction and USB peripheral data interaction do not affect each other.

[0096] In some embodiments of the present application, when performing USB peripheral data interaction using the USB / IP protocol, the built-in peripheral adaptation functions of cloud desktop protocols such as USBredir, VMware, Citrix, etc. need to be turned off and not enabled. Instead, the native USB / IP protocol of the Linux kernel is used for USB peripheral data interaction. USB / IP belongs to the technical solution of USB over net and has the advantage of being cross-platform, having nothing to do with the underlying operating system, virtualization platform, etc.

[0097] In some embodiments of the present application, in order to enable end-to-end network communication using the USB / IP protocol, a USB / IP kernel module can be built into the operating systems of the cloud terminal and the cloud server. The USB / IP kernel module has the function of parsing and mapping from the hardware layer for reading USB peripherals to the upper-layer application layer protocol.

[0098] When performing USB peripheral data interaction using the USB / IP protocol, the cloud terminal is used as a USB / IP server, and the cloud terminal can continuously monitor the insertion situation of the local USB interface and support hot plugging.

[0099] S22. When the cloud terminal monitors that a USB device is inserted into the USB interface, it sends the USB peripheral data corresponding to the USB device to the cloud server based on the USB / IP protocol.

[0100] In some embodiments of the present application, when the cloud terminal monitors that a USB device is inserted into the USB interface, a device access permission application is popped up. After the user authorizes, the application layer of the cloud terminal reads the device list and passes the device list data to the native layer. The native layer reads the actual data of the inserted USB device, that is, the USB peripheral data, through the libusb driver, and encapsulates the data into the USB / IP protocol for network transmission. Among them, the native layer is the local service layer of the cloud terminal, mainly including some local services and some link libraries, etc. Libusb is a library written in C, which provides a general access method for USB devices. Applications can easily access USB devices through libusb without writing USB device driver programs.

[0101] S23. The cloud server receives the USB peripheral data sent by the cloud terminal through the cloud desktop application.

[0102] S24. The cloud server parses the USB peripheral data based on the USB / IP protocol to obtain the corresponding USB event.

[0103] S25. The cloud server controls the operating system of the cloud server to execute the operation corresponding to the USB event.

[0104] In some embodiments of the present application, after receiving the USB peripheral data sent by the cloud terminal, the cloud server parses it according to the USB / IP protocol. After parsing, it interacts with the operating system through the libusb driver, and the operating system responds to the USB event.

[0105] In the above manner, network transmission is adopted end-to-end, shielding the adaptation of all hardware virtualization layers. On the cloud terminal, data can be read through the standard libusb, encapsulated into the USB / IP protocol and then transmitted to the operating system of the cloud server through the network. In the operating system of the cloud server, the data is parsed through the USB / IP protocol and converted into the standard interface of the Windows USB peripheral to request the operating system to perform corresponding actions, completing the recognition and parsing of the entire peripheral. The above entire process performs the interactive transmission of USB peripheral data through the USB / IP protocol, which is completely decoupled from the cloud desktop protocol and is not restricted by the technology of the cloud desktop. In the context of the overall replacement of the cloud desktop capabilities, the original USB peripheral functions will not be affected.

[0106] See Figure 3 , which is a logical schematic diagram of the cloud desktop interaction method provided by the embodiments of the present application. Here, the cloud terminal uses the Android operating system and the cloud server uses the Windows operating system as an example. As Figure 3 shown, between the cloud terminal and the cloud server, a P2P channel is established through the NAT intranet penetration solution by the index node to achieve P2P direct connection. Cloud desktop applications are installed on both the cloud terminal and the cloud server. The cloud desktop application integrates mainstream desktop protocols such as Blast, HDX-ICA, Spice, etc. and the USB / IP function, and has a Selector protocol selector. The Selector protocol selector can adaptively select the protocol based on network quality detection. When performing cloud desktop interaction, through the built-in protocol selector Selector in the cloud desktop application, the target cloud desktop protocol is selected from multiple cloud desktop protocols built into the cloud desktop application. The cloud server uses the cloud desktop application to capture desktop data and perform encoding, decoding, and transmission of the desktop data based on the cloud desktop protocol, thereby realizing the interaction of the desktop data. In this way, the original out-of-band transmission of desktop data interaction with the cloud terminal through the hypervisor layer is moved to the application layer to complete and changed to in-band transmission. In addition, the cloud desktop application also integrates a cross-platform protocol for USB peripherals, that is, the USB / IP protocol. Between the cloud terminal and the cloud server, the USB / IP protocol is used to perform the interaction of USB peripheral data through the cloud desktop application. In this way, the protocol used for USB peripheral data interaction is completely decoupled from the cloud desktop protocol used for desktop data interaction and is not restricted by the technology of the cloud desktop. In the context of the overall replacement of the cloud desktop capabilities, the original USB peripheral functions will not be affected.

[0107] The above end-to-end solution solves the problem of cross-platform cloud desktops under the VDI (Virtual Desktop Infrastructure) architecture. Users can flexibly switch cloud desktop protocols, and only need to complete the corresponding protocol integration within the operating system image of the cloud server.

[0108] Based on the cloud desktop interaction method provided in the above embodiments, correspondingly, the present application also provides a specific implementation manner of the cloud desktop interaction device. Please refer to the following embodiments.

[0109] See Figure 4 , which is a schematic diagram of the cloud desktop interaction device provided in the embodiments of the present application. The cloud desktop interaction device 400 is applied to a cloud server. The operating system of the cloud server is installed with a cloud desktop application. The cloud server interacts with a cloud terminal installed with a cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols, such as Figure 4 As shown, the device 400 may include the following modules:

[0110] A protocol selector module 401, configured to determine a target cloud desktop protocol from multiple cloud desktop protocols of the cloud desktop application;

[0111] A data acquisition module 402, configured to acquire desktop data of the cloud server through the cloud desktop application;

[0112] An encoding and decoding module 403, configured to encode the desktop data based on the codec of the target cloud desktop protocol;

[0113] A sending module 404, configured to send the encoded desktop data to the cloud terminal.

[0114] The cloud desktop interaction device provided in the embodiments of the present application installs and integrates a cloud desktop application with multiple cloud desktop protocols in the operating system of the cloud server. Through the cloud desktop application, it uses an appropriate cloud desktop protocol to acquire and encode desktop data within the operating system, rather than completing it at the virtualization layer of the host. Without the participation of the underlying virtualization, it changes the out-of-band transmission of the cloud desktop protocol to in-band transmission, decoupling the entire solution from the underlying virtualization, which is beneficial to improving the virtualization density of the host. For the transformation of existing network cloud servers, it has great advantages. There is no need to modify the virtualization layer, and only the cloud desktop application needs to be installed, making the overall end-to-end technical solution of the cloud desktop more flexible, easier to replace, and the overall solution more lightweight.

[0115] In some embodiments of the present application, the protocol selector module 401 includes:

[0116] A network quality detection unit, configured to determine network quality information between the cloud server and the cloud terminal;

[0117] A protocol determination unit determines a target cloud desktop protocol from multiple cloud desktop protocols based on network quality information.

[0118] In some embodiments of the present application, the network quality detection unit is specifically configured to:

[0119] Send heartbeat packets to the cloud terminal in a loop;

[0120] Determine the number of responses and / or response latency of the cloud terminal to the heartbeat packets within a detection period;

[0121] Based on the number of responses and / or response latency, determine the packet loss rate and / or average network latency within the detection period;

[0122] Use the packet loss rate and / or average network latency as the network quality information between the cloud server and the cloud terminal;

[0123] Determining a target cloud desktop protocol from multiple cloud desktop protocols based on network quality information includes:

[0124] Based on the packet loss rate and / or average network latency, determine a target cloud desktop protocol from multiple cloud desktop protocols.

[0125] In some embodiments of the present application, the apparatus 400 may further include: a channel construction module, configured to establish a P2P channel with the cloud terminal using the intranet penetration technology before determining the network quality information between the cloud server and the cloud terminal;

[0126] Correspondingly, the network quality detection unit is configured to:

[0127] Based on the P2P channel, determine the network quality information between the cloud server and the cloud terminal.

[0128] In some embodiments of the present application, the cloud desktop application further includes a USB / IP protocol, the cloud terminal serves as a USB / IP server, and the cloud server serves as a USB / IP client. The apparatus may further include:

[0129] A receiving module, configured to receive USB peripheral data sent by the cloud terminal through the cloud desktop application;

[0130] A parsing module, configured to parse the USB peripheral data based on the USB / IP protocol to obtain corresponding USB events;

[0131] A control module, configured to control the operating system of the cloud server to perform operations corresponding to the USB events.

[0132] The above cloud desktop interaction apparatus provided by the embodiments of the present application can implement Figures 1 to 3 each process implemented by the cloud server in the method embodiments. To avoid repetition, it will not be elaborated here.

[0133] See Figure 5 , which is a schematic diagram of the cloud desktop interaction device provided by the embodiment of the present application. The cloud desktop interaction device 500 is applied to a cloud terminal. A cloud desktop application is installed in the cloud terminal. The cloud terminal exchanges data with a cloud server installed with a cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols, such as Figure 5 As shown, the device 500 may include the following modules:

[0134] A receiving module 501, configured to receive desktop data sent by the cloud server;

[0135] A codec module 502, configured to decode the desktop data through the codec of the target cloud desktop protocol in the cloud desktop application. The target cloud desktop protocol is the encoding protocol adopted by the desktop data, and the target cloud desktop protocol belongs to multiple cloud desktop protocols;

[0136] A display module 503, configured to display the decoded desktop data.

[0137] For the cloud desktop interaction device provided by the embodiment of the present application, a cloud desktop application integrating multiple cloud desktop protocols is installed in the cloud terminal. Through the cloud desktop application, appropriate cloud desktop protocols are used to interact with the operating system of the cloud server for desktop data, rather than being completed at the virtualization layer of the host computer, without the participation of the underlying virtualization. The out-of-band transmission of the cloud desktop protocol is changed to in-band transmission, so that the entire solution is decoupled from the underlying virtualization, which is beneficial to improving the virtualization density of the host computer. It has great advantages for the transformation of the existing network cloud server. There is no need to modify the virtualization layer, only need to install the cloud desktop application, making the overall cloud desktop end-to-end technical solution more flexible, more convenient to replace, and the overall solution more lightweight.

[0138] In some embodiments of the present application, the device 500 may further include: a channel construction module, configured to use the intranet penetration technology to establish a P2P channel with the cloud server before receiving the desktop data sent by the cloud server;

[0139] Correspondingly, the receiving module 501 is specifically configured to:

[0140] Based on the P2P channel, receive the desktop data sent by the cloud server.

[0141] In some embodiments of the present application, the cloud desktop application further includes a USB / IP protocol. The cloud terminal serves as a USB / IP server, and the cloud server serves as a USB / IP client. The device 500 may further include:

[0142] A listening module, configured to circularly monitor the insertion situation of the local USB interface to determine whether a USB device is inserted into the USB interface;

[0143] A sending module, configured to send USB peripheral data corresponding to a USB device to a cloud server based on the USB / IP protocol when it is monitored that a USB device is inserted into the USB interface.

[0144] The above cloud desktop interaction device provided by the embodiments of the present application can implement Figures 4 to 6 each process implemented by the cloud terminal in the method embodiments. To avoid repetition, details are not described here.

[0145] Figure 6 The figure shows a schematic hardware structure diagram of an electronic device provided by the embodiments of the present application.

[0146] The electronic device 600 may include a processor 601 and a memory 602 storing computer program instructions.

[0147] Specifically, the above processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0148] The memory 602 may include a mass storage for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 602 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, the memory 602 is a non-volatile solid state memory. The memory 602 may include a read only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, in general, the memory 602 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the cloud desktop interaction methods in the above embodiments.

[0149] The processor 601 reads and executes the computer program instructions stored in the memory 602 to implement any of the cloud desktop interaction methods in the above embodiments.

[0150] In one example, the electronic device 600 may further include a communication interface 603 and a bus 610. Among them, as Figure 6 shown, the processor 601, the memory 602, and the communication interface 603 are connected through the bus 610 to complete communication with each other.

[0151] The communication interface 603 is mainly used to implement communication between various modules, devices, units, and / or devices in the embodiments of the present application.

[0152] The bus 610 includes hardware, software, or both, and couples the components of the online data flow metering device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. In a suitable case, the bus 610 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0153] In addition, in combination with the cloud desktop interaction method in the above embodiments, the embodiments of the present application may provide a computer storage medium to implement. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by the processor, any one of the cloud desktop interaction methods in the above embodiments is implemented.

[0154] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0155] The functional blocks shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments for performing the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave. A "machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0156] It should also be noted that in the exemplary embodiments mentioned in the present application, some methods or systems are described based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0157] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It should also be understood that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware for performing the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0158] As described above, this is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. A cloud desktop interaction method, characterized in that, Applied to a cloud server, a cloud desktop application is installed in the operating system of the cloud server. The cloud server interacts with a cloud terminal installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The method includes: Determine a target cloud desktop protocol from multiple cloud desktop protocols in the cloud desktop application; Obtain the desktop data of the cloud server through the cloud desktop application; Encode the desktop data based on the codec of the target cloud desktop protocol; Send the encoded desktop data to the cloud terminal.

2. The method according to claim 1, characterized in that, The determining a target cloud desktop protocol from multiple cloud desktop protocols in the cloud desktop application includes: Determine the network quality information between the cloud server and the cloud terminal; Based on the network quality information, determine a target cloud desktop protocol from the multiple cloud desktop protocols.

3. The method according to claim 2, characterized in that, The determining the network quality information between the cloud server and the cloud terminal includes: Circularly send heartbeat packets to the cloud terminal; Determine the number of responses and / or response latency of the cloud terminal to the heartbeat packets within a detection period; Based on the number of responses and / or the response latency, determine the packet loss rate and / or average network latency within the detection period; Use the packet loss rate and / or the average network latency as the network quality information between the cloud server and the cloud terminal; The determining a target cloud desktop protocol from the multiple cloud desktop protocols based on the network quality information includes: Based on the packet loss rate and / or the average network latency, determine a target cloud desktop protocol from the multiple cloud desktop protocols.

4. The method according to claim 2, characterized in that, Before determining the network quality information between the cloud server and the cloud terminal, the method further includes: Use the intranet penetration technology to establish a P2P channel with the cloud terminal; The determining the network quality information between the cloud server and the cloud terminal includes: Based on the P2P channel, determine the network quality information between the cloud server and the cloud terminal.

5. The method according to claim 1, characterized in that, The cloud desktop application further includes a USB / IP protocol. The cloud terminal serves as a USB / IP server, and the cloud server serves as a USB / IP client. The method further includes: Receive USB peripheral data sent by the cloud terminal through the cloud desktop application; Parse the USB peripheral data based on the USB / IP protocol to obtain corresponding USB events; Control the operating system of the cloud server to execute operations corresponding to the USB events.

6. A cloud desktop interaction method, characterized in that, Applied to a cloud terminal, a cloud desktop application is installed in the cloud terminal. The cloud terminal interacts with a cloud server installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The method includes: Receive the desktop data sent by the cloud server; Decode the desktop data through the codec of the target cloud desktop protocol in the cloud desktop application. The target cloud desktop protocol is the encoding protocol used for the desktop data, and the target cloud desktop protocol belongs to the multiple cloud desktop protocols; Display the decoded desktop data.

7. The method according to claim 6, characterized in that, Before receiving the desktop data sent by the cloud server, the method further includes: Establish a P2P channel with the cloud server by using the intranet penetration technology; Receiving the desktop data sent by the cloud server includes: Receiving the desktop data sent by the cloud server based on the P2P channel.

8. The method according to claim 6, characterized in that, The cloud desktop application also includes the USB / IP protocol. The cloud terminal serves as a USB / IP server, and the cloud server serves as a USB / IP client. The method further includes: Circularly monitor the insertion situation of the local USB interface to determine whether a USB device is inserted into the USB interface; When it is monitored that a USB device is inserted into the USB interface, send the USB peripheral data corresponding to the USB device to the cloud server based on the USB / IP protocol.

9. A cloud desktop interaction device, characterized in that, Applied to a cloud server, a cloud desktop application is installed in the operating system of the cloud server. The cloud server interacts with a cloud terminal installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The device includes: A protocol selector module for determining a target cloud desktop protocol from multiple cloud desktop protocols of the cloud desktop application; A data acquisition module for acquiring the desktop data of the cloud server through the cloud desktop application; A codec module for encoding the desktop data based on the codec of the target cloud desktop protocol; A sending module for sending the encoded desktop data to the cloud terminal.

10. A cloud desktop interaction device, characterized in that, Applied to a cloud terminal, a cloud desktop application is installed in the cloud terminal. The cloud terminal interacts with a cloud server installed with the cloud desktop application. The cloud desktop application includes codecs for multiple cloud desktop protocols. The device includes: A receiving module for receiving the desktop data sent by the cloud server; A codec module for decoding the desktop data through the codec of the target cloud desktop protocol in the cloud desktop application. The target cloud desktop protocol is the encoding protocol adopted by the desktop data, and the target cloud desktop protocol belongs to the multiple cloud desktop protocols; A display module for displaying the decoded desktop data.

11. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the cloud desktop interaction method as described in any one of claims 1-5 or 6-8 is implemented.

12. A computer-readable storage medium, characterized in that, Computer program instructions are stored on a computer-readable storage medium. When the computer program instructions are executed by a processor, the cloud desktop interaction method as described in any one of claims 1-5 or 6-8 is implemented.

Images