Multi-channel virtual desktop remote display method and system and storage medium

Abstract

The invention discloses a multi-channel virtual desktop remote display method and system and a storage medium, and belongs to the technical field of data processing. The method comprises the steps that S1, a server simulates a virtual display device, a virtual input device and a virtual sound card device for each virtual desktop; s2, a client is connected to the server through a network to start a remote display server for each virtual desktop; s3, establishing a plurality of channels between the client and the remote display server; s4, the remote display server transmits the virtual desktop data to the client; s5, the client remotely displays and controls the virtual desktop; a virtual desktop video area detection method is adopted, a dynamic video playing area and a static image area on a virtual desktop are distinguished, virtual desktop data in the dynamic video playing area are compressed by using a video compression algorithm, and virtual desktop data in the static image area are compressed by using an image compression algorithm to realize classified compression. And the network bandwidth is saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a multi-channel virtual desktop remote display method and system and a storage medium. BACKGROUND

[0002] In application scenarios such as remote video conferencing, online education, and virtual reality, virtual desktop technology is widely used. However, when transmitting virtual desktop data, due to the huge amount of data, video compression technology is often used to reduce bandwidth requirements. Traditional methods usually use video compression algorithms to compress the entire virtual desktop data, resulting in a low compression ratio and increasing bandwidth requirements. In addition, in order to ensure the remote display effect of the virtual desktop, the clarity and smoothness of the video usually need to be ensured. Therefore, how to further improve the video compression ratio and save network bandwidth while ensuring the remote display effect of the virtual desktop is an important problem faced by current virtual desktop technology. SUMMARY

[0003] The present application aims to overcome the shortcomings of the prior art and provide a multi-channel virtual desktop remote display method and system and a storage medium.

[0004] The purpose of the present application is achieved by the following technical solutions: the present application provides a multi-channel virtual desktop remote display method, comprising the following steps: S1: the server simulates a virtual display device, a virtual input device, and a virtual sound card device for each virtual desktop, and each virtual desktop communicates with the corresponding virtual display device, virtual input device, and virtual sound card device through its own virtual device driver; S2: the client connects to the server to start a remote display server for each virtual desktop, and the remote display server provides data communication and device interaction for the virtual sound card device, virtual input device, and virtual sound card device through a virtual desktop manager layer; S3: the client and the remote display server first establish a main channel, and then establish a display channel, an input channel, and a sound channel; the display channel is used to receive remote display update data, the input channel is used to send mouse and keyboard events, and the sound channel is used to receive audio streams and send client audio input; S4: the remote display server transmits virtual desktop display data, mouse and keyboard interaction data, and sound data to the client through the display channel, input channel, and sound channel; S5: the client displays the data transmitted by the remote display server, and synchronously sends user operations to the remote display server, thereby realizing remote display and control of the virtual desktop by the client; The virtual desktop video area detection method is used for reducing the occupation of network transmission bandwidth when the virtual desktop plays high-definition video, distinguishing the dynamic video playing area from the static image area on the virtual desktop, compressing the virtual desktop data in the dynamic video playing area using a video compression algorithm, and compressing the virtual desktop data in the static image area using an image compression algorithm to realize classified compression.

[0005] Preferably, the client and the remote display server communicate through multiple channels, including a main channel, a mouse and keyboard input channel, a display channel, a pointer channel, an audio playing channel, an audio recording channel and a USB device mapping channel; at the client, each channel is processed by a separate thread to communicate with the remote display server, each channel type corresponds to a different data type, and each channel uses a separate TCP port.

[0006] Preferably, the virtual desktop video area detection method comprises the following steps: The virtual desktop is divided into a plurality of block areas using a virtual desktop content differentiation transmission mechanism, and only the block areas in which the content has changed are transmitted by utilizing the similarity of the content of adjacent virtual desktops. The boundary of the dynamic video playing area is determined according to the characteristics of high content change rate and rich color information of the dynamic video playing area.

[0007] Preferably, the content change rate of each block area in the virtual desktop is counted before the boundary of the dynamic video playing area is determined, and the block area with a change rate higher than a first preset value is regarded as a high change rate area. The horizontal and vertical straight lines are obtained using the position information of the high change rate area, a rectangle containing all the high change rate areas is constructed, and the rectangle is regarded as the boundary of the dynamic video playing area. It is judged whether the boundary of the dynamic video playing area has a change event, and if the change event occurs, the current dynamic video playing area is destroyed.

[0008] Preferably, the color histogram information is used to correct the boundary determination result of the dynamic video playing area. Preferably, the change event is that the dynamic video playing area expands or shrinks or the position information changes.

[0009] The second aspect of the present application provides a multi-channel virtual desktop remote display system, which is used to implement any of the above multi-channel virtual desktop remote display methods, and comprises: A high change rate area identification module is used to count the content change rate of each block area in the virtual desktop before the boundary of the dynamic video playing area is determined, and the block area with a change rate higher than a first preset value is regarded as a high change rate area. The rectangular boundary determining module is used for obtaining horizontal and vertical straight lines according to the position information of the high change rate area, and constructing a rectangle containing all the high change rate areas as the boundary of the dynamic video playing area. The video area state tracking module is used for judging whether the boundary of the dynamic video playing area has a change event, and destroying the current dynamic video playing area if the change event occurs.

[0010] The third aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores computer executable instructions, the computer executable instructions are loaded and executed by a processor, and the computer executable instructions realize the multi-channel virtual desktop remote display method.

[0011] The present application has the following advantages: 1) The dynamic video playing area and the static image area on the virtual desktop are distinguished, the virtual desktop data in the dynamic video playing area is compressed using a video compression algorithm, and the virtual desktop data in the static image area is compressed using an image compression algorithm to realize classified compression, thereby saving network bandwidth.

[0012] 2) The color histogram information is used to correct the boundary determination result of the dynamic video playing area, thereby improving the determination accuracy of the dynamic video playing area. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The figure is a multi-channel virtual desktop remote display transmission framework. Figure 2 The figure is a virtual desktop video area detection method flowchart. DETAILED DESCRIPTION

[0014] The technical solutions of the present application will be described clearly and completely below in conjunction with embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0015] Referring to Figures 1-2 The first aspect of the present application provides a multi-channel virtual desktop remote display method, including the following steps: S1: The server simulates a virtual display device, a virtual input device and a virtual sound card device for each virtual desktop, and each virtual desktop communicates with the corresponding virtual display device, virtual input device and virtual sound card device through the virtual device driver thereof; S2: the client connects to the server to start a remote display server for each virtual desktop, and the remote display server provides data communication and device interaction for virtual graphics card devices, virtual input devices and virtual sound card devices through a virtual desktop manager layer; S3: the client and the remote display server first establish a main channel, and then establish a display channel, an input channel and a sound channel; the display channel is used to receive remote display update data, the input channel is used to send mouse and keyboard events, and the sound channel is used to receive audio streams and send client audio input; S4: the remote display server transmits virtual desktop display data, mouse and keyboard interaction data and sound data to the client through the display channel, the input channel and the sound channel; S5: the client displays the data transmitted by the remote display server, and synchronously sends user operations to the remote display server, thereby realizing remote display and control of the virtual desktop by the client; The virtual desktop video area detection method used in S3-S5 is used to reduce the occupation of network transmission bandwidth when the virtual desktop plays high-definition videos, to distinguish dynamic video playing areas from static image areas on the virtual desktop, to compress virtual desktop data in the dynamic video playing areas using a video compression algorithm, and to compress virtual desktop data in the static image areas using an image compression algorithm to realize classified compression.

[0016] The multi-channel virtual desktop remote display method is a key technical guarantee for access efficiency, peripheral extensibility and security controllability of cloud desktop services in a virtualization cloud platform. A self-controllable virtual desktop remote display control protocol is adopted to realize multi-channel transmission, and different channels are used for protocol exchange, video, pictures, sound, USB mapping, mouse and keyboard. As shown in Figure 1 The client connects to the server to start a remote display server for each virtual machine through the FastRAP protocol, and the remote display server provides seamless data communication and device interaction for virtual graphics card devices, VDI libraries and VDI interfaces through a virtual machine manager layer, thereby realizing remote display and control of the virtual machine desktop by the client. The client connects to the server through the FastRAP protocol, and the FastRAP protocol uses a server public key certificate to perform SSL / TLS encryption protection on data transmitted between the client and the remote display server through each channel.

[0017] In some embodiments, the client and the remote display server communicate through multiple channels, including a main channel, a mouse and keyboard input channel, a display channel, a pointer channel, an audio playing channel, an audio recording channel and a USB device mapping channel; in the client, each channel is processed by a separate thread for communication with the remote display server, each channel type corresponds to a different data type, and each channel uses a separate TCP port.

[0018] In some embodiments, the virtual desktop video area detection method comprises the following steps: The virtual desktop is divided into several block areas using a virtual desktop content differential transmission mechanism, and only the block areas in which the content has changed are transmitted using the similarity of the content of adjacent virtual desktops. According to the characteristics of the dynamic video playing area content change rate and rich color information, the boundary of the dynamic video playing area is determined.

[0019] In some embodiments, the content change rate of each block area in the virtual desktop is counted before the boundary of the dynamic video playing area is determined, and the block area with a change rate higher than a first preset value is regarded as a high change rate area. Using the position information of the high change rate area, horizontal and vertical straight lines are obtained, and a rectangle containing all the high change rate areas is constructed as the boundary of the dynamic video playing area. It is judged whether the boundary of the dynamic video playing area has a change event, and if the change event occurs, the current dynamic video playing area is destroyed.

[0020] In some embodiments, the boundary determination result of the dynamic video playing area is corrected using color histogram information. In some embodiments, the change event is the expansion or reduction of the dynamic video playing area or the change of the position information.

[0021] The second aspect of the present application provides a multi-channel virtual desktop remote display system, which is used to implement any of the above multi-channel virtual desktop remote display methods, and comprises: The high change rate area identification module is used to count the content change rate of each block area in the virtual desktop before the boundary of the dynamic video playing area is determined, and the block area with a change rate higher than a first preset value is regarded as a high change rate area. The rectangle boundary determination module is used to obtain horizontal and vertical straight lines according to the position information of the high change rate area, and construct a rectangle containing all the high change rate areas as the boundary of the dynamic video playing area. The video area state tracking module is used to judge whether the boundary of the dynamic video playing area has a change event, and if the change event occurs, the current dynamic video playing area is destroyed.

[0022] The third aspect of the present application provides a computer readable storage medium, which stores computer executable instructions, and when the computer executable instructions are loaded and executed by a processor, any of the above multi-channel virtual desktop remote display methods is implemented.

[0023] The foregoing is considered as illustrative only of the principles of the application. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the application to the exact construction and operation described. Accordingly, all such variations are intended to be included within the scope of the present application as defined in the claims below and their equivalents.

Claims

1. A method for remote display of a multi-channel virtual desktop, characterized in that: Includes the following steps: S1: The server provides each virtual desktop with a simulated virtual display device, virtual input device, and virtual sound card device. Each virtual desktop communicates with the corresponding virtual display device, virtual input device, and virtual sound card device through its own virtual device driver. S2: The client connects to the server to start the remote display server for each virtual desktop. The remote display server provides data communication and device interaction for virtual graphics card devices, virtual input devices and virtual sound card devices through the virtual desktop manager layer. S3: The client and the remote display server first establish a main channel, and then establish a display channel, an input channel, and a sound channel; the display channel is used to receive remote display update data, the input channel is used to send mouse and keyboard events, and the sound channel is used to receive audio streams and send client audio input; S4: The remote display server transmits virtual desktop display data, mouse and keyboard interaction data, and sound data to the client through the display channel, input channel, and sound channel; S5: The client displays the data transmitted by the remote display server and synchronously sends the user's operations to the remote display server, enabling the client to remotely display and control the virtual desktop; The virtual desktop video region detection method used in S3-S5 is used to reduce the network transmission bandwidth occupation when playing high-definition videos on the virtual desktop. It distinguishes between the dynamic video playback area and the static image area on the virtual desktop, uses a video compression algorithm to compress the virtual desktop data in the dynamic video playback area, and uses an image compression algorithm to compress the virtual desktop data in the static image area to achieve classified compression.

2. The multi-channel virtual desktop remote display method according to claim 1, characterized in that: The client and remote display server communicate through multiple channels, including a main channel, a mouse and keyboard input channel, a display channel, a pointer channel, an audio playback channel, an audio recording channel, and a USB device mapping channel. On the client side, each channel is handled by a separate thread, communicating with the remote display server. Each channel type corresponds to a different data type, and each channel uses a separate TCP port.

3. The multi-channel virtual desktop remote display method according to claim 1, characterized in that: The virtual desktop video region detection method includes the following steps: The virtual desktop content differentiation transmission mechanism divides the virtual desktop into several block areas. By leveraging the similarity of the content of adjacent virtual desktops, only the block areas in which the content of the desktop has changed are transmitted. Based on the characteristics of high content change rate and rich color information in the dynamic video playback area, the boundary of the dynamic video playback area is determined.

4. The multi-channel virtual desktop remote display method according to claim 3, characterized in that: Before the boundary of the dynamic video playback area is determined, the rate of change of the content of each block area in the virtual desktop is statistically analyzed, and the block area with a rate of change higher than the first preset value is regarded as a high rate of change area. By utilizing the location information of high-rate-of-change areas, horizontal and vertical lines are obtained, and a rectangle containing all high-rate-of-change areas is constructed as the boundary of the dynamic video playback area. Determine if the boundary of the dynamic video playback area has changed; if it has, destroy the current dynamic video playback area.

5. The multi-channel virtual desktop remote display method according to claim 4, characterized in that: The boundary determination results of the dynamic video playback area are corrected using color histogram information.

6. The multi-channel virtual desktop remote display method according to claim 4, characterized in that: The aforementioned change events are the expansion or reduction of the dynamic video playback area or changes in its location information.

7. A multi-channel virtual desktop remote display system, characterized in that: The method for implementing the multi-channel virtual desktop remote display method as described in any one of claims 1-6 includes: The high change rate region identification module is used to count the change rate of the content of each block area in the virtual desktop before the boundary of the dynamic video playback area is determined, and the block area with a change rate higher than the first preset value is identified as a high change rate region. The rectangular boundary determination module is used to obtain horizontal and vertical straight lines based on the location information of high change rate areas, and construct a rectangle that includes all high change rate areas as the boundary of the dynamic video playback area. The video region status tracking module is used to determine whether the boundary of the dynamic video playback region has changed. If a change occurs, the current dynamic video playback region is destroyed.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the multi-channel virtual desktop remote display method as described in any one of claims 1-6.

Images