Cloud desktop display method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a cloud desktop display method and system. The method comprises the following steps: capturing graphic data of a cloud desktop virtual machine after a cloud desktop user successfully logs in; encoding the graphic data to obtain video stream data; and transmitting the video stream data to a cloud desktop client, so that the cloud desktop client renders a window, and displaying the video stream data in the window. According to the embodiment of the invention, video stream coding transmission and hardware decoding processing are adopted, so that the phenomenon of coexistence of multiple windows is avoided. The user only needs to interact with the unique operation window, so that the operation process is simplified, the user experience is improved, the utilization of system resources is optimized, and the problems of misoperation and system jamming caused by redundant windows are reduced. The efficiency and the stability of the cloud desktop are improved, and the requirement for efficient remote interaction in remote office and enterprise application is met.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud desktops, and in particular to a cloud desktop display method, a cloud desktop display system, an electronic device, and a computer-readable storage medium. Background Art

[0002] In cloud desktop technology, users remotely access and operate virtual machines through clients, just like using local computers. Cloud desktops provide great convenience for remote office, enterprise applications and distributed computing, and are widely used in enterprise environments. However, in actual applications, some problems may occur during the display of cloud desktops, affecting the user's operating experience.

[0003] Currently, many cloud desktop systems use QXL (a graphics accelerator in a virtual machine that improves the graphics processing performance of a virtual desktop) driver to achieve remote display. However, despite the significant advantages of QXL in graphics processing and remote display, it has caused some display problems in actual deployment, especially in the window display management of the cloud desktop client.

[0004] Specifically, the QXL driver is activated and provides graphics display functions immediately after the virtual machine is started. Although this design ensures the display requirements of the virtual machine during the startup process, it also brings some unnecessary side effects. For example, after the virtual machine is started, the QXL driver immediately displays a window on the client. This window is used to display the entire process of the virtual machine startup, including the boot screen, BIOS information, and login interface. However, as the user logs in successfully, a new window will be generated on the client, and the user's operations and application interface will be displayed in this new window.

[0005] This phenomenon causes the cloud desktop client to display two different windows throughout the process. The first window appears when the cloud desktop virtual machine is started, showing the virtual machine's boot and login process; the second window is generated after a successful login, and is used to remotely display the actual operation interface. The problem is that users only need the second window for operation and interaction, but due to the design limitations of the QXL driver, the first window cannot be closed or removed. This not only brings unnecessary screen usage, but also confuses users, especially when switching windows, which may lead to misoperation or operation delays. Summary of the invention

[0006] In view of the above problems, embodiments of the present invention are proposed to provide a cloud desktop display method, a cloud desktop display system, an electronic device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0007] In order to solve the above problems, an embodiment of the present invention discloses a cloud desktop display method, which includes: capturing graphic data of a cloud desktop virtual machine after a cloud desktop user successfully logs in; encoding the graphic data to obtain video stream data; transmitting the video stream data to a cloud desktop client so that the cloud desktop client renders a window and displays the video stream data in the window.

[0008] Optionally, capturing the graphic data of the cloud desktop virtual machine after a cloud desktop user successfully logs in includes: after the cloud desktop user successfully logs in, starting a daemon process on the cloud desktop virtual machine; and capturing the graphic data using the daemon process.

[0009] Optionally, encoding the graphic data to obtain video stream data includes: utilizing the daemon process supporting the Simple Independent Computing Environment (SPICE) protocol to encode the graphic data to obtain the video stream data of the Advanced Video Coding H.264 standard or the High Efficiency Video Coding H.265 standard.

[0010] Optionally, transmitting the video stream data to the cloud desktop client includes: using the cloud desktop virtual machine to transmit the video stream data to the cloud desktop server; and forwarding the video stream data to the cloud desktop client through the cloud desktop server.

[0011] An embodiment of the present invention also discloses a method for displaying a cloud desktop, which includes: obtaining video stream data of a cloud desktop virtual machine, wherein the video stream data is obtained by capturing and encoding graphic data of the cloud desktop virtual machine after a cloud desktop user successfully logs in; rendering a window, and displaying the video stream data in the window.

[0012] Optionally, displaying the video stream data in the window includes: performing hardware decoding processing on the video stream data of the Advanced Video Coding H.264 standard or the High Efficiency Video Coding H.265 standard, and displaying the video data after the hardware decoding processing in the window.

[0013] An embodiment of the present invention also discloses a cloud desktop display system, which includes: a graphics data capture module, used to capture the graphics data of the cloud desktop virtual machine after the cloud desktop user successfully logs in; a graphics data encoding module, used to encode the graphics data to obtain video stream data; a video stream data transmission module, used to transmit the video stream data to the cloud desktop client, so that the cloud desktop client renders a window and displays the video stream data in the window.

[0014] Optionally, the graphic data capture module includes: a process startup module, used to start the daemon process on the cloud desktop virtual machine after the cloud desktop user successfully logs in; and a data acquisition module, used to capture the graphic data using the daemon process.

[0015] Optionally, the graphic data encoding module is used to utilize the daemon process supporting the simple independent computing environment SPICE protocol to encode the graphic data to obtain video stream data of the advanced video coding H.264 standard or the high efficiency video coding H.265 standard.

[0016] Optionally, the video stream data transmission module includes: a virtual machine transmission module, used to use the cloud desktop virtual machine to transmit the video stream data to the cloud desktop server; a server forwarding module, used to forward the video stream data to the cloud desktop client through the cloud desktop server.

[0017] An embodiment of the present invention also discloses a cloud desktop display system, which includes: a video stream data acquisition module, used to acquire video stream data of a cloud desktop virtual machine, wherein the video stream data is obtained by capturing and encoding graphic data of the cloud desktop virtual machine after a cloud desktop user successfully logs in; a video stream data display module, used to render a window, and display the video stream data in the window.

[0018] Optionally, the video stream data display module is used to perform hardware decoding processing on the video stream data of the advanced video coding H.264 standard or the high efficiency video coding H.265 standard, and display the video data after the hardware decoding processing in the window.

[0019] An embodiment of the present invention also discloses an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enables the electronic device to execute the cloud desktop display method as described above.

[0020] The embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the cloud desktop display method as described above.

[0021] The embodiments of the present invention include the following advantages:

[0022] The cloud desktop display solution provided by the embodiment of the present invention captures the graphic data of the cloud desktop virtual machine after the cloud desktop user successfully logs in, and then encodes the graphic data to obtain video stream data. Next, the video stream data is transmitted to the cloud desktop client so that the cloud desktop client renders the window and displays the video stream data in the window.

[0023] The embodiment of the present invention captures the graphic data of the cloud desktop virtual machine after the user successfully logs in, encodes and processes these graphic data into a video stream, and renders a separate window after transmitting it to the client. This window not only displays the actual operation interface after login, but also replaces the window of the previous boot process, avoiding the coexistence of two windows. The user only needs to interact with a single window, thereby eliminating the trouble of redundant windows, simplifying the interface operation, and greatly improving the user's operation experience. By capturing and transmitting video stream data after the user successfully logs in, the graphics rendering burden of the QXL driver during the entire startup process is avoided. Through the video stream data transmission and rendering method, the graphics processing can be concentrated on the operation interface that the user really needs, thereby reducing the resource consumption of the system and making the display solution of the entire cloud desktop more efficient. The cloud desktop client only displays one window, and the user's focus is concentrated on this window, reducing the risk of misoperation caused by multiple windows. At the same time, the elimination of window switching also reduces the response delay of the system, improves the smoothness of interaction, and enhances the user's work efficiency and experience. The graphics data encoding and transmission mechanism is more flexible than the direct rendering method of the QXL driver. By capturing the graphic data of the cloud desktop virtual machine and transmitting it through video streaming, the display content and display logic of the virtual machine can be managed more effectively, reducing system instability factors caused by poor window management. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of steps of a method for displaying a cloud desktop according to an embodiment of the present invention;

[0025] Figure 2 is a flowchart of another method for displaying a cloud desktop according to an embodiment of the present invention;

[0026] Figure 3 It is a structural block diagram of a display system of a cloud desktop according to an embodiment of the present invention;

[0027] Figure 4 It is a structural block diagram of another cloud desktop display system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The display solution of the cloud desktop proposed in the embodiment of the present invention aims at the multi-window problem in the prior art. It captures the graphic data of the cloud desktop virtual machine after the user successfully logs in, encodes the graphic data into video stream data, and then transmits the video stream data to the cloud desktop client to render a single window. Compared with the traditional solution, the QXL driver generates a boot process window when the virtual machine starts, and generates a second window after the user logs in, which avoids the coexistence of multiple windows. The user only needs to interact with the only operation window, which simplifies the operation process and improves the user experience. At the same time, it also optimizes the utilization of system resources and reduces the misoperation and system freeze caused by redundant windows. The efficiency and stability of the cloud desktop are improved, and the needs for efficient remote interaction in remote office and enterprise applications are met.

[0030] Reference Figure 1 , showing a flow chart of the steps of a method for displaying a cloud desktop according to an embodiment of the present invention. The method for displaying a cloud desktop involves a cloud desktop virtual machine, a cloud desktop server, and a cloud desktop client. The cloud desktop server communicates with the cloud desktop virtual machine and the cloud desktop client respectively. The user of the cloud desktop client operates the cloud desktop virtual machine through the cloud desktop server. The method for displaying a cloud desktop may specifically include the following steps:

[0031] Step 101 , capturing graphic data of a cloud desktop virtual machine after a cloud desktop user successfully logs in.

[0032] In an embodiment of the present invention, in a cloud desktop environment, the cloud desktop virtual machine is the controlled end of the cloud desktop, responsible for processing and generating graphic data generated during user operations. After the cloud desktop user successfully logs in, the operating system of the cloud desktop virtual machine enters the normal desktop operating environment, and all user inputs (such as keyboard and mouse operations) and graphical interface information fed back by the system are generated and displayed. At this time, the cloud desktop virtual machine needs to capture these graphic data as the core content transmitted to the cloud desktop client.

[0033] For example, when a cloud desktop user successfully logs in to a cloud desktop virtual machine, every frame of the cloud desktop virtual machine operating system will be captured in real time, including system operations, video playback content, etc. These graphic data will then be sent to the cloud desktop client through encoding and transmission. In this way, all user operations and interface changes can be accurately recorded and transmitted.

[0034] Step 102: Encode the graphic data to obtain video stream data.

[0035] In the embodiment of the present invention, after the graphic data is captured, directly transmitting the graphic data to the cloud desktop client is not the best choice, because the original graphic data is very large, and the transmission will occupy a lot of bandwidth resources, especially when the graphic data contains high-resolution pictures or complex multimedia content. In order to improve the transmission efficiency and reduce the occupation of network bandwidth, the captured graphic data needs to be encoded.

[0036] Coding refers to compressing graphic data to reduce the size of the data while maintaining the image quality as much as possible. There are many commonly used algorithms for video coding, the most common of which include Advanced Video Coding (AVC or H.264) or High Efficiency Video Coding (HEVC or H.265). These coding algorithms can significantly compress graphic data and generate video stream data that can be effectively transmitted.

[0037] In this step, each frame of captured graphic data is processed through an encoder. The encoding process not only compresses the image data, but also adds necessary metadata to the video stream, such as timestamps, to ensure that each frame can be accurately reproduced during the subsequent decoding and display process.

[0038] Encoding is not only applicable to static graphic data, but also to dynamic scenes. For example, a user opens a 3D graphics application on a cloud desktop, which continuously generates highly complex images. Through encoding, these highly complex graphic data can be compressed into smaller data streams, enabling them to be transmitted with lower network bandwidth, ensuring a smooth user experience.

[0039] During the encoding process, encoding parameters such as frame rate, resolution, and compression ratio are dynamically adjusted according to the current network conditions and system resources to balance image quality and transmission efficiency. For example, when the network conditions are good, a higher resolution and frame rate can be selected for encoding, while when the network delay is large, the resolution or frame rate may need to be reduced to ensure the stability of transmission.

[0040] Step 103: Transmit the video stream data to the cloud desktop client, so that the cloud desktop client renders a window and displays the video stream data in the window.

[0041] In an embodiment of the present invention, after the encoding of the graphic data is completed, the video stream data needs to be transmitted from the cloud desktop virtual machine to the cloud desktop client. This process involves multiple network communication links, and ensuring that the video stream data can be stably and quickly transmitted to the cloud desktop client is the key to achieving smooth operation of the cloud desktop.

[0042] First, the cloud desktop virtual machine can transmit the video stream data to the cloud desktop client through an appropriate protocol, such as the Simple Protocol for Independent Computing Environments (SPICE) protocol. In this process, the video data stream may need to be optimized to adapt to different network conditions, such as buffering the video data stream or optimizing the transmission order.

[0043] After the transmission is completed, the cloud desktop client receives the video stream data. The task of the cloud desktop client is to display the video stream data in a window. Specifically, the cloud desktop client can perform hardware decoding on the video stream data. Hardware decoding refers to the use of the cloud desktop client's graphics processing unit (GPU) to perform video decoding instead of relying on the CPU. Hardware decoding is more efficient than software decoding, and is particularly suitable for processing complex video streams (such as H.264 or H.265 encoded video streams), which can significantly reduce CPU usage and improve system performance.

[0044] For example, the screen of the cloud desktop virtual machine has been transmitted to the cloud desktop client in the form of encoded video streams. The cloud desktop client decodes these video streams into displayable image content through the hardware decoding function of the GPU and presents them in a window. Due to the high efficiency of hardware decoding, the cloud desktop client can display the cloud desktop operation with low latency and high image quality, and even when processing high-definition or complex graphics applications, it can still maintain a smooth operation experience.

[0045] The cloud desktop display solution provided by the embodiment of the present invention captures the graphic data of the cloud desktop virtual machine after the cloud desktop user successfully logs in, and then encodes the graphic data to obtain video stream data. Next, the video stream data is transmitted to the cloud desktop client so that the cloud desktop client renders the window and displays the video stream data in the window.

[0046] The embodiment of the present invention captures the graphic data of the cloud desktop virtual machine after the user successfully logs in, encodes and processes these graphic data into a video stream, and renders a separate window after transmitting it to the client. This window not only displays the actual operation interface after login, but also replaces the window of the previous boot process, avoiding the coexistence of two windows. The user only needs to interact with a single window, thereby eliminating the trouble of redundant windows, simplifying the interface operation, and greatly improving the user's operation experience. By capturing and transmitting video stream data after the user successfully logs in, the graphics rendering burden of the QXL driver during the entire startup process is avoided. Through the video stream data transmission and rendering method, the graphics processing can be concentrated on the operation interface that the user really needs, thereby reducing the resource consumption of the system and making the display solution of the entire cloud desktop more efficient. The cloud desktop client only displays one window, and the user's focus is concentrated on this window, reducing the risk of misoperation caused by multiple windows. At the same time, the elimination of window switching also reduces the response delay of the system, improves the smoothness of interaction, and enhances the user's work efficiency and experience. The graphics data encoding and transmission mechanism is more flexible than the direct rendering method of the QXL driver. By capturing the graphic data of the cloud desktop virtual machine and transmitting it through video streaming, the display content and display logic of the virtual machine can be managed more effectively, reducing system instability factors caused by poor window management.

[0047] In an exemplary embodiment of the present invention, an implementation method of capturing the graphic data of a cloud desktop virtual machine after a cloud desktop user successfully logs in is to start a daemon process on the cloud desktop virtual machine after the cloud desktop user successfully logs in; and use the daemon process to capture the graphic data. Specifically, when a cloud desktop user of a cloud desktop virtual machine successfully logs in to the cloud desktop virtual machine, the cloud desktop virtual machine will start the daemon process in the operating system. The daemon process is a dedicated process in the operating system, which is responsible for capturing the graphic data generated on the cloud desktop virtual machine in real time. The graphic data here includes all visual elements such as desktop display, application windows, icons, and user operation interfaces. The daemon process will not be started before the cloud desktop user logs in, thereby reducing resource usage. Through this design, the daemon process only starts working after the cloud desktop user enters the operating system desktop, ensuring that the captured image data is closely related to the user's operation, thereby improving the overall efficiency of the cloud desktop.

[0048] After the daemon is started, it first interacts with the graphics hardware of the cloud desktop virtual machine through the graphics rendering interface of the operating system to obtain each frame of graphics data in real time. The acquired graphics data is raw, uncompressed data. This data is usually large in size, and directly transmitting it to the cloud desktop client consumes a lot of bandwidth. In order to improve data transmission efficiency, the daemon compresses and encodes the captured graphics data. Common encoding methods include H.264 and H.265. These encoding technologies can significantly reduce the size of data while ensuring image quality.

[0049] For example, when a cloud desktop user successfully logs in to the cloud desktop virtual machine and opens a browser on the virtual desktop to watch an online video, the daemon captures every frame of the video in the browser window in real time and encodes these frames into video stream data. Due to the use of efficient H.264 or H.265 encoding, high-definition image data that originally required a lot of bandwidth to transmit is compressed into smaller data packets, so that it can be smoothly transmitted in a low-bandwidth network environment. This method ensures that there is almost no delay between the image seen by the user on the cloud desktop client and the actual operation, and the user experience is close to local operation.

[0050] In an embodiment of the present invention, the startup of the daemon process and the capture of graphic data not only optimize the data transmission between the cloud desktop virtual machine and the cloud desktop client, but also avoid the invalid occupation of resources. In the traditional method, the cloud desktop virtual machine often starts to capture and compress data before the user logs in, which will cause the cloud desktop virtual machine to occupy a large amount of CPU and memory resources during the system startup or login process, especially when the cloud desktop virtual machine processes complex graphics or multimedia content, the performance problem will be more serious. By starting the daemon process only after the cloud desktop user logs in successfully, this implementation method avoids unnecessary resource consumption. Moreover, it can dynamically adapt to different application scenarios. For example, if the user only performs lightweight operations (such as word processing or web browsing) after logging in, the daemon process will capture less graphic data, and the encoding and transmission burden will also be lighter. When the user performs complex operations (such as high-definition video playback or 3D modeling), the daemon process can efficiently capture and compress these high-load graphic data, ensuring that the cloud desktop system can still maintain fluency under high load.

[0051] In an exemplary embodiment of the present invention, an implementation method of encoding and processing graphic data to obtain video stream data is to use a daemon process that supports the simple independent computing environment SPICE protocol to encode and process the graphic data to obtain video stream data of the advanced video coding H.264 standard or the high-efficiency video coding H.265 standard. Specifically, the SPICE protocol plays a key role. SPICE is a remote display protocol used in a virtualized environment. It supports efficient transmission of graphic data and user input data of cloud desktop virtual machines, allowing cloud desktop clients to interact with cloud desktop virtual machines with low latency. The SPICE protocol supports different video compression formats, of which H.264 and H.265 are currently the two most commonly used video compression standards. H.264 encoding can significantly reduce the data size when processing high-definition video, while H.265 further improves the compression efficiency, especially for 4K and higher resolution videos.

[0052] The daemon encodes the graphic data through the SPICE protocol and selects the encoding standard that is suitable for the network environment and the performance of the cloud desktop client. For example, if a user plays a high-definition video on a virtual desktop, the daemon will encode the graphic data into a video stream in H.264 or H.265 format. The amount of data after encoding is greatly reduced, and even under network conditions with limited bandwidth, the video can still be played smoothly and with high image quality. At the same time, after receiving these video streams, the client device can accelerate the processing through hardware decoding, thereby reducing the CPU usage and improving the overall user experience.

[0053] For example, when a user uses complex 3D modeling software on a virtual desktop, the daemon not only captures high-precision images of each frame, but also efficiently compresses them through H.265 encoding. Since H.265 can provide a higher compression ratio than H.264 at the same image quality, this is especially useful when dealing with a lot of details or high-resolution scenes. Ultimately, the cloud desktop client can quickly receive and present these high-quality graphics data with lower bandwidth and resource usage.

[0054] The video stream data in the embodiment of the present invention is encoded through H.264 or H.265, which significantly reduces the volume of graphic data, thereby reducing the consumption of network bandwidth. This not only improves the efficiency of data transmission, but also ensures that the cloud desktop system can still maintain a high response speed and picture quality in a low-bandwidth or high-latency network environment. The cloud desktop client processes the video stream data through hardware decoding, which reduces the CPU load and improves the performance and fluency of the overall system. Finally, the use of the SPICE protocol enables the entire system to adaptively adjust in different network environments, thereby providing a more flexible and stable cloud desktop experience.

[0055] In an exemplary embodiment of the present invention, one implementation method of transmitting video stream data to a cloud desktop client is to use a cloud desktop virtual machine to transmit the video stream data to a cloud desktop server; and forward the video stream data to the cloud desktop client through the cloud desktop server. In the application of cloud desktop technology, transmitting video stream data to a cloud desktop client is one of the key links to ensure that users can operate the remote desktop system in real time. In order to achieve this function, the cloud desktop system usually needs to pass the video stream data from the cloud desktop virtual machine to the final cloud desktop client through a complex transmission path to ensure that the interface seen by the user is smooth, real-time and synchronized with the cloud desktop virtual machine.

[0056] In the actual application process, after the video stream data is generated, it will be transmitted to the cloud desktop server through the network components built into the cloud desktop virtual machine. The cloud desktop server is the core node of the entire cloud desktop architecture. It is not only responsible for managing the operation of the cloud desktop virtual machine, but also undertakes the distribution and forwarding functions of video stream data. After receiving the video stream data, the cloud desktop server will manage and process the data stream data. In some cases, the cloud desktop server may need to further optimize or re-encode the video stream data to ensure that it matches the network conditions of the cloud desktop client. For example, the quality and bit rate of the video stream data are dynamically adjusted according to the network bandwidth of different cloud desktop clients. At this time, the role of the cloud desktop server is not only a simple transmission medium, but it can also play the role of intelligent scheduling to ensure that the video stream data can be smoothly transmitted to the cloud desktop client under various network conditions. When the cloud desktop server has processed the video stream data, it will forward the data to the cloud desktop client.

[0057] After receiving the video stream data from the cloud desktop server, the cloud desktop client needs to decode and render it so that the user can see the desktop content of the cloud desktop virtual machine. The cloud desktop client usually has a built-in video decoder that can decode video stream data in compression formats such as H.264 and H.265. After decoding, the cloud desktop client presents the video data on the display. At this point, the user can perform real-time operations and experience similar to operating a local computer.

[0058] In the embodiment of the present invention, the encoding and main processing of the video stream data are completed in the cloud desktop virtual machine and the cloud desktop server, and the cloud desktop client only needs to decode and render the video stream data. This reduces the cloud desktop client's demand for graphics processing capabilities, and users can use the cloud desktop on lower-performance devices (such as thin and light notebooks, tablets, etc.), thereby greatly improving the applicability of the cloud desktop. The cloud desktop server can adapt to different network conditions by intelligently adjusting the quality and bit rate of the video stream data. When the network bandwidth is sufficient, the user can get a high-definition desktop experience; when the network conditions are poor, the cloud desktop server can reduce the resolution or compression rate of the video stream data to ensure smoothness. This dynamic adjustment mechanism ensures that users can still get a good user experience in different network environments. Reference Figure 2 , showing a flow chart of steps of another method for displaying a cloud desktop according to an embodiment of the present invention. The method for displaying a cloud desktop involves a cloud desktop virtual machine, a cloud desktop server, and a cloud desktop client. The cloud desktop server communicates with the cloud desktop virtual machine and the cloud desktop client respectively. The user of the cloud desktop client operates the cloud desktop virtual machine through the cloud desktop server. The method for displaying a cloud desktop may specifically include the following steps:

[0059] Step 201 : obtaining video stream data of a cloud desktop virtual machine, where the video stream data is obtained by capturing and encoding graphic data of the cloud desktop virtual machine after a cloud desktop user successfully logs in.

[0060] In an embodiment of the present invention, in a cloud desktop system, the graphic data of the cloud desktop virtual machine is encoded and forwarded to the cloud desktop client, so that the user can remotely control and view the real-time content of the virtual desktop.

[0061] When a cloud desktop user successfully logs in to a cloud desktop virtual machine, the operating system of the cloud desktop virtual machine starts the graphical interface and displays the user's desktop. At this time, the daemon process in the operating system starts and begins to capture all graphic data on the user's desktop, including changes in the user interface, open applications, and displayed multimedia content. The daemon process is not only responsible for capturing these graphic data, but also encodes the graphic data, usually using efficient compression algorithms such as H.264 or H.265 to reduce the size of the data.

[0062] The cloud desktop server obtains these encoded video stream data from the cloud desktop virtual machine to ensure that users can view and control the operation of the cloud desktop virtual machine in real time on the cloud desktop client. In this step, the main task of the cloud desktop server is to maintain a stable communication connection with the cloud desktop virtual machine to ensure that the video stream data of the cloud desktop virtual machine can be continuously received. This process requires an efficient and stable network transmission protocol to ensure data integrity and smooth transmission. The cloud desktop server will use remote display protocols such as SPICE or Remote Desktop Protocol (RDP) to communicate with the cloud desktop virtual machine.

[0063] For example, when a user opens a video player on a cloud desktop virtual machine, the operating system of the cloud desktop virtual machine starts a daemon process and uses the daemon process to capture the image data displayed in each frame of the player window and compress and encode it. The server receives the encoded video stream data from the cloud desktop virtual machine so that it can be subsequently transmitted to the cloud desktop client for decoding and display. The entire process ensures that the video stream data generated by the cloud desktop virtual machine can be accurately transmitted to the cloud desktop server and finally presented on the cloud desktop client.

[0064] In order to ensure the stability and real-time performance of data transmission, the cloud desktop server needs to maintain a low-latency, high-bandwidth connection with the cloud desktop virtual machine through the network. Any transmission delay or data loss may cause the screen seen by the cloud desktop client to freeze or lag, affecting the user experience. Therefore, during the data transmission process, the cloud desktop server may also perform some flow control and data verification operations to ensure the transmission quality.

[0065] Step 202: Render the window and display the video stream data in the window.

[0066] In an embodiment of the present invention, after receiving the video stream data, the cloud desktop client uses hardware decoding to process it, so as to display the graphic content of the cloud desktop virtual machine in a rendered or drawn window.

[0067] The task of the cloud desktop client is to process the video stream data through hardware decoding. Hardware decoding refers to using the GPU in the cloud desktop client device to decode the video data instead of relying on the CPU for decoding. Compared with software decoding, hardware decoding is more efficient, especially when dealing with complex encoding formats (such as H.264 or H.265), which can significantly reduce the load on the CPU and improve the overall performance of the cloud desktop client. This optimization is particularly important when the performance of the cloud desktop client device is limited, because hardware decoding can greatly reduce the CPU burden and avoid freezes or delays.

[0068] For example, when a user controls operations in a cloud desktop virtual machine on a cloud desktop client, all graphics data captured by the daemon process will be transmitted to the cloud desktop client in the form of a video stream. For example, when a user plays a high-definition video in a cloud desktop virtual machine, the cloud desktop server will transmit the encoded H.264 video stream data to the cloud desktop client. The GPU of the cloud desktop client is responsible for decoding this data and displaying the decoded image on the user's screen. Due to the high efficiency of hardware decoding, users can watch high-definition videos on the cloud desktop client with a smoothness close to local operations, without lag caused by excessive CPU resource occupation.

[0069] In addition, the cloud desktop server needs to take into account fluctuations in network conditions when forwarding data. For example, if the network bandwidth between the cloud desktop client and the cloud desktop server is low or there is network congestion, the cloud desktop server may need to adjust the frame rate or resolution of the transmission to ensure that users can smoothly receive the video stream data without causing serious degradation of picture quality or excessive delays due to network problems. The cloud desktop server may use technologies such as adaptive bit rate to optimize the transmission effect in this process.

[0070] In general, the cloud desktop server plays a crucial role in the transfer and control of the entire data transmission chain. It not only needs to efficiently receive and forward the video stream data from the cloud desktop virtual machine, but also needs to make corresponding optimization adjustments based on network conditions and cloud desktop client performance to ensure that end users can smoothly decode and view the cloud desktop's graphic content on the cloud desktop client.

[0071] In an exemplary embodiment of the present invention, before the cloud desktop server forwards the video stream data to the cloud desktop client, an implementation method is to establish a video stream transmission channel between the cloud desktop server and the cloud desktop client after the cloud desktop virtual machine is powered on and the user logs in successfully. Specifically, the cloud desktop virtual machine completes the startup, and the operating system loads and initializes the necessary system services and applications. After the user successfully logs in to the cloud desktop virtual machine, the operating system confirms the user's permissions and initializes the user session. At this stage, the communication link between the cloud desktop virtual machine and the cloud desktop server has not yet been established, so the cloud desktop server cannot receive or process the video stream data of the cloud desktop virtual machine. After the user logs in successfully, the operating system starts a daemon process, which is responsible for capturing and encoding graphic data on the cloud desktop virtual machine. At this time, the connection between the cloud desktop server and the cloud desktop client has not yet been established, so it is necessary to establish a stable transmission channel in this step. The transmission channel is a data exchange path that ensures that the video stream data generated by the cloud desktop virtual machine can be accurately and efficiently transmitted to the cloud desktop client.

[0072] The steps to establish a transmission channel include configuring network parameters, establishing a data connection, and performing protocol negotiation. The connection between the cloud desktop server and the cloud desktop client can be established through remote display protocols (such as SPICE or RDP), which support efficient video streaming and two-way transmission of user input. Through these protocols, the cloud desktop server and the cloud desktop client can negotiate transmission parameters, such as the resolution, frame rate, and compression standard of the video stream, to ensure the best image quality and smoothness during the actual transmission process.

[0073] For example, when a user performs graphic-intensive operations (such as video editing or 3D modeling) in a virtual desktop, the cloud desktop virtual machine generates a large amount of video stream data. This data needs to be transmitted to the cloud desktop client through a video stream transmission channel. If the transmission channel is established before the user logs in, the transmission channel may be unstable because the cloud desktop virtual machine has not been fully started or the cloud desktop server is not ready to receive data, thus affecting the end user experience. Therefore, establishing a transmission channel after the user successfully logs in can ensure that all systems and services are ready, and the transmission process is more stable and efficient.

[0074] The embodiment of the present invention establishes a video stream transmission channel after the user successfully logs in, ensuring that the operating system and application services are fully started, thereby improving the stability and reliability of transmission. It reduces transmission problems caused by incomplete system startup or network instability, ensures efficient transmission and high-quality display of video stream data, and improves overall user satisfaction and remote operation efficiency.

[0075] In an exemplary embodiment of the present invention, an implementation method for displaying video stream data in a window is to perform hardware decoding processing on video stream data of the Advanced Video Coding H.264 standard or the High Efficiency Video Coding H.265 standard, and display the video data after hardware decoding processing in the window.

[0076] In the cloud desktop system, the user's operating interface is presented in real time through video streaming. Video stream data is usually compressed using the H.264 or H.265 standard and transmitted from the cloud desktop virtual machine to the cloud desktop client. Hardware decoding is performed by a dedicated chip or acceleration module (such as a GPU or a dedicated video decoding chip). Compared with relying entirely on software decoding, it has higher efficiency and performance. The hardware decoder can process a large amount of image data at the same time without placing an excessive burden on the CPU of the cloud desktop client.

[0077] After the decoded video data is processed, the cloud desktop client renders the data into the window. The rendering process is usually completed through hardware acceleration, and the image is converted into pixel format and displayed directly on the screen. During this process, the operation interface of the cloud desktop client is updated in real time to ensure that every operation feedback of the user can be reflected on the screen in a timely manner. For example, when a user opens an application on the cloud desktop virtual machine, a series of operations such as generating, dragging, and minimizing the application window are displayed in a smooth video form. This process requires efficient video decoding and rendering to ensure that users do not feel obvious freezes or delays.

[0078] The embodiment of the present invention adopts hardware decoding to significantly improve the processing speed of video streams, especially in the scenario of high resolution and high frame rate, hardware decoding can realize real-time decoding and rendering, almost eliminating the freeze or delay that users may encounter when operating the cloud desktop. Compared with software decoding, hardware decoding can offload the decoding task to a dedicated processor (such as a GPU or a dedicated decoding chip), thereby reducing the CPU load. This allows the cloud desktop system to run on lower-performance client devices, thereby increasing the scope of application of the system, especially on lightweight terminal devices such as tablets, ultrabooks and other devices, hardware decoding is particularly important.

[0079] Based on the above description of a display method embodiment of a cloud desktop, a virtual machine management solution based on cloud desktop technology is introduced below. It aims to achieve efficient graphic data transmission and remote control through the collaborative work of the server, client and computing node (i.e. virtual machine) to improve user experience.

[0080] The virtual machine management solution based on cloud desktop technology consists of three key parts:

[0081] 1. Server: manages the scheduling, data transmission and protocol processing of virtual machines, and is responsible for establishing connections with clients and forwarding the graphics data of virtual machines.

[0082] 2. Client: The device used by the remote user connects to the server through the cloud desktop protocol, receives and decodes the graphics data of the virtual machine, and realizes remote operation.

[0083] 3. Computing node (virtual machine): The computing environment that actually hosts the operating system and applications, responsible for generating graphic data and encoding it into video stream data through the video stream processing process.

[0084] Specific implementation steps:

[0085] Step 1: Capture and encode graph data of computing nodes

[0086] On a computing node (i.e., a virtual machine), after the operating system is started, the system will load a daemon process. When a user successfully logs into the computing node, the daemon process begins to capture the graphics data in the virtual machine. These graphics data usually include the user's operating interface, application windows, and other content.

[0087] After capturing the graphic data, the daemon compresses and encodes it using the H.264 or H.265 standard. H.264 and H.265 are currently widely used high-efficiency video coding standards that can significantly reduce the amount of data while maintaining high image quality, thereby adapting to different network conditions.

[0088] Step 2: Establish the transmission channel between the server and the client

[0089] After the virtual machine is started and the user successfully logs in, the server will establish a stable video stream transmission channel with the client. The transmission channel is configured through a cloud desktop protocol (such as SPICE or RDP). SPICE is a commonly used cloud desktop protocol in virtual machine management, which can support efficient graphics data transmission and remote control functions.

[0090] The purpose of establishing a transmission channel is to provide an efficient, low-latency graphics data transmission path between the virtual machine and the client. Through this channel, the encoded video stream data generated by the virtual machine will be sent from the server to the client, and the integrity and real-time performance of the data during transmission will be guaranteed.

[0091] Step 3: The server forwards the video stream data to the client

[0092] When the server receives the video stream data from the virtual machine, it forwards the data to the client through the established transmission channel. Since the video stream data has been encoded through H.264 or H.265, its data volume is small and can be transmitted smoothly under limited network bandwidth.

[0093] The server is not only responsible for forwarding data, but also for handling negotiations between protocols, including parameters such as data stream compression rate, frame rate, and resolution. According to network conditions, the server can flexibly adjust the parameters of the video stream to ensure a stable remote operation experience in various network environments.

[0094] Step 4: The client receives and decodes the video stream data

[0095] After the client receives the video stream data from the server, it will use the hardware decoder to decode the data. The H.264 and H.265 standard video stream data can be efficiently processed in the hardware decoder, and the decoded graphic data will be directly displayed on the user's screen.

[0096] The advantage of hardware decoding is that it is more efficient than software decoding and can significantly reduce CPU resource consumption, thereby improving the performance of client devices. Whether using a high-performance desktop computer or a mobile device with limited resources, hardware decoding can ensure smooth display of the cloud desktop.

[0097] Step 5: User input feedback and real-time interaction

[0098] On the client, users can operate through devices such as mouse and keyboard, and these operations will be transmitted back to the server through cloud desktop protocols (such as SPICE), and then forwarded to the computing node by the server. After receiving the user's input, the computing node will perform the corresponding tasks according to the user's operation and update the graphical interface in real time.

[0099] Through the above steps, efficient transmission of graphic data from the virtual machine to the client is achieved, ensuring that the user can control the virtual machine in real time and enjoy an experience close to local operation.

[0100] The embodiment of the present invention significantly reduces the amount of video stream data generated by the virtual machine by adopting the H.264 and H.265 encoding standards, improves the transmission efficiency, and can ensure the stability of video quality and data transmission, especially in a network environment with limited bandwidth. The client uses hardware decoding to process video stream data, which reduces the occupancy of CPU resources, improves the performance of the client, and ensures that the cloud desktop can be operated smoothly on low-end devices or mobile devices. The server dynamically adjusts the video stream parameters according to the network bandwidth and delay conditions to ensure that users can obtain a good remote control experience in various network environments. The real-time transmission of user input and the instant response of the virtual machine are realized through the cloud desktop protocol, so that users can obtain an operating experience close to that of the local desktop.

[0101] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0102] Reference Figure 3 , shows a structural block diagram of a cloud desktop display system according to an embodiment of the present invention. The cloud desktop display system may specifically include the following modules.

[0103] A graphic data capture module 31 is used to capture the graphic data of the cloud desktop virtual machine after the cloud desktop user successfully logs in;

[0104] A graphic data encoding module 32, used for encoding the graphic data to obtain video stream data;

[0105] The video stream data transmission module 33 is used to transmit the video stream data to the cloud desktop client, so that the cloud desktop client renders a window and displays the video stream data in the window.

[0106] In an exemplary embodiment of the present invention, the graphic data capturing module 31 comprises:

[0107] A process startup module, used to start the daemon process on the cloud desktop virtual machine after the cloud desktop user successfully logs in;

[0108] A data acquisition module is used to capture the graphic data using the daemon process.

[0109] In an exemplary embodiment of the present invention, the graphic data encoding module 32 is used to utilize the daemon process supporting the simple independent computing environment SPICE protocol to encode the graphic data to obtain video stream data of the advanced video coding H.264 standard or the high efficiency video coding H.265 standard.

[0110] In an exemplary embodiment of the present invention, the video stream data transmission module 33 includes:

[0111] A virtual machine transmission module, used to transmit the video stream data to the cloud desktop server using the cloud desktop virtual machine;

[0112] The server forwarding module is used to forward the video stream data to the cloud desktop client through the cloud desktop server.

[0113] Reference Figure 4 , shows a structural block diagram of another cloud desktop display system according to an embodiment of the present invention. The cloud desktop display system may specifically include the following modules.

[0114] The video stream data acquisition module 41 is used to acquire the video stream data of the cloud desktop virtual machine, wherein the video stream data is obtained by capturing and encoding the graphic data of the cloud desktop virtual machine after the cloud desktop user successfully logs in;

[0115] The video stream data display module 42 is used for rendering a window and displaying the video stream data in the window.

[0116] In an exemplary embodiment of the present invention, the video stream data display module 42 is used to perform hardware decoding processing on the video stream data of the advanced video coding H.264 standard or the high efficiency video coding H.265 standard, and display the video data after the hardware decoding processing in the window.

[0117] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0118] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0119] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0120] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0123] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0124] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0125] The above is a detailed introduction to a cloud desktop display method and a cloud desktop display system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for displaying a cloud desktop, characterized in that: The method comprises: Capture the graphic data of the cloud desktop virtual machine after the cloud desktop user successfully logs in; Encoding the graphic data to obtain video stream data; The video stream data is transmitted to the cloud desktop client so that the cloud desktop client renders a window and displays the video stream data in the window.

2. The method according to claim 1, characterized in that The capturing of the graphic data of the cloud desktop virtual machine after the cloud desktop user successfully logs in includes: After the cloud desktop user successfully logs in, starting the daemon process on the cloud desktop virtual machine; The graphics data is captured using the daemon process.

3. The method according to claim 2, characterized in that The step of encoding the graphic data to obtain video stream data comprises: The graphic data is encoded using the daemon that supports the Simple Independent Computing Environment (SPICE) protocol to obtain the video stream data of the Advanced Video Coding H.264 standard or the High Efficiency Video Coding H.265 standard.

4. The method according to claim 1, characterized in that: The transmitting the video stream data to the cloud desktop client includes: Transmitting the video stream data to a cloud desktop server using the cloud desktop virtual machine; The video stream data is forwarded to the cloud desktop client through the cloud desktop server.

5. A method for displaying a cloud desktop, characterized in that: The method comprises: Acquire video stream data of a cloud desktop virtual machine, where the video stream data is obtained by capturing and encoding graphic data of the cloud desktop virtual machine after a cloud desktop user successfully logs in; A rendering window is provided, wherein the video stream data is displayed in the window.

6. The method according to claim 5, characterized in that Displaying the video stream data in the window includes: The video stream data of the advanced video coding H.264 standard or the high efficiency video coding H.265 standard is subjected to hardware decoding processing, and the video data subjected to the hardware decoding processing is displayed in the window.

7. A cloud desktop display system, characterized in that: The system comprises: A graphics data capture module is used to capture the graphics data of the cloud desktop virtual machine after the cloud desktop user successfully logs in; A graphic data encoding module, used for encoding the graphic data to obtain video stream data; The video stream data transmission module is used to transmit the video stream data to the cloud desktop client so that the cloud desktop client renders a window and displays the video stream data in the window.

8. A cloud desktop display system, characterized in that: The system comprises: A video stream data acquisition module is used to acquire video stream data of a cloud desktop virtual machine, wherein the video stream data is obtained by capturing and encoding graphic data of the cloud desktop virtual machine after a cloud desktop user successfully logs in; The video stream data display module is used for rendering a window and displaying the video stream data in the window.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the cloud desktop display method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the cloud desktop display method as described in any one of claims 1 to 6.

Images