Transmission method, system and equipment based on cloud desktop display content and medium

Abstract

The embodiment of the invention provides a transmission method and system based on cloud desktop display content, the method is applied to a server side, and the method comprises the following steps: obtaining real-time picture data based on full-screen canvas; encoding the real-time picture data; and transmitting the coded real-time picture data to a client. According to the embodiment of the invention, the picture change is integrated through the real-time picture data based on the full-screen canvas, so that the server can uniformly manage the picture update, thereby improving the control capability of the transmission frame rate, and being not limited by the local command frequency. Meanwhile, a complete data basis is provided for real-time picture data transmission based on the full-screen canvas, the picture updating stability is improved, and the smoothness of a remote desktop is improved. In addition, the encoded real-time picture data transmission reduces the network burden of dispersed transmission, and optimizes the transmission efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and particularly to a method for transmitting display content based on a cloud desktop, a system for transmitting display content based on a cloud desktop, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the transmission of virtual machine desktop display content using the existing Simple Protocol for Independent Computing Environments (SPICE) protocol, when the server receives a rendering command from the virtual machine, each command is separately compressed, and the compressed data is sent to the client for display.

[0003] Since the change of the screen usually occurs only in a local area and each command represents only a part of the image, this per-command processing method makes it difficult for the server to effectively control the transmission frame rate. When rendering commands are generated frequently, the screen update speed cannot be stable, affecting the smoothness of the remote desktop. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method for transmitting display content based on a cloud desktop, a system for transmitting display content based on a cloud desktop, an electronic device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0005] To solve the above problems, an embodiment of the present invention discloses a method for transmitting display content based on a cloud desktop, which is applied to a server, and the method includes:

[0006] Obtain real-time screen data based on a full-screen canvas;

[0007] Encode the real-time screen data;

[0008] Transmit the encoded real-time screen data to the client.

[0009] Optionally, the obtaining real-time screen data based on a full-screen canvas includes:

[0010] Receive and parse a rendering instruction;

[0011] Add the parsed rendering instruction to a command tree;

[0012] Traverse the command tree, execute the rendering instructions in the command tree, render the marked changed areas, and update the pixel data of the full-screen canvas to obtain the real-time screen data.

[0013] Optionally, rendering the area marked as changed and updating the pixel data of the full-screen canvas to obtain the real-time video data includes:

[0014] Performing a rendering operation on the changed area to generate a rendering result;

[0015] Updating the pixel data of the full-screen canvas according to the rendering result, and obtaining the real-time video data through incremental rendering.

[0016] Optionally, before encoding the real-time video data, the method further includes:

[0017] Adding the real-time video data to a video queue;

[0018] Losslessly compressing the real-time video data in the video queue.

[0019] Optionally, adding the real-time video data to the video queue includes:

[0020] Storing the real-time video data in blocks in the video queue;

[0021] Performing priority sorting on the data blocks in the video queue according to the importance of the video content;

[0022] Among them, the priority of the data block corresponding to the key frame is higher than that of the data block corresponding to the non-key frame.

[0023] Optionally, encoding the real-time video data includes:

[0024] Detecting the current transmission condition;

[0025] Adjusting the encoding parameters according to the transmission condition;

[0026] Encoding the real-time video data by using the adjusted encoding parameters.

[0027] Optionally, adjusting the encoding parameters according to the transmission condition includes:

[0028] When the transmission condition allows high bandwidth, increasing the encoding frame rate and / or bit rate;

[0029] When the transmission condition is limited, reducing the encoding frame rate and / or bit rate.

[0030] An embodiment of the present invention also discloses a transmission system for cloud desktop display content, and the system includes:

[0031] A video data acquisition module, configured to acquire real-time video data based on a full-screen canvas;

[0032] A screen data encoding module for encoding the real-time screen data;

[0033] A screen data transmission module for transmitting the encoded real-time screen data to the client.

[0034] Optionally, the screen data acquisition module includes:

[0035] An instruction receiving and parsing module for receiving and parsing rendering instructions;

[0036] A command tree adding module for adding the parsed rendering instructions to the command tree;

[0037] A rendering update module for traversing the command tree, executing the rendering instructions in the command tree, rendering the marked changed areas, and updating the pixel data of the full-screen canvas to obtain the real-time screen data.

[0038] Optionally, the rendering update module includes:

[0039] A rendering result generation module for performing a rendering operation on the changed areas and generating a rendering result;

[0040] A pixel data update module for updating the pixel data of the full-screen canvas according to the rendering result and obtaining the real-time screen data through incremental rendering.

[0041] Optionally, the system further includes:

[0042] A screen queue adding module for adding the real-time screen data to the screen queue before the screen data encoding module encodes the real-time screen data;

[0043] A lossless compression module for performing lossless compression on the real-time screen data in the screen queue.

[0044] Optionally, the screen queue adding module includes:

[0045] A block storage module for storing the real-time screen data in blocks in the screen queue;

[0046] A data block sorting module for sorting the data blocks in the screen queue according to the importance of the screen content;

[0047] Among them, the priority of the data block corresponding to the key frame is higher than that of the data block corresponding to the non-key frame.

[0048] Optionally, the screen data encoding module includes:

[0049] A transmission condition detection module for detecting the current transmission condition;

[0050] An encoding parameter adjustment module for adjusting encoding parameters according to the transmission conditions;

[0051] A real-time data encoding module for encoding the real-time screen data by using the adjusted encoding parameters.

[0052] Optionally, the encoding parameter adjustment module is configured to increase the encoding frame rate and / or bit rate when the transmission conditions allow high bandwidth; and decrease the encoding frame rate and / or bit rate when the transmission conditions are limited.

[0053] An embodiment of the present invention also discloses an electronic device, including: one or more processors; and one or more machine-readable media storing instructions thereon, which when executed by the one or more processors, cause the electronic device to execute the transmission method for cloud desktop display content as described above.

[0054] An embodiment of the present invention also discloses a computer-readable storage medium, and a computer program stored thereon causes a processor to execute the transmission method for cloud desktop display content as described above.

[0055] The embodiments of the present invention include the following advantages:

[0056] The transmission solution for cloud desktop display content provided by the embodiments of the present invention is applied to the server side. The transmission solution acquires real-time screen data based on a full-screen canvas; encodes the real-time screen data; and transmits the encoded real-time screen data to the client side.

[0057] Compared with the background art, the embodiments of the present invention have the following beneficial effects:

[0058] The transmission solution for cloud desktop display content provided by the embodiments of the present invention integrates the screen changes through real-time screen data based on a full-screen canvas, enabling the server side to uniformly manage the screen updates, thereby improving the control ability of the transmission frame rate and being not limited by the local command frequency. At the same time, the transmission of real-time screen data based on a full-screen canvas provides a complete data basis, improves the stability of the screen updates, and enhances the fluency of the remote desktop. In addition, the transmission of the encoded real-time screen data reduces the network burden of decentralized transmission and optimizes the transmission efficiency. Description of the Drawings

[0059] Figure 1 is a step flowchart of a transmission method for cloud desktop display content according to an embodiment of the present invention;

[0060] Figure 2 is a schematic diagram of a full-screen H.264 cloud desktop solution based on the vision network according to an embodiment of the present invention;

[0061] Figure 3It is a block diagram of a transmission system for cloud desktop display content according to an embodiment of the present invention. Detailed implementation manners

[0062] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0063] An embodiment of the present invention provides a cloud desktop display content transmission solution applied to a server, one of the purposes is to optimize the remote transmission efficiency and stability of cloud desktop (such as virtual machine desktop) display content. A cloud desktop is a virtual desktop solution based on cloud computing technology. By migrating the desktop environment from local devices to a cloud server, it provides users with a flexible, efficient, and secure remote desktop access experience. The cloud desktop uses virtualization technology to create virtual machines on the server, runs a complete desktop operating system, and users access the virtual desktop through a client device via the network to obtain the same usage experience as the local desktop. This solution obtains real-time frame data based on a full-screen canvas, integrates the cloud desktop display content into a unified complete frame. Subsequently, the real-time frame data is encoded to meet different transmission requirements. Finally, the encoded real-time frame data is transmitted to the client to achieve efficient and stable remote display. Compared with the traditional command-by-command transmission method, this solution significantly improves the frame rate control ability, picture update stability, and network transmission efficiency through the unified management and encoding optimization of the full-screen canvas.

[0064] Refer to Figure 1 , which shows a step flowchart of a method for transmitting cloud desktop display content according to an embodiment of the present invention. This method for transmitting cloud desktop display content can be applied to a server, and the server can be a system such as a video processing system (hereinafter simply referred to as the system). Among them, the cloud desktop can include but is not limited to virtual machine desktops, etc. Hereinafter, the cloud desktop is taken as an example of a virtual machine desktop for explanation. The method for transmitting cloud desktop display content can specifically include the following steps:

[0065] Step 101, obtain real-time frame data based on a full-screen canvas.

[0066] A full-screen canvas refers to a virtual display area that covers the entire desktop or the complete range of a specified interface, and can capture and reflect all picture changes in real time. Different from the traditional block-by-block or command-by-command capture method, the method for obtaining real-time frame data based on a full-screen canvas generates complete frame data by centrally processing all rendering instructions, avoiding the dispersion of local updates, ensuring that the picture data is presented in an overall form, and laying a foundation for subsequent encoding and transmission.

[0067] Specifically, the system receives rendering instructions generated by the virtual machine, which may include operations such as drawing graphics, filling colors, or updating text. By parsing these instructions, the system converts them into updates of pixel data on the full-screen canvas. The rendering process typically adopts a full-screen perspective, that is, no matter where the screen changes occur, they are ultimately reflected on the full-screen canvas. For example, when the user drags a window on the virtual machine desktop, the system captures the area changes caused by the window movement and integrates them into the full-screen canvas to form a complete frame of real-time video data. In addition, the pixel data of the full-screen canvas is usually stored in a standard format so that the subsequent encoding module can directly recognize and process it.

[0068] In the actual application process, the implementation of obtaining real-time video data depends on the system's rendering engine and data buffering mechanism. The rendering engine is responsible for mapping the instructions to the pixel points on the full-screen canvas, while the buffering mechanism ensures that the data can be quickly transferred to the next step after generation, avoiding video lag caused by processing delays.

[0069] Step 102, encode the real-time video data.

[0070] The encoding process starts with the real-time video data extracted from the full-screen canvas. These data are usually complete frames stored in pixel form, containing information about the entire display area. The system compresses this data through encoding techniques to reduce the data volume, thereby reducing the bandwidth requirements during network transmission and ensuring that the client can decode and restore the video content.

[0071] Specific implementations of encoding can use a variety of video compression algorithms. Common ones include Advanced Video Coding (H.264 / MPEG-4 Part 10, Advanced Video Coding, abbreviated as H.264), etc. However, this step does not limit the specific algorithm, but emphasizes the overall processing of real-time video data. For example, the system can input the real-time video data into an encoder, and the encoder generates compressed video frames according to preset parameters (such as frame rate, bit rate). These video frames may be key frames (complete video data) or predicted frames (data based on the differences between previous and subsequent frames), depending on the encoding strategy. The advantage of encoding is that it converts the real-time video data on the full-screen canvas from its original high-data-volume state into a more compact form. For example, an uncompressed 1080p frame may occupy several megabytes of space, while after encoding, it may be reduced to several hundred kilobytes, significantly reducing the transmission burden.

[0072] In the actual application process, the encoding process involves data format conversion and the application of compression algorithms. Real-time video data may be converted from the RGB format to the YUV format to meet the input requirements of most encoders. Subsequently, the encoder performs operations such as chunking, transformation, and quantization on the data, and finally outputs the encoded real-time video data. In addition, the system needs to ensure that the encoded data is compatible with the decoding capabilities of the client to ensure that the transmitted video can be accurately restored.

[0073] Step 103: Transmit the encoded real-time video data to the client.

[0074] The encoded real-time video data is complete frame data generated based on the full-screen canvas and exists in the form of a video stream or data packets after compression. The system transmits this data to the client through a network protocol to ensure that the video can be decoded and presented to the user on the client. Compared with traditional command-by-command transmission, this method is based on the full-screen canvas, reducing data dispersion and improving the integrity of transmission.

[0075] The specific implementation of transmission depends on network communication technologies. The system usually uses standard transmission protocols such as the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP) to complete data sending. For example, using the TCP protocol can ensure the reliability of data transmission and ensure that the encoded real-time video data arrives at the client in order. UDP may be used to pursue lower latency and is suitable for scenarios with higher real-time requirements. During the transmission process, the system encapsulates the encoded real-time video data into network data packets and sends them to the corresponding interface of the client through the network interface of the server. After receiving the data, the client needs to decode and render it on the display device. For example, when a user performs a window dragging operation on a virtual machine desktop, the encoded real-time video data generated by the server is transmitted to the client through the network, and the client decodes and displays the complete window movement video. This process ensures the real-time update of the remote desktop.

[0076] In the actual application process, transmission involves the entire process of data encapsulation, packetization, and sending. The encoded real-time video data may be split into multiple data packets, and the system needs to ensure that each data packet carries sufficient information (such as frame identifiers) for the client to correctly reassemble.

[0077] The transmission solution based on the display content of the cloud desktop provided by the embodiments of the present invention is applied to the server. This transmission solution acquires real-time video data based on the full-screen canvas, encodes the real-time video data, and transmits the encoded real-time video data to the client.

[0078] Compared with the background art, the embodiments of the present invention have the following beneficial effects:

[0079] The real-time screen data transmission solution based on the cloud desktop display content provided by the embodiments of the present invention integrates screen changes through real-time screen data based on the full-screen canvas, enabling the server to uniformly manage screen updates, thereby improving the control ability of the transmission frame rate and being unrestricted by the local command frequency. At the same time, the real-time screen data transmission based on the full-screen canvas provides a complete data basis, improves the stability of screen updates, and enhances the fluency of the remote desktop. In addition, the transmission of the encoded real-time screen data reduces the network burden of scattered transmission and optimizes the transmission efficiency.

[0080] In an exemplary embodiment of the present invention, one implementation of obtaining real-time screen data based on the full-screen canvas is as follows: receive and parse rendering instructions; add the parsed rendering instructions to the command tree; traverse the command tree, execute the rendering instructions in the command tree, render the marked changed areas, and update the pixel data of the full-screen canvas to obtain real-time screen data.

[0081] The server receives and parses the rendering instructions. The rendering instructions are usually generated by the virtual machine and contain operation instructions such as drawing graphics, filling colors, or updating text. The received rendering instructions may be in the form of raw data, such as coordinates, color values, etc. The server converts them into recognizable operation content through parsing. For example, when the user drags a window on the virtual machine desktop, the server receives an instruction indicating the change in the window position. After parsing, it is clear which areas of the screen need to be updated. The parsed rendering instructions lay the foundation for subsequent processing, ensuring that the system can understand and execute these instructions.

[0082] Next, add the parsed rendering instructions to the command tree. The command tree is a tree-like data structure used to organize and manage rendering instructions for efficient subsequent processing. The parsed instructions are added to the command tree one by one. Each node may represent a specific rendering operation, such as drawing a rectangle or filling an area. This structured storage method makes the instructions no longer scattered but form an ordered set. For example, window dragging involves multiple area updates. The command tree arranges these operations in order to ensure that no operations are missed or repeated during rendering.

[0083] Subsequently, the server traverses the command tree, executes the rendering instructions therein, and renders the areas marked as changed, updating the pixel data of the full-screen canvas to obtain real-time frame data. During the traversal process, each rendering instruction is processed one by one in the order of the command tree. The rendering is only performed on the areas marked as changed (i.e., the "dirty regions"), rather than the entire canvas. This incremental rendering method reduces the computational load. For example, when the window is dragged, only the regions with differences before and after the window movement are rendered, and the other unchanged parts remain as they are. The rendering result is updated to the pixel data of the full-screen canvas, usually stored in the Red Green Blue (RGB) or Alpha Red Green Blue (ARGB) format. The full-screen canvas serves as a unified virtual display area, integrating all changes and finally generating real-time frame data.

[0084] In this embodiment, by receiving and parsing the rendering instructions, the system accurately captures the frame changes. The command tree organizes the instructions to improve the processing efficiency. Rendering and updating the full-screen canvas for the changed areas not only reduces the computational burden but also ensures the integrity of the frame. Compared with the traditional scattered processing of each command, it improves the acquisition efficiency and stability of the real-time frame data.

[0085] In an exemplary embodiment of the present invention, an implementation manner of rendering the areas marked as changed and updating the pixel data of the full-screen canvas to obtain real-time frame data is as follows: perform a rendering operation on the changed areas to generate a rendering result; update the pixel data of the full-screen canvas according to the rendering result, and obtain the real-time frame data through the incremental rendering method.

[0086] Perform a rendering operation on the changed areas to generate a rendering result. The changed areas (often referred to as "dirty regions") refer to the parts of the full-screen canvas where the content has been updated since the last rendering, usually marked by the server according to the rendering instructions or pixel differences. When performing the rendering operation, the server only processes these areas marked as changed, rather than redrawing the entire full-screen canvas. For example, when the user enters text on the virtual machine desktop, only the text input area is marked as the changed area, and the server updates the pixel values of this area according to the corresponding rendering instructions (such as drawing characters) to generate a rendering result. The rendering result is usually pixel data represented in the RGB format, reflecting the latest frame content of the changed area.

[0087] Next, update the pixel data of the full-screen canvas according to the rendering result, and obtain real-time video data through incremental rendering. The full-screen canvas, as a unified virtual display area, stores the pixel data of the entire desktop, usually represented in ARGB or RGB format. During the update process, the rendering result generated in the previous step is written to the changed area corresponding to the full-screen canvas, while keeping the pixel data in other areas unchanged. For example, if the text input only affects a small area of the desktop, the server will overwrite the rendering result of this area (such as the pixel values of the new characters) onto the full-screen canvas to form the updated complete video data. The incremental rendering method means that only the changed area is incrementally updated, rather than redrawing the entire screen, ensuring that the generation of real-time video data is both fast and efficient. Finally, the updated pixel data of the full-screen canvas is the real-time video data, which directly reflects the current state of the virtual machine desktop and can be used for subsequent encoding and transmission.

[0088] This implementation depends on the rendering engine and pixel data management mechanism of the server. The rendering engine is responsible for performing the rendering operations on the changed area to generate accurate pixel updates, and the update of pixel data is achieved through memory mapping or buffer operations to ensure real-time performance and consistency. By rendering the results for the changed area and incrementally updating the full-screen canvas, compared with the background technology of processing the entire screen command by command, this method reduces redundant calculations, improves the generation efficiency of real-time video data, and maintains the integrity of the video while providing a high-quality data basis for efficient transmission.

[0089] In an exemplary embodiment of the present invention, one implementation before encoding the real-time video data is: adding the real-time video data to a video queue; performing lossless compression on the real-time video data in the video queue.

[0090] Real-time video data refers to the complete frame data generated based on the full-screen canvas, usually stored in RGB or ARGB format, and contains the current display content of the virtual machine desktop. The video queue is a data structure used to temporarily store and manage real-time video data to ensure that it can be processed in order before encoding. After the server adds the real-time video data to the video queue, these data are arranged in frames to form an ordered sequence of data to be processed. For example, when the user continuously scrolls the page on the virtual machine desktop, the server will generate multiple frames of real-time video data, and each frame is added to the video queue in turn to avoid data loss or confusion. The key to this process lies in the buffering function of the queue. The system can adjust the access rhythm of the data according to the processing capacity or network conditions to provide a stable data stream for subsequent steps.

[0091] Next, lossless compression is performed on the real-time video data in the video queue. Lossless compression refers to a processing method that reduces the data volume without losing video information. Common algorithms include Run-Length Encoding (RLE for short). The server retrieves the real-time video data from the video queue and applies lossless compression technology to compress the original data into a more compact format. For example, for a frame of real-time video data containing a large area of a single color (such as a desktop background), RLE can simplify the continuously repeated pixel values into a short description, thus significantly reducing the data volume. The compressed real-time video data still retains all the original information, ensuring that the video is restored without loss after decompression by the client.

[0092] This embodiment relies on the data queue management and compression algorithm of the server to implement. The video queue is usually maintained through a memory buffer, supporting dynamic capacity adjustment. Lossless compression needs to ensure real-time performance, that is, the compression speed matches the generation frequency of the video data. By adding the real-time video data to the video queue and performing lossless compression, the data management order is optimized, and the memory and bandwidth requirements are reduced. Compared with the prior art of direct transmission by each command, the data processing efficiency before encoding and the stability of transmission preparation are significantly improved.

[0093] In an exemplary embodiment of the present invention, one way to add real-time video data to the video queue is: storing the real-time video data in blocks in the video queue; sorting the data blocks in the video queue according to the importance of the video content; among them, the priority of the data block corresponding to the key frame is higher than that of the data block corresponding to the non-key frame.

[0094] Storing in blocks means dividing the entire frame of real-time video data into multiple smaller data blocks, each data block representing a part of the video, and then storing them in the video queue. The video queue is an ordered structure for temporarily storing and managing real-time video data. Through block storage, the server can process the data with a finer granularity. For example, when a user plays a video on a virtual machine desktop, the server can divide a frame of real-time video data into multiple regions (such as a video region and a background region) and store them in the video queue respectively.

[0095] The importance of the video content is usually determined by the frame type corresponding to the data block, such as key frames and non-key frames. The server assigns priorities to each data block according to preset rules, where the priority of the data block corresponding to the key frame is higher than that of the data block corresponding to the non-key frame. Key frames are usually independent frames of the complete video (such as the starting frame of a video), while non-key frames are prediction frames based on differences (such as transition frames in a video). For example, in a video playback scenario, the server marks the key frame data block as high priority to ensure its priority processing, while the non-key frame data block is ranked later.

[0096] This implementation mode relies on the data chunking algorithm and priority management mechanism of the server. Chunk storage can be achieved through fixed-size splitting or dynamic area partitioning, and priority sorting is completed based on frame type identifiers (such as key frame markers). By storing the real-time video data in chunks and sorting them by importance, the fineness of queue management and the processing order are optimized. Compared with the disorderly transmission of commands one by one in the background technology, the efficiency of data processing and the transmission priority of key content are significantly improved, providing higher-quality data support for subsequent encoding and transmission.

[0097] In an exemplary embodiment of the present invention, an implementation mode of encoding real-time video data is as follows: detecting the current transmission conditions; adjusting the encoding parameters according to the transmission conditions; and encoding the real-time video data using the adjusted encoding parameters.

[0098] Real-time video data refers to the complete frame data generated based on the full-screen canvas and includes the current display content of the virtual machine desktop. Transmission conditions include parameters such as network bandwidth, latency, and packet loss rate, which reflect the communication status between the server and the client. The server detects these conditions in real time through a network monitoring module, for example, measuring whether the current bandwidth is sufficient or whether the latency is higher than the threshold. For example, when a user watches a video through a remote desktop, the server may detect that the bandwidth has dropped from 10 Mbps to 2 Mbps, and this information provides a basis for subsequent adjustment of the encoding strategy.

[0099] Encoding parameters include frame rate, bit rate, resolution, etc., which directly affect the compression degree and quality of real-time video data. The server dynamically adjusts these parameters according to the detection results. For example, when the bandwidth is sufficient, the frame rate is increased (such as from 15 frames per second to 30 frames per second) to improve smoothness, and when the bandwidth is limited, the bit rate is decreased (such as from 2 Mbps to 500 Kbps) to reduce the amount of data. This adjustment ensures that the encoded real-time video data adapts to the current transmission conditions. For example, when the network latency is high, the server may preferentially reduce the resolution to ensure transmission stability. The adjustment process relies on preset rules or algorithms to ensure smooth parameter changes and meet the requirements of video quality.

[0100] Subsequently, the server applies the adjusted parameters to the encoder to compress the real-time video data and generate a video stream suitable for transmission. The encoding may use standards such as H.264, outputting key frames and predictive frames. For example, the adjusted high frame rate parameter can generate a smoother video stream, while the low bit rate parameter compresses the data to adapt to narrow bandwidths. The encoded real-time video data is stored in a compact format for easy transmission to the client.

[0101] This embodiment relies on network detection technology and encoder parameter control. By detecting transmission conditions, adjusting encoding parameters, and using them for encoding, it dynamically adapts to network changes. Compared with the fixed per-command transmission in the background technology, it significantly improves encoding flexibility and transmission stability, ensuring a balance between picture quality and efficiency.

[0102] In an exemplary embodiment of the present invention, one way to adjust encoding parameters according to transmission conditions is as follows: when the transmission conditions allow for high bandwidth, increase the encoding frame rate and / or bit rate; when the transmission conditions are limited, decrease the encoding frame rate and / or bit rate.

[0103] The encoding frame rate refers to the number of video frames generated per second, and the bit rate is the amount of data transmitted per second. Both directly affect the smoothness and clarity of the picture. When the server detects high bandwidth (such as above 10 Mbps), it will increase the frame rate (such as adjusting from 15 frames per second to 30 frames per second) and / or bit rate (such as increasing from 1 Mbps to 3 Mbps). For example, when a user watches a high-definition video on a virtual machine desktop, high bandwidth allows the server to increase the frame rate to generate a smoother picture or increase the bit rate to provide a higher clarity image.

[0104] Limited transmission conditions usually refer to insufficient bandwidth (such as below 2 Mbps) or high latency. The server identifies these situations based on the detection results. In this case, the server will decrease the frame rate (such as from 30 frames per second to 15 frames per second) and / or bit rate (such as from 3 Mbps to 500 Kbps) to reduce the volume of real-time picture data after encoding. For example, when network congestion causes a decrease in bandwidth, the server decreases the frame rate to maintain basic smoothness or reduces the bit rate to avoid transmission interruptions.

[0105] This embodiment relies on the network monitoring module and encoder parameter control mechanism of the server. The monitoring module collects transmission condition data in real time, and the encoder adjusts parameters according to preset thresholds (such as bandwidth upper and lower limits). By increasing the frame rate and / or bit rate at high bandwidth and decreasing them at limited bandwidth, it dynamically matches the transmission conditions. Compared with the fixed per-command transmission in the background technology, it significantly improves the flexibility of encoding and the stability of picture transmission, ensuring efficient adaptation in different network environments.

[0106] Based on the above related descriptions of the embodiments of a transmission method for cloud desktop display content, the following introduces a full-screen H.264 cloud desktop solution based on the Visual Networking, which is applied to the server side (such as a video processing system). By obtaining real-time picture data based on a full-screen canvas, encoding the real-time picture data, and transmitting it to the client side, an efficient and stable remote desktop display function is realized. Visual Networking is a high-speed network architecture based on video communication, which can carry high-quality video stream transmission. This solution makes full use of its characteristics, combines the full-screen canvas and H.264 technology to optimize the transmission efficiency and picture quality of virtual machine desktop content. Refer to Figure 2 , which shows a schematic diagram of a full-screen H.264 cloud desktop solution based on the Visual Networking according to an embodiment of the present invention.

[0107] This solution obtains real-time picture data based on a full-screen canvas, which may specifically include: the server receives and parses rendering instructions (such as drawing graphics or updating text) generated by the virtual machine. Add the parsed rendering instructions to the command tree. Traverse the command tree, execute the rendering instructions therein, and only render the marked changed areas to update the pixel data of the full-screen canvas to generate real-time picture data. As Figure 2 shown, the server first receives video instructions ( Figure 2 marked as "drawing instruction a" in

[0108] ). The server parses these video instructions and converts them into pixel updates on the full-screen canvas. The full-screen canvas, as a unified virtual display area, covers the entire desktop and stores pixel data represented in RGB or ARGB format. For the changed areas (such as the "dirty area" caused by window dragging), the server performs a rendering operation to generate a rendering result and updates the pixel data of the full-screen canvas by means of incremental rendering to ensure that the real-time picture data reflects the latest state. Figure 2As shown, the server stores the real-time screen data in chunks into the screen queue (marked as "server canvas" in the figure), and sets priorities according to the importance of the screen content. For example, in a video playback scenario, the server divides the real-time screen data into multiple data chunks (such as video area and background area), and arranges the key frame data chunks first. After compression, the storage and transmission burdens are reduced. The screen queue ensures that the data is processed in order, the priority sorting ensures that the key content is encoded first, and lossless compression reduces the data volume without losing information, providing an efficient input for subsequent H.264 encoding.

[0109] Perform H.264 encoding on the real-time screen data. The specific implementation methods include: detecting the current transmission conditions (such as the bandwidth and latency of the visual network); adjusting the encoding parameters according to the transmission conditions; and encoding the real-time screen data using the adjusted encoding parameters. The adjustment strategy is: when the visual network transmission conditions allow high bandwidth, increase the encoding frame rate (such as from 15 frames per second to 30 frames per second) and / or bit rate (such as from 1 Mbps to 3 Mbps); when the transmission conditions are limited, reduce the encoding frame rate and / or bit rate (such as to 500 Kbps). For example, under high bandwidth, a smoother high-definition video stream is encoded, and when limited, the data is compressed to ensure transmission stability. H.264 encoding converts the real-time screen data into a video stream of key frames and predicted frames, making full use of its efficient compression characteristics (such as multi-reference frame prediction), significantly reducing the bandwidth requirement while maintaining the screen quality. Figure 2 In the [description], after the server completes the encoding, it generates the real-time screen data encoded for the full-screen canvas and prepares to transmit it to the client.

[0110] Finally, the server transmits the encoded real-time screen data to the client through the visual network. The client canvas receives and decodes the data, presenting the complete virtual machine desktop content. The high-speed transmission ability of the visual network ensures that the H.264 video stream can be delivered in real time. For example, when a user remotely watches a video, the encoded real-time screen data is transmitted in the form of a single data packet, reducing the network burden caused by scattered data packets and improving the transmission efficiency.

[0111] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequences, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0112] Refer to Figure 3 , which shows a structural block diagram of a transmission system for cloud desktop display content according to an embodiment of the present invention. The transmission system for cloud desktop display content may specifically include the following modules.

[0113] A screen data acquisition module 31, configured to acquire real-time screen data based on a full-screen canvas;

[0114] A screen data encoding module 32, configured to encode the real-time screen data;

[0115] A screen data transmission module 33, configured to transmit the encoded real-time screen data to a client.

[0116] In an exemplary embodiment of the present invention, the screen data acquisition module 31 includes:

[0117] An instruction receiving and parsing module, configured to receive and parse rendering instructions;

[0118] A command tree adding module, configured to add the parsed rendering instructions to a command tree;

[0119] A rendering update module, configured to traverse the command tree, execute the rendering instructions in the command tree, render the marked changed areas, and update the pixel data of the full-screen canvas to obtain the real-time screen data.

[0120] In an exemplary embodiment of the present invention, the rendering update module includes:

[0121] A rendering result generation module, configured to perform a rendering operation on the changed areas to generate a rendering result;

[0122] A pixel data update module, configured to update the pixel data of the full-screen canvas according to the rendering result, and obtain the real-time screen data through an incremental rendering method.

[0123] In an exemplary embodiment of the present invention, the system further includes:

[0124] A screen queue adding module, configured to add the real-time screen data to a screen queue before the screen data encoding module 32 encodes the real-time screen data;

[0125] A lossless compression module, configured to perform lossless compression on the real-time screen data in the screen queue.

[0126] In an exemplary embodiment of the present invention, the screen queue adding module includes:

[0127] A block storage module, configured to store the real-time screen data in blocks to the screen queue;

[0128] A data block sorting module, configured to perform priority sorting on the data blocks in the screen queue according to the importance of the screen content;

[0129] Among them, the priority of the data block corresponding to the key frame is higher than that of the data block corresponding to the non-key frame.

[0130] In an exemplary embodiment of the present invention, the video data encoding module 32 includes:

[0131] A transmission condition detection module, configured to detect the current transmission condition;

[0132] An encoding parameter adjustment module, configured to adjust the encoding parameters according to the transmission condition;

[0133] A real-time data encoding module, configured to encode the real-time video data by using the adjusted encoding parameters.

[0134] In an exemplary embodiment of the present invention, the encoding parameter adjustment module is configured to increase the encoding frame rate and / or bit rate when the transmission condition allows high bandwidth; and decrease the encoding frame rate and / or bit rate when the transmission condition is limited.

[0135] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment.

[0136] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0137] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a device, or a computer program product. Therefore, the embodiments of the present invention can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate for implementing in the process Figure 1 a process or multiple processes and / or blocks Figure 1a device for the functions specified in one or more boxes.

[0139] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device that implements the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 a box or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 a box or more boxes.

[0141] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0142] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0143] The above has introduced in detail a method for transmitting display content based on a cloud desktop and a system for transmitting display content based on a cloud desktop. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A transmission method for the display content of a cloud desktop, characterized in that, Applied to the server side, the method includes: Obtain real-time screen data based on a full-screen canvas; Encode the real-time screen data; Transmit the encoded real-time screen data to the client.

2. The method according to claim 1, wherein The obtaining of the real-time screen data based on a full-screen canvas includes: Receive and parse a rendering instruction; Add the parsed rendering instruction to a command tree; Traverse the command tree, execute the rendering instructions in the command tree, render the marked changed areas, and update the pixel data of the full-screen canvas to obtain the real-time screen data.

3. The method according to claim 2, wherein The rendering of the marked changed areas and the updating of the pixel data of the full-screen canvas to obtain the real-time screen data include: Execute a rendering operation for the changed areas to generate a rendering result; Update the pixel data of the full-screen canvas according to the rendering result, and obtain the real-time screen data through incremental rendering.

4. The method according to claim 1, characterized in that, Before encoding the real-time screen data, the method further includes: Add the real-time screen data to a screen queue; Perform lossless compression on the real-time screen data in the screen queue.

5. The method according to claim 4, wherein The adding of the real-time screen data to the screen queue includes: Store the real-time screen data in blocks in the screen queue; Sort the data blocks in the screen queue according to the importance of the screen content; Among them, the priority of the data block corresponding to the key frame is higher than that of the data block corresponding to the non-key frame.

6. The method according to claim 1, characterized in that, The encoding of the real-time screen data includes: Detect the current transmission condition; Adjust the encoding parameters according to the transmission condition; Encode the real-time screen data using the adjusted encoding parameters.

7. The method according to claim 6, characterized in that, The adjusting of the encoding parameters according to the transmission condition includes: When the transmission condition allows high bandwidth, increase the encoding frame rate and / or bit rate; When the transmission condition is limited, decrease the encoding frame rate and / or bit rate.

8. A transmission system based on the display content of a cloud desktop, characterized in that, The system includes: A screen data acquisition module for obtaining real-time screen data based on a full-screen canvas; A screen data encoding module for encoding the real-time screen data; A screen data transmission module for transmitting the encoded real-time screen data to the client.

9. An electronic device, characterized in that, Includes: One or more processors; And One or more machine-readable media storing instructions, which when executed by the one or more processors, cause the electronic device to execute the transmission method of the cloud desktop display content according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program stored therein causes the processor to execute the transmission method of the cloud desktop display content according to any one of claims 1 to 7.

Images