Virtual desktop video acceleration method based on SPICE protocol and related device

Abstract

The invention provides a virtual desktop video acceleration method based on an SPICE protocol and a related device, and relates to the technical field of computer networks and virtualization. Through real-time identification of change areas, judgment of area types through frequency analysis, adaptive coding and matched decoding, the problem of poor video transmission performance in the prior art is effectively solved, video and non-video areas can be accurately identified, an adaptive coding strategy is implemented, redundant data transmission is effectively reduced, the decoding efficiency is improved, and the video transmission efficiency is improved. Therefore, the fluency of the virtual desktop video and the user experience are improved.

Description

Technical Field

[0001] This application relates to the fields of computer network and virtualization technology, and in particular to a virtual desktop video acceleration method and related apparatus based on the SPICE protocol. Background Technology

[0002] In virtual desktop infrastructure, the SPICE protocol, as a mainstream remote display protocol, directly impacts user experience through video transmission performance. Existing technologies attempt to distinguish dynamic content areas by statistically analyzing screen change frequency through region identification mechanisms, but this process has significant flaws. Servers typically capture screen images using fixed-period sampling, lacking pixel-level difference analysis capabilities, making it impossible to accurately filter static areas, resulting in a large amount of redundant data being included in the processing flow. In the video detection stage, existing solutions rely solely on simple thresholds to determine update frequency, failing to combine content features for comprehensive analysis. This easily leads to misclassifying non-video dynamics such as scrolling text as video content or overlooking genuine video areas. Regarding encoding strategies, the server uses a uniform compression algorithm for all changing areas, failing to enable inter-frame prediction optimization for video areas and over-relying on lossless compression for non-video areas, resulting in wasted bandwidth resources and image quality loss. During client-side decoding, hardware decoding functions are not dynamically matched to the encoding type. Video areas fail to fully utilize GPU acceleration capabilities, while decoding non-video areas excessively consumes CPU resources, causing playback stuttering and latency issues. Furthermore, insufficient compatibility of coordinate comparison algorithms in cross-platform environments and low screen data acquisition efficiency across different virtual machine architectures further exacerbate transmission bottlenecks. These defects result in insufficient smoothness of virtual desktop video playback, severely impacting the user's interactive experience.

[0003] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0004] This application provides a virtual desktop video acceleration method and related apparatus based on the SPICE protocol, which can accurately identify video and non-video areas, implement adaptive encoding strategies, effectively reduce redundant data transmission, improve decoding efficiency, and thus improve the smoothness and user experience of virtual desktop videos.

[0005] Firstly, the virtual desktop video acceleration method based on the SPICE protocol provided in this application adopts the following technical solution: A virtual desktop video acceleration method based on the SPICE protocol includes: The server identifies the areas that have actually changed in the virtual machine's output image in real time through periodic sampling and coordinate comparison; The identified areas of change are statistically analyzed for frequency. When the update frequency reaches a preset threshold, the area is determined to be a video area; otherwise, it is determined to be a non-video area. An adaptive coding strategy is implemented based on the region determination results. The H.264 coding algorithm is used to compress the video region, and the GLZ lossless compression algorithm is used to encode the non-video region. The encoded data is sent to the client, which selects the appropriate decoding method based on the encoding type: hardware decoding is enabled for H.264 data, and CPU decoding is used for GLZ data.

[0006] Optionally, the region identification mechanism specifically includes: Capture the coordinates of the image region of the current virtual machine screen at a fixed sampling period; Compare the region coordinates of the current sampling period with the historical coordinates recorded in the previous period; Pixel-level difference analysis accurately identifies areas of content that have changed, while filtering out static areas that have not changed.

[0007] Optionally, the video detection mechanism includes: Continuously track the update frequency of changing areas within a preset time window; A region is defined as a video region when its update frequency exceeds 15 frames per second. For regions with update frequencies below a threshold, text scrolling and real video content are distinguished by combining content feature analysis.

[0008] Optionally, the adaptive coding strategy includes: H.264 inter-frame predictive coding is enabled for video regions, and the GOP structure and quantization parameters are dynamically adjusted. The GLZ dictionary compression algorithm is used for non-video areas to preserve the original precision of text and graphics; Establish an independent encoding channel for high-frequency non-video areas to avoid edge blurring caused by H.264 lossy compression.

[0009] Optionally, the method further includes an end-to-end transmission optimization mechanism: The server adds a region metadata identifier before data transmission, specifying the encoding type and spatial coordinates; The client processes multi-region decoding tasks in parallel based on metadata identifiers; By dynamically adjusting the bitrate of video areas through bandwidth monitoring, priority is given to ensuring the transmission quality of interactive areas.

[0010] Optionally, the method supports optimized implementations for cross-platform deployment: Obtain screen image data for CPUs with different architectures through the standard interface of the virtualization layer; It employs a platform-independent coordinate comparison algorithm, compatible with virtual machine environments of x86 and ARM architectures; Provides a unified GLZ compression library interface for Windows and Linux systems.

[0011] Optionally, the method includes an error recovery mechanism: When network packet loss is detected, a retransmission request is initiated for the non-video region encoded by GLZ. A fast recovery strategy using reference frames is adopted for H.264 video regions, skipping non-critical frames to maintain smoothness; Establish a fallback process for decoding failures, automatically switching to software decoding to ensure basic display functionality.

[0012] Secondly, this application provides a virtual desktop video acceleration device based on the SPICE protocol, comprising: The recognition module is used by the server to identify, in real time, the areas that have actually changed in the virtual machine's output image through periodic sampling and coordinate comparison. The analysis module is used to perform frequency statistical analysis on the identified change areas. When the update frequency reaches a preset threshold, it is determined to be a video area; otherwise, it is determined to be a non-video area. The strategy implementation module is used to implement an adaptive coding strategy based on the region determination result. The video region is compressed using the H.264 coding algorithm, and the non-video region is encoded using the GLZ lossless compression algorithm. The output module is used to send the encoded data to the client. The client selects the corresponding decoding method according to the encoding type, enabling hardware decoding for H.264 data and CPU decoding for GLZ data.

[0013] Thirdly, this application provides a computer device, the device comprising: a memory and a processor, wherein the processor, when executing computer instructions stored in the memory, performs the method described above.

[0014] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described above.

[0015] In summary, this application effectively solves the problem of poor video transmission performance in the prior art by real-time identification of changing areas, frequency analysis to determine area type, adaptive encoding, and matching decoding. It can accurately identify video and non-video areas, implement adaptive encoding strategies, effectively reduce redundant data transmission, improve decoding efficiency, and thus improve the smoothness and user experience of virtual desktop videos. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application; Figure 2 This is a flowchart illustrating the first embodiment of the virtual desktop video acceleration method based on the SPICE protocol in this application; Figure 3 This is a structural block diagram of the first embodiment of the virtual desktop video acceleration device based on the SPICE protocol of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] Reference Figure 1 , Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application.

[0019] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0020] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0021] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a virtual desktop video acceleration program based on the SPICE protocol.

[0022] exist Figure 1In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this application can be set in the computer device, and the computer device calls the virtual desktop video acceleration program based on the SPICE protocol stored in the memory 1005 through the processor 1001, and executes the virtual desktop video acceleration method based on the SPICE protocol provided in the embodiment of this application.

[0023] This application provides a virtual desktop video acceleration method based on the SPICE protocol, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the virtual desktop video acceleration method based on the SPICE protocol of this application.

[0024] In this embodiment, the virtual desktop video acceleration method based on the SPICE protocol includes the following steps: The server identifies the areas that have actually changed in the virtual machine's output image in real time through periodic sampling and coordinate comparison; The identified areas of change are statistically analyzed for frequency. When the update frequency reaches a preset threshold, the area is determined to be a video area; otherwise, it is determined to be a non-video area. An adaptive coding strategy is implemented based on the region determination results. The H.264 coding algorithm is used to compress the video region, and the GLZ lossless compression algorithm is used to encode the non-video region. The encoded data is sent to the client, which selects the appropriate decoding method based on the encoding type: hardware decoding is enabled for H.264 data, and CPU decoding is used for GLZ data.

[0025] It should be noted that traditional methods for accelerating video based on the SPICE protocol have shortcomings in region recognition, video detection, and intelligent codec switching, affecting the smoothness of virtual machine video playback and user experience. To address this, this embodiment proposes a virtual desktop video acceleration method based on the SPICE protocol. This method identifies actually changing regions in the virtual machine's output image in real time through periodic sampling and coordinate comparison on the server side; it performs frequency statistical analysis on the identified changing regions, determining video and non-video regions based on the update frequency; and it implements an adaptive encoding strategy based on the region determination results, using H.264 encoding to compress video regions and GLZ lossless compression to encode non-video regions; finally, the encoded data is sent to the client, which selects the appropriate decoding method according to the encoding type: hardware decoding for H.264 data and CPU decoding for GLZ data.

[0026] It is understood that the following explanations are provided for some key terms in this embodiment: SPICE is a remote display protocol used in Virtual Desktop Infrastructure (VDI) environments. Its main function is to allow users to remotely access virtual machine desktops and provide services such as graphics, audio, and USB redirection. This protocol aims to optimize the remote user experience.

[0027] Periodic sampling refers to the server continuously capturing image data of the virtual machine screen at preset time intervals or event triggers. This method allows for the acquisition of snapshots of screen content changing over time, providing foundational data for subsequent identification of changed areas.

[0028] Coordinate comparison refers to comparing the coordinate information of the image region captured in the current sampling period with the historical coordinate information recorded in previous periods. This comparison process aims to identify the specific location and range of content changes on the screen.

[0029] Frequency statistical analysis refers to counting and analyzing the number of times an identified area of ​​change is updated within a certain time window. By statistically analyzing the update frequency of a region, the dynamism of its content can be assessed, thus helping to determine whether it is video content.

[0030] The preset threshold is a predetermined value used in frequency statistical analysis to distinguish between video regions and non-video regions. When the update frequency of a changing region reaches or exceeds the threshold, the region is determined to be a video region; otherwise, it is determined to be a non-video region.

[0031] Adaptive coding strategy refers to dynamically selecting and applying different coding algorithms to compress different regions of the virtual machine's output image based on the region determination results. This strategy aims to adopt appropriate coding methods for the characteristics of video and non-video content, balancing compression efficiency, image quality, and computational resource consumption.

[0032] The H.264 encoding algorithm is a video compression standard with high compression efficiency, widely used in various video streaming and storage scenarios. This algorithm achieves a high compression ratio by utilizing temporal and spatial redundancy between video frames.

[0033] GLZ lossless compression is a general-purpose lossless data compression algorithm suitable for non-video data such as text and graphics. This algorithm compresses data by finding and replacing repeating data patterns, and the original data can be completely recovered after decompression.

[0034] Hardware decoding refers to the use of a dedicated hardware decoder to decode encoded video data. Hardware decoding typically offers high efficiency and low CPU usage, making it suitable for processing high-bitrate video streams.

[0035] CPU decoding refers to using the central processing unit to execute software decoding algorithms to decode encoded data. CPU decoding is highly flexible and can support multiple encoding formats, but it may consume more CPU resources for videos with high resolution or frame rate.

[0036] This embodiment provides a virtual desktop video acceleration method based on the SPICE protocol, the specific implementation of which may include the following steps: First, the server is configured to identify areas in the virtual machine's output image that have actually changed in real time. This identification process can be achieved through periodic sampling and coordinate comparison. For example, the server can capture a complete image of the virtual machine screen every preset time interval, such as every second, and compare the currently captured image with the image from the previous moment pixel by pixel to identify all areas where pixel values ​​have changed. As another implementation, the server can capture only the areas on the screen triggered by mouse movement or keyboard input events, mark them as changed areas, and then compare the coordinates of these areas with historical records to determine whether they represent new changes.

[0037] Secondly, frequency statistical analysis is performed on the identified change areas, and the results are used to determine whether they are video or non-video areas. Specifically, the server can maintain an update counter for each identified change area and count the number of times the area is identified as a change area within a preset time period (e.g., the most recent second). When the update frequency of an area reaches or exceeds a preset value, such as ten updates per second, the area is determined to be a video area. Conversely, if the update frequency is lower than the preset value, the area is determined to be a non-video area.

[0038] Furthermore, an adaptive coding strategy is implemented based on the region determination results. For image data determined to be video regions, H.264 encoding can be used for compression. For example, the H.264 encoder can be configured to encode video regions at a preset bitrate or to compress using preset encoding quality parameters. For image data determined to be non-video regions, GLZ lossless compression can be used for encoding. For example, the GLZ encoder can scan the pixel data of non-video regions line by line and use dictionary matching or deduplication techniques for compression to ensure the integrity of image information.

[0039] Finally, the encoded data is sent to the client, which selects the appropriate decoding method based on the encoding type. When the client receives data marked as H.264 encoded, it can activate its hardware decoder for decoding. For example, the video decoding module in the client's graphics processing unit (GPU) is activated to process the H.264 video stream. When the client receives data marked as GLZ encoded, it can utilize its central processing unit (CPU) to execute a software decoding algorithm for decoding. For example, the GLZ decoding library integrated in the client's operating system is invoked, and the CPU calculates and recovers the original non-video image data.

[0040] Through the above technical solution, this embodiment effectively solves the problems of insufficient recognition efficiency, detection accuracy, and encoding switching flexibility in traditional virtual desktop video acceleration methods based on the SPICE protocol. This method achieves real-time recognition of changing regions in the virtual machine output image, ensuring recognition accuracy. Simultaneously, it distinguishes between video and non-video content based on their dynamic characteristics, and then performs targeted encoding and decoding, resulting in high efficiency. Therefore, it improves the smoothness of video playback in the virtual desktop environment, enhances the user experience, and makes reasonable use of the client's hardware resources.

[0041] In some of the above embodiments of this example, the server is proposed to identify the changing regions in the virtual machine output image by periodic sampling and coordinate comparison. However, in actual implementation, if only simple coordinate comparison is performed, it may not be able to accurately capture subtle changes in the screen content, or may misjudge static areas as changing areas, thereby affecting the accuracy of subsequent encoding strategies and the overall acceleration effect.

[0042] To address this, this embodiment further proposes a region identification mechanism, which specifically includes: capturing the image region coordinates of the current virtual machine screen at a fixed sampling period; comparing the region coordinates of the current sampling period with the historical coordinates recorded in the previous period; and accurately identifying the changed content regions through pixel-level difference analysis while filtering out unchanged static regions.

[0043] Specifically, the coordinates of the current virtual machine screen's image region are captured at a fixed sampling period. This fixed sampling period refers to a preset time interval, such as 30 frames per second or 60 frames per second. The server will periodically acquire the complete image data of the virtual machine screen or the coordinate information of its key areas. The capture operation can be implemented by calling the API interface provided by the hypervisor or by directly accessing the virtual machine's video memory, ensuring that the latest state of the screen can be obtained in a timely manner.

[0044] Subsequently, the region coordinates of the current sampling period are compared with the historical coordinates recorded in the previous period. After capturing the screen image data of the current period, the server compares it with the screen image data or its feature information recorded in the previous sampling period. This comparison can be a preliminary region-level comparison, such as comparing the checksum or hash value of a predefined grid region on the screen to quickly filter out macroscopic regions that may have changed.

[0045] Building upon this foundation, pixel-level difference analysis precisely identifies changed content regions while filtering out unchanged static regions. For regions deemed potentially changed, the system performs further refined pixel-level difference analysis. Specifically, it compares the RGB or brightness values ​​of the current image with those of historical images pixel by pixel. When the difference in pixel values ​​exceeds a preset threshold, the pixel is considered to have changed. By performing connected component analysis on these changed pixels, the system accurately delineates the boundaries of regions where content changes have actually occurred and marks them as "changed regions." Meanwhile, regions where no significant changes are detected after pixel-level difference analysis are classified as "unchanged static regions," thus avoiding unnecessary subsequent processing of these areas.

[0046] Through the aforementioned technical solution, the server can capture images of the virtual machine screen at fixed time intervals and compare them precisely with the screen state of the previous moment. In particular, the introduction of pixel-level difference analysis effectively avoids misjudgments that may be caused by simple coordinate comparison, accurately distinguishing between areas on the screen where content has actually changed and static areas that remain unchanged. This high-precision region identification mechanism ensures that subsequent adaptive encoding strategies can accurately apply to the areas that truly need updating, avoiding repeated encoding of static content. This significantly improves the overall efficiency and resource utilization of the virtual desktop video acceleration method, guaranteeing a smooth user experience and image quality.

[0047] In some of the above embodiments, although frequency statistical analysis is performed on the identified change areas, and a preliminary determination of whether a region is a video area or a non-video area is made based on whether the update frequency reaches a preset threshold, relying solely on a single frequency threshold may not accurately distinguish all types of dynamic content. For example, rapidly scrolling text or complex UI animations may exhibit a high update frequency. If these are misclassified as video areas and lossy compression is applied, it may lead to a decrease in image quality. Conversely, some low-frame-rate real video content may be misclassified as non-video areas because its frequency does not reach the threshold, thus failing to fully utilize the advantages of video encoding.

[0048] To address this, this embodiment further proposes a video detection mechanism, which includes continuously tracking the update frequency of changing areas within a preset time window; determining a region as a video region when its update frequency exceeds 15 frames per second; and distinguishing between text scrolling and real video content by combining content feature analysis for regions with update frequencies below a threshold.

[0049] To more accurately reflect the dynamic characteristics of changing regions, this embodiment employs a method of continuously tracking their update frequency within a preset time window. Specifically, the server can maintain an update event queue or circular buffer for each identified changing region. When an update occurs in this region, the current timestamp is recorded in the queue. When calculating the update frequency, the number of events in the queue within the preset time window (e.g., the most recent 1 second, 2 seconds, or longer) is counted to obtain the average update frequency of the region over a period of time. This continuous tracking mechanism effectively avoids misjudgments caused by instantaneous frequency fluctuations, providing more stable frequency data. After continuously tracking and obtaining the update frequency of changing regions, this embodiment sets a clear judgment criterion: when the update frequency of a region exceeds 15 frames per second, it is judged as a video region. 15 frames per second is a universally accepted lower limit for video smoothness in the industry; content with a frequency higher than this is generally considered to be a video stream with continuous dynamic characteristics. By setting this quantization threshold, highly dynamic, continuously changing regions can be effectively identified as video content for subsequent processing using efficient video encoding algorithms such as H.264. For regions with update frequencies lower than the above threshold, frequency alone is insufficient to accurately determine their nature. Therefore, this embodiment further combines content feature analysis to distinguish text scrolling from real video content. Specifically, pixel data in these areas can be analyzed. For example, text scrolling is often characterized by an overall translation of image content in a specific direction (such as vertical or horizontal), with a high degree of similarity in the structure and color distribution of pixel blocks before and after the translation. The presence of large areas of text content moving regularly can be determined by calculating motion vectors of inter-frame differences, detecting the continuity of image edges, or using optical character recognition (OCR) technology. For low-frame-rate real video content, the changes may be more complex and irregular, such as containing rich color and texture variations, or non-linear motion patterns. These irregular changes can be identified by analyzing image entropy, color histogram changes, texture features, or more complex motion estimation algorithms, thereby distinguishing them from text scrolling.

[0050] Through the above technical solution, this embodiment introduces a more refined video detection mechanism on top of basic regional frequency statistical analysis. By continuously tracking the update frequency of changing regions within a preset time window and setting a clear threshold of 15 frames per second, it can more stably and accurately identify high-dynamic video content. More importantly, for regions with update frequencies below the threshold, they are no longer simply classified as non-video regions, but are further analyzed in conjunction with content features to accurately distinguish between text scrolling and real low-frame-rate video content. This multi-dimensional, hierarchical judgment strategy effectively solves the problem of misjudgment that may be caused by relying solely on a single frequency threshold. It ensures that real video content (including low-frame-rate video) can be correctly identified and efficiently compressed using H.264 encoding, thereby ensuring video smoothness and quality while avoiding unnecessary lossy compression of non-video content (such as text scrolling), thus improving overall encoding efficiency and image display quality. Especially when dealing with complex virtual desktop scenarios containing various dynamic content, it can significantly improve the user experience.

[0051] In some of the embodiments described above, although an adaptive encoding strategy is implemented based on the region determination result, using H.264 encoding to compress video regions and GLZ lossless compression to encode non-video regions, in practical applications, simple H.264 or GLZ encoding may not achieve optimal compression efficiency and display quality for different types of video and non-video content. For example, H.264 encoding may require more precise parameter adjustments to balance quality and bandwidth when dealing with complex video content, while for frequently updated non-video regions, indiscriminate processing may lead to loss of image details or wasted bandwidth.

[0052] To address this, this embodiment further proposes an optimized adaptive coding strategy, which includes: enabling H.264 inter-frame predictive coding for video regions and dynamically adjusting the GOP structure and quantization parameters; using the GLZ dictionary compression algorithm for non-video regions to preserve the original precision of text and graphics; and establishing an independent coding channel for high-frequency non-video regions to avoid edge blurring caused by H.264 lossy compression.

[0053] Specifically, when encoding video regions, the system employs H.264 inter-frame predictive coding. Inter-frame predictive coding analyzes the temporal redundancy between video frames, utilizing motion information and pixel differences between adjacent frames to predict the content of the current frame, encoding only the prediction residual, thus significantly reducing the data volume. Based on this, the encoder dynamically adjusts the GOP (Group of Pictures) structure according to the motion complexity of the video content and the frequency of scene transitions. For example, it shortens the GOP length to improve response speed when scene transitions are frequent, and lengthens the GOP length to improve compression efficiency when motion is smooth. Simultaneously, the quantization parameter (QP) is also dynamically adjusted based on the importance of the video region, bandwidth limitations, or user preferences to optimize the compression ratio while maintaining visual quality. For example, for high-motion, high-detail video regions, the QP can be appropriately reduced to improve quality; for background or low-motion regions, the QP can be increased to save bandwidth.

[0054] For non-video areas, this embodiment employs the GLZ dictionary compression algorithm for encoding to preserve the original precision of text and graphics. GLZ (Generalized Lempel-Ziv) is a lossless compression algorithm that works by building and using a dictionary to replace recurring byte sequences. When encoding non-video areas, the algorithm scans pixel data, identifies repeating patterns, color blocks, or text character sequences, and stores them in a dictionary. When the same sequence is encountered subsequently, only its index in the dictionary is stored, rather than the original data. This is highly effective for non-video areas containing numerous repeating patterns, solid color blocks, or clear text (such as desktop backgrounds, window borders, menus, and text editor content), achieving high compression ratios while ensuring pixel-level lossless restoration.

[0055] Furthermore, to address potential issues with frequently updated non-video regions, this embodiment establishes an independent encoding channel for these regions to avoid edge blurring caused by H.264 lossy compression. Even if a non-video region updates frequently (potentially approaching the threshold for video regions), the system will explicitly mark it as a "high-frequency non-video region" and force it to be encoded using the GLZ dictionary compression algorithm through an independent channel, rather than including it in the H.264 encoding process. H.264, as a lossy compression algorithm, discards some high-frequency details during its quantization process, potentially causing blurring or jagged edges and fine lines in the compressed text. By providing an independent channel for high-frequency non-video regions and forcing the use of GLZ lossless compression, it ensures that these regions maintain their original precision, effectively avoiding quality degradation caused by lossy compression.

[0056] Through the above technical solutions, this embodiment can significantly improve the encoding efficiency and display quality of virtual desktop video acceleration. Specifically, H.264 inter-frame predictive coding is enabled for video areas, and the GOP structure and quantization parameters are dynamically adjusted. This allows H.264 encoding to be finely adapted according to the complexity and motion characteristics of the video content, effectively utilizing temporal redundancy. While ensuring visual smoothness, it optimizes the compression ratio and bandwidth usage. Simultaneously, the GLZ dictionary compression algorithm is used for non-video areas, ensuring the original accuracy of text and graphics and avoiding edge blurring and detail loss that may result from lossy compression. This is particularly suitable for desktop application scenarios with high clarity requirements. Furthermore, an independent encoding channel is established for high-frequency non-video areas, effectively preventing these areas from being misjudged as video and subjected to H.264 lossy compression due to frequent updates. This completely solves the edge blurring problem that may occur during the transmission of high-frequency non-video content such as text scrolling or cursor movement, ensuring the smoothness and accuracy of the user interaction experience. These optimization measures work together to enable the virtual desktop to achieve efficient and high-quality transmission and display under different content types.

[0057] In some of the above implementations, although it is proposed to perform region identification, frequency statistical analysis and adaptive encoding strategy according to region type to distinguish video regions and non-video regions, and to use different compression algorithms for encoding, problems such as low transmission efficiency, unbalanced client decoding load and user experience degradation caused by network bandwidth fluctuations may be encountered during the transmission of encoded data to the client and the client's decoding process. Especially in mixed content scenarios, how to ensure the transmission quality of different types of data and the smooth display on the client still needs further optimization.

[0058] To address this, this embodiment further proposes an end-to-end transmission optimization mechanism, which includes: the server adding a region metadata identifier before data transmission to indicate the encoding type and spatial coordinates; the client processing multiple region decoding tasks in parallel based on the metadata identifier; and dynamically adjusting the bitrate of video regions through bandwidth monitoring to prioritize the transmission quality of interactive regions.

[0059] Specifically, before sending the encoded data to the client, the server appends a region metadata identifier to each data block or data stream segment. This identifier, as supplementary information, explicitly indicates the encoding type used by the current data block (e.g., H.264 or GLZ) and its spatial location on the original virtual machine screen (e.g., the coordinates and dimensions of the top-left corner of a rectangular region). This metadata can be transmitted along with the data in the form of packet headers, stand-alone control messages, or embedded fields, ensuring that the client can quickly parse and obtain key context information upon receipt.

[0060] Upon receiving data with region metadata identifiers, the client can intelligently allocate its decoding resources based on this information. For example, for video region data identified as H.264 encoded, the client can distribute it to a dedicated hardware decoder (such as the video decoding unit of a GPU) for processing, leveraging hardware acceleration capabilities. For non-video region data identified as GLZ encoded, the client can allocate it to the CPU for software decoding. Simultaneously, the client can pre-plan the write locations of the decoded pixel data based on the spatial coordinates provided in the metadata, thereby enabling parallel decoding and rendering of different regions and avoiding performance bottlenecks that might result from a single decoding path.

[0061] Furthermore, to address dynamic changes in the network environment, this embodiment also employs a bandwidth monitoring mechanism to dynamically adjust the bitrate of video regions. The server or client (or both working together) continuously monitors the current network bandwidth status, for example, by assessing data transmission rate, packet loss rate, or latency. When a decrease in network bandwidth is detected, the server correspondingly reduces the bitrate of H.264 video regions, for example, by adjusting quantization parameters, reducing frame rate, or optimizing the GOP (Group of Pictures) structure, to reduce the amount of data transmitted and adapt to limited bandwidth. Simultaneously, for regions containing user-interactive content (such as mouse pointers, text input boxes, etc.), even if they are determined to be non-video regions and use GLZ lossless encoding, their transmission quality is prioritized to ensure the real-time response and clear display of these critical interactive elements, thereby maintaining a good user experience.

[0062] Through the above technical solution, the server adds region metadata identifiers before data transmission, enabling the client to clearly identify the encoding type of each data block and its position on the screen. This allows for intelligent scheduling of decoding resources; for example, H.264 video data can be processed by a hardware decoder, while GLZ non-video data can be processed by the CPU, achieving parallel decoding of multiple regions. This significantly improves the client's decoding efficiency and overall rendering performance, avoiding bottlenecks from a single decoding path. Furthermore, by monitoring network bandwidth in real time and dynamically adjusting the bitrate of video regions, this embodiment can flexibly balance transmission efficiency and image quality according to actual network conditions. Especially when bandwidth is limited, priority is given to ensuring the transmission quality of interactive areas, ensuring real-time response to user operations and clear display of key information. Therefore, in complex network environments, this provides users with a smoother and more stable virtual desktop experience.

[0063] In some of the embodiments described above, a virtual desktop video acceleration method based on the SPICE protocol is proposed. This method identifies changing regions through periodic sampling and coordinate comparison, determines whether a region is video or non-video based on its update frequency, and then implements an adaptive encoding strategy. However, in actual deployment and application, due to the diversity of virtualization environments, including different CPU architectures and operating system platforms, this method may face compatibility challenges and struggle to achieve efficient and unified video acceleration services in heterogeneous environments.

[0064] In response, this embodiment further proposes an optimized implementation of the method to support cross-platform deployment, specifically including: obtaining screen image data of CPUs with different architectures through the virtualization layer standard interface; adopting a platform-independent coordinate comparison algorithm to be compatible with virtual machine environments of x86 and ARM architectures; and providing a unified GLZ compression library interface for Windows and Linux systems.

[0065] Obtaining screen image data from CPUs of different architectures through standard virtualization layer interfaces refers to using standardized application programming interfaces (APIs) or protocols provided by the hypervisor or virtualization platform to access and capture screen display buffer data within the virtual machine. These interfaces aim to abstract away the differences in underlying physical hardware, allowing upper-layer applications to obtain screen image data in a unified format without needing to know whether the host CPU is x86 or ARM. For example, the display channel interface defined in the SPICE protocol or specific APIs provided by virtualization solutions such as KVM and VMware can be used to obtain the virtual machine's framebuffer content. This approach ensures platform independence in the screen data acquisition process, providing a unified data source for subsequent processing.

[0066] Employing a platform-independent coordinate comparison algorithm, compatible with both x86 and ARM architecture virtual machine environments, means that the pixel-level difference analysis and coordinate calculation algorithms used to identify actually changed regions in the virtual machine output image do not depend on the instruction set, memory layout, or data representation of a specific CPU architecture. This implies that the algorithm design should avoid using x86 or ARM-specific assembly instructions or optimizations, and instead be implemented using standard high-level languages ​​such as C / C++, ensuring consistent behavior of data types and operations across different platforms. For example, when comparing pixel values ​​and calculating the bounding box of changed regions, it should be ensured that the algorithm can correctly and efficiently identify changed pixels and their corresponding screen coordinates regardless of whether it is an x86 or ARM architecture virtual machine environment, thereby guaranteeing the accuracy and compatibility of region identification.

[0067] Providing a unified GLZ compression library interface for Windows and Linux systems refers to offering a standardized programming interface that can be called on both Windows and Linux operating systems for the GLZ lossless compression algorithm used in non-video areas. This interface encapsulates the implementation details of the GLZ compression algorithm and handles differences in file I / O, memory management, and thread scheduling between different operating systems. Through this unified interface, regardless of whether the server or client runs on Windows or Linux, the same function calls can be used to perform GLZ compression or decompression operations, thus simplifying the complexity of cross-platform development and ensuring the stability and consistency of GLZ compression across different operating system environments.

[0068] Through the above technical solution, this embodiment effectively solves the problem of insufficient deployment and compatibility of virtual desktop video acceleration methods in heterogeneous virtualization environments. By utilizing the standard interface of the virtualization layer, the system can shield the differences in the underlying CPU architecture and uniformly acquire screen image data, thereby ensuring platform independence of the data source. Simultaneously, the adoption of a platform-independent coordinate comparison algorithm enables the core change region identification logic to run stably in virtual machine environments with different architectures such as x86 and ARM, avoiding algorithm failure or performance degradation due to platform differences. Furthermore, a unified GLZ compression library interface is provided for Windows and Linux systems, simplifying the encoding implementation of non-video regions and ensuring the consistency of compression efficiency and data accuracy across different operating systems. Therefore, the virtual desktop video acceleration method of this embodiment has stronger cross-platform compatibility and deployment flexibility, providing users with a consistent and efficient virtual desktop experience regardless of the underlying hardware and operating system environment.

[0069] In the aforementioned virtual desktop video acceleration methods, the adoption of adaptive encoding strategies for different regions can effectively improve video stream transmission efficiency and image quality. However, in real-world network environments, data transmission often faces issues such as network instability and packet loss. This can lead to incomplete or corrupted data received by the client, thereby affecting the correctness of decoding and the smoothness of the displayed image. Especially in highly interactive or real-time-critical virtual desktop scenarios, such interruptions or stuttering can severely impact the user experience.

[0070] To address this, this embodiment further proposes that the above method also includes an error recovery mechanism. This error recovery mechanism aims to initiate retransmission requests for GLZ-encoded non-video regions when network packet loss is detected; to employ a reference frame fast recovery strategy for H.264 video regions, skipping non-critical frames to maintain smoothness; and to establish a degradation processing flow for decoding failures, automatically switching to software decoding to ensure basic display functions.

[0071] Specifically, an error recovery mechanism refers to a strategy whereby, when anomalies occur during data transmission or processing (such as network packet loss, data corruption, decoding failure, etc.), the system can take pre-defined countermeasures to minimize the impact of the anomalies on system functionality and user experience, and restore the system to a normal working state as much as possible. Its purpose is to improve the robustness and reliability of the system, ensuring that acceptable service quality can still be provided under adverse conditions. When network packet loss is detected, a retransmission request is initiated for the GLZ-encoded non-video region. Network packet loss detection can be implemented in various ways, such as using sequence numbers and acknowledgment mechanisms in transport layer protocols (such as TCP), or using custom heartbeat packets and packet sequence number verification at the application layer. Once packet loss is detected, especially for non-video region data encoded using the GLZ lossless compression algorithm, due to its high requirements for data integrity (the loss of any single byte may lead to decoding failure or display errors for the entire region), the server or client will initiate a retransmission request to the other party. After receiving the request, the server will resend the lost data packets to ensure that the client can receive the complete non-video region data, thereby guaranteeing the original accuracy of text and graphics.

[0072] Meanwhile, a reference frame fast recovery strategy is employed for H.264 video regions, skipping non-key frames to maintain smoothness. H.264 encoding typically includes I-frames (key frames), P-frames (predictive frames), and B-frames (bidirectional predictive frames). I-frames are encoded independently, without relying on other frames; P-frames and B-frames, however, rely on other frames for predictive encoding. When packet loss occurs in the video stream, if P-frames or B-frames are lost, subsequent frames that depend on these frames will not be decoded correctly. The reference frame fast recovery strategy means that when video data packet loss is detected, the decoder actively skips the damaged P-frames and B-frames, waiting for the next available I-frame or an undamaged P-frame as a new reference frame, thereby quickly restoring video playback. This strategy sacrifices the integrity of some frames (potentially resulting in brief pixelation or pauses), but prioritizes the continuity and smoothness of the video stream, avoiding prolonged stuttering.

[0073] Furthermore, a fallback process is established for decoding failures, automatically switching to software decoding to ensure basic display functionality. Decoding failures can be caused by various reasons, such as hardware decoder malfunctions, driver issues, or severely corrupted data that cannot be recovered. The fallback process means that when the hardware decoder fails, the system can automatically detect this anomaly and quickly switch the decoding task to the CPU for software decoding. Although software decoding is generally less efficient and may lead to a drop in frame rate or an increase in CPU usage, it serves as a backup plan to ensure that the client can still display basic desktop content in the event of hardware decoding failure, avoiding a complete black screen or program crash, thus ensuring that the user can continue to perform basic operations.

[0074] Through the above technical solutions, the virtual desktop video acceleration method of this embodiment can significantly improve the robustness of the system and the user experience when facing unstable network environments. Specifically, for non-video areas encoded with GLZ, precise packet loss detection and retransmission requests ensure the integrity and display accuracy of key information such as text and graphics, avoiding display errors caused by data loss. Simultaneously, for H.264 video areas, a reference frame fast recovery strategy is adopted. By skipping non-critical frames, the video stream can quickly resume playback when packet loss occurs, effectively maintaining the continuity and smoothness of the image and avoiding prolonged stuttering, thus ensuring the user's viewing experience. Furthermore, when the decoder malfunctions or data is severely corrupted, causing decoding failure, an automatic switch to software decoding degradation processing flow is implemented, providing users with a reliable backup plan. Even in the most unfavorable circumstances, basic desktop display functions are guaranteed, greatly enhancing the availability and stability of the virtual desktop service. These measures work together to enable the virtual desktop to provide high-quality, highly reliable services even under complex network conditions.

[0075] In a virtual desktop environment, user A is watching an online video while simultaneously editing a text file. The SPICE protocol server continuously monitors the output of the virtual machine screen.

[0076] First, the server uses a periodic sampling and coordinate comparison mechanism to identify areas on the screen that have actually changed in real time. For example, the server captures the coordinates of the image area of ​​the current virtual machine screen at a fixed sampling period of 30 frames per second. It compares the coordinates of the area in the current sampling period with the historical coordinates recorded in the previous period, and accurately identifies changed content areas such as the video playback window, the cursor blinking in the text editing area, and text input through pixel-level difference analysis, while filtering out static areas that have not changed, such as the desktop background and toolbar.

[0077] Next, the server performs frequency statistical analysis on the identified areas of change. Within a preset time window, the server continuously tracks the update frequency of each area of ​​change. For example, if the update frequency of the video playback window consistently exceeds 15 frames per second, the server classifies it as a video area. Conversely, cursor blinking and text input in the text editing area, with update frequencies far below 15 frames per second, are classified as non-video areas. For areas with lower update frequencies but potentially containing dynamic content, the server combines content feature analysis to distinguish between text scrolling and actual video content.

[0078] Based on the region determination results, the server implements an adaptive encoding strategy. For video playback windows identified as video regions, the server uses the H.264 encoding algorithm for compression. During this process, the server enables H.264 inter-frame predictive coding and dynamically adjusts the GOP structure and quantization parameters according to the dynamic nature of the video content to optimize the bitrate while ensuring video smoothness. For text editing windows identified as non-video regions, the server uses the GLZ lossless compression algorithm for encoding, preserving the original precision of text and graphics to ensure clear and readable text. In addition, to avoid edge blurring caused by H.264 lossy compression, the server establishes a separate encoding channel for processing some frequently changing non-video regions (such as rapidly scrolling text).

[0079] After encoding is complete, the server adds a region metadata identifier before data transmission, specifying the type of each encoded data (H.264 or GLZ) and its spatial coordinates on the screen, and then sends the encoded data to the client.

[0080] After receiving the data, the client processes multi-region decoding tasks in parallel based on the metadata identifier. For data with the H.264 identifier, the client enables a hardware decoder for decoding, utilizing the GPU's video processing unit to efficiently restore the video content and reduce the CPU load. For data with the GLZ identifier, the client uses the CPU for decoding, ensuring lossless restoration of text and graphics. During data transmission, the server also dynamically adjusts the bitrate of video regions through bandwidth monitoring, prioritizing the transmission quality of user-interactive areas (such as mouse movement and text input).

[0081] This method also supports cross-platform deployment. The server obtains screen image data from CPUs of different architectures (such as x86 and ARM) through the virtualization layer standard interface and uses a platform-independent coordinate comparison algorithm. The client provides a unified GLZ compression library interface for Windows and Linux systems, ensuring compatibility across different operating systems.

[0082] Furthermore, this method includes an error recovery mechanism. When network packet loss is detected, the server initiates a retransmission request for GLZ-encoded non-video regions to ensure data integrity. For H.264 video regions, a reference frame fast recovery strategy is used, skipping non-critical frames to maintain smooth video playback. If the client detects decoding failure, the system automatically switches to software decoding to ensure basic display functions and avoid screen interruption.

[0083] Compared to existing technologies, this method achieves accurate identification of changing areas on the virtual machine screen and filters out static areas through a fixed sampling period and pixel-level difference analysis in its region identification mechanism. In terms of video detection, this method not only determines video regions based on an update frequency threshold (e.g., 15 frames per second) but also combines content feature analysis to distinguish between scrolling text and real video, improving the accuracy of video region identification. In its adaptive coding strategy, this method employs H.264 inter-frame predictive coding for video regions and dynamically adjusts the GOP structure and quantization parameters. For non-video regions, it uses GLZ dictionary compression to preserve original accuracy and establishes independent coding channels for high-frequency non-video regions, avoiding the impact of lossy compression on text and graphics quality. Furthermore, through region metadata identification and parallel decoding in the end-to-end transmission optimization mechanism, as well as bandwidth monitoring and dynamic bitrate adjustment, this method improves transmission efficiency and user experience, and adds error recovery and cross-platform deployment capabilities—all aspects that have not been elaborated or fully optimized in existing technologies.

[0084] Furthermore, embodiments of this application also propose a computer-readable storage medium storing a program for accelerating virtual desktop video based on the SPICE protocol. When the program for accelerating virtual desktop video based on the SPICE protocol is executed by a processor, it implements the steps of the method for accelerating virtual desktop video based on the SPICE protocol as described above.

[0085] Reference Figure 3 , Figure 3 This is a structural block diagram of the first embodiment of the virtual desktop video acceleration device based on the SPICE protocol of this application.

[0086] like Figure 3 As shown, the virtual desktop video acceleration device based on the SPICE protocol proposed in this application includes: The recognition module 10 is used by the server to identify the areas that have actually changed in the virtual machine output image in real time through periodic sampling and coordinate comparison. Analysis module 20 is used to perform frequency statistical analysis on the identified change areas. When the update frequency reaches a preset threshold, it is determined to be a video area; otherwise, it is determined to be a non-video area. The strategy implementation module 30 is used to implement an adaptive coding strategy based on the region determination result, using the H.264 coding algorithm to compress video regions and the GLZ lossless compression algorithm to encode non-video regions. The output module 40 is used to send the encoded data to the client. The client selects the corresponding decoding method according to the encoding type, enabling hardware decoding for H.264 data and CPU decoding for GLZ data.

[0087] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of this application. In specific applications, those skilled in the art can make settings as needed, and this application does not impose any restrictions on this.

[0088] This embodiment effectively solves the problem of poor video transmission performance in the prior art by real-time identification of changing areas, frequency analysis to determine area type, adaptive encoding, and matching decoding. It can accurately identify video and non-video areas, implement adaptive encoding strategies, effectively reduce redundant data transmission, improve decoding efficiency, and thus improve the smoothness of virtual desktop video and user experience.

[0089] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0090] In addition, for technical details not described in detail in this embodiment, please refer to the method for virtual desktop video acceleration based on the SPICE protocol provided in any embodiment of this application, which will not be repeated here.

[0091] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0092] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application. The above are only preferred embodiments of this application and do not limit the patent scope of this application. All equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A virtual desktop video acceleration method based on the SPICE protocol, characterized in that, include: The server identifies the areas that have actually changed in the virtual machine's output image in real time through periodic sampling and coordinate comparison; The identified areas of change are statistically analyzed for frequency. When the update frequency reaches a preset threshold, the area is determined to be a video area; otherwise, it is determined to be a non-video area. An adaptive coding strategy is implemented based on the region determination results. The H.264 coding algorithm is used to compress the video region, and the GLZ lossless compression algorithm is used to encode the non-video region. The encoded data is sent to the client, which selects the appropriate decoding method based on the encoding type: hardware decoding is enabled for H.264 data, and CPU decoding is used for GLZ data.

2. The method according to claim 1, characterized in that, The specific area identification mechanism includes: Capture the coordinates of the image region of the current virtual machine screen at a fixed sampling period; Compare the region coordinates of the current sampling period with the historical coordinates recorded in the previous period; Pixel-level difference analysis accurately identifies areas of content that have changed, while filtering out static areas that have not changed.

3. The method according to claim 1, characterized in that, Video detection mechanisms include: Continuously track the update frequency of changing areas within a preset time window; A region is defined as a video region when its update frequency exceeds 15 frames per second. For regions with update frequencies below a threshold, text scrolling and real video content are distinguished by combining content feature analysis.

4. The method according to claim 1, characterized in that, Adaptive coding strategies include: H.264 inter-frame predictive coding is enabled for video regions, and the GOP structure and quantization parameters are dynamically adjusted. The GLZ dictionary compression algorithm is used for non-video areas to preserve the original precision of text and graphics; Establish an independent encoding channel for high-frequency non-video areas to avoid edge blurring caused by H.264 lossy compression.

5. The method according to claim 1, characterized in that, The method also includes an end-to-end transmission optimization mechanism: The server adds a region metadata identifier before data transmission, specifying the encoding type and spatial coordinates; The client processes multi-region decoding tasks in parallel based on metadata identifiers; By dynamically adjusting the bitrate of video areas through bandwidth monitoring, priority is given to ensuring the transmission quality of interactive areas.

6. The method according to claim 1, characterized in that, The method supports optimized implementations for cross-platform deployment: Obtain screen image data for CPUs with different architectures through the standard interface of the virtualization layer; It employs a platform-independent coordinate comparison algorithm, compatible with virtual machine environments of x86 and ARM architectures; Provides a unified GLZ compression library interface for Windows and Linux systems.

7. The method according to claim 1, characterized in that, The method includes an error recovery mechanism: When network packet loss is detected, a retransmission request is initiated for the non-video region encoded by GLZ. A fast recovery strategy using reference frames is adopted for H.264 video regions, skipping non-critical frames to maintain smoothness; Establish a fallback process for decoding failures, automatically switching to software decoding to ensure basic display functionality.

8. A virtual desktop video acceleration device based on the SPICE protocol, characterized in that, include: The recognition module is used by the server to identify, in real time, the areas that have actually changed in the virtual machine's output image through periodic sampling and coordinate comparison. The analysis module is used to perform frequency statistical analysis on the identified change areas. When the update frequency reaches a preset threshold, it is determined to be a video area; otherwise, it is determined to be a non-video area. The strategy implementation module is used to implement an adaptive coding strategy based on the region determination result. The video region is compressed using the H.264 coding algorithm, and the non-video region is encoded using the GLZ lossless compression algorithm. The output module is used to send the encoded data to the client. The client selects the corresponding decoding method according to the encoding type, enabling hardware decoding for H.264 data and CPU decoding for GLZ data.

9. A computer device, characterized in that, The device includes a memory and a processor, wherein the processor, when executing computer instructions stored in the memory, performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images