Remote desktop transmission system and method

Abstract

The invention relates to the technical field of transmission, in particular to a remote desktop transmission system and method. The system comprises a sending end and a receiving end, and the sending end collects a current screen image according to a predetermined frame rate, obtains a frame image data stream, carries out frame difference processing on the frame image data stream, and determines a difference mark sequence set; the sending end carries out coding processing on the difference mark sequence set to obtain difference compression data; the differential compressed data is packaged into a frame data packet, and the frame data packet is sent to a receiving end through a universal serial bus USB2.0 interface; the receiving end decodes the frame data of the current frame in the frame data, determines a complete image of the current frame, and displays the complete image on the receiving end; and the receiving end updates the cached reference image according to the complete image of the current frame, continuously decodes the frame data of the next frame of the current frame in the frame data in combination with the reference image, determines the complete image of the next frame, and displays the complete image at the receiving end until the frame data of all frames in the frame data are decoded.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image transmission, in particular to a remote desktop transmission system and method. BACKGROUND

[0002] At present, remote desktop control technology is widely used in remote office, remote technical support, remote communication, remote maintenance management and other fields. In these applications, real-time sharing of remote desktop has high requirements for resolution, frame rate and delay.

[0003] With the development of display technology, 4K and above ultra-high definition resolution desktops are gradually popularized, but the data volume increases sharply. For example, a common 4K frame image of 3840x2160 resolution contains about 8.3 million pixels, and the data volume of an uncompressed 4K image is extremely large. If the 4K desktop is transmitted at a frequency of 15 frames per second, the uncompressed data bandwidth requirement can reach several Gbps, and the traditional USB 2.0 interface cannot meet the requirement. Therefore, when transmitting images, the existing remote desktop transmission usually encodes the images before transmission to reduce the transmission bandwidth requirement.

[0004] In related technologies, simple image encoding is used for direct transmission, which has limited compression rate at high resolution such as 4K, and is prone to network congestion and increased transmission delay. In view of this, some optimization schemes are provided in related technologies, such as using mainstream video encoding technology to encode the remote desktop as a video stream, or using a special encoding standard for screen content characteristics.

[0005] However, although the aforementioned video encoding standard uses intra / inter-frame prediction, entropy encoding and other advanced technologies to provide high-quality images at a lower code rate, it performs well in bandwidth-limited environments. However, the aforementioned encoding scheme either requires a high-performance processor to support complex encoding and decoding, or requires a special hardware / chip to support, which has certain limitations in universality or cost. SUMMARY

[0006] The embodiments of the present application provide a remote desktop transmission system and method, which can reduce the complexity of image encoding and decoding to a certain extent, and does not require hardware / chip support, thereby improving the universality of the remote desktop transmission scheme and reducing the cost of remote desktop transmission.

[0007] In one aspect, the embodiments of the present application provide a remote desktop transmission system, which comprises a sending end and a receiving end, wherein,

[0008] The sending end acquires a current screen image according to a predetermined frame rate, obtains a frame image data stream, and performs frame difference processing on the frame image data stream to determine a difference marker sequence set;

[0009] The sending end encodes the difference mark sequence set to obtain difference compression data, and encapsulates the difference compression data into frame data packets and sends the frame data packets to the receiving end through a universal serial bus (USB) 2.0 interface;

[0010] The receiving end decodes the frame data of the current frame to determine a complete image of the current frame, and displays the complete image of the current frame on the receiving end.

[0011] The receiving end updates the cached reference image according to the complete image of the current frame, and continues to decode the frame data of a next frame of the current frame in combination with the reference image to determine a complete image of the next frame, and displays the complete image of the next frame on the receiving end, until the frame data of all frames in the frame data stream are decoded.

[0012] In a possible implementation, the sending end performs frame difference processing on the frame image data stream to determine the difference mark sequence set, including:

[0013] The sending end determines the difference mark sequence of the frame image of each frame except the first frame in the frame image data stream.

[0014] The sending end determines the difference mark sequence set according to the obtained difference mark sequence.

[0015] In a possible implementation, the sending end determines the difference mark sequence of the frame image of each frame except the first frame in the frame image data stream, including:

[0016] The sending end performs the following operations on the frame image of each frame except the first frame in the frame image data stream:

[0017] The sending end determines a current frame image and a reference frame image of a reference frame of the current frame image; the current frame image is the frame image of any frame except the first frame.

[0018] The sending end traverses the current frame image and the reference frame image pixel by pixel to determine the difference mark sequence.

[0019] In a possible implementation, the sending end traverses the current frame image and the reference frame image pixel by pixel to determine the difference mark sequence, including:

[0020] The sending end compares the current frame image and the reference frame image pixel by pixel in a left-to-right and top-to-bottom order to determine an unchanged pixel segment length corresponding to the current frame image and / or changed pixel data blocks.

[0021] The transmitting end determines the difference marker sequence of the current frame image based on the length of the unchanged pixel segment and the changed pixel data block corresponding to the current frame image.

[0022] In one possible implementation, the transmitting end encodes the differential marker sequence set to obtain differentially compressed data, including:

[0023] The sending end performs the following operations on each differentially labeled sequence in the differentially labeled sequence set:

[0024] First compressed data to determine the length of the unchanged pixel segment in the difference marker sequence; and / or,

[0025] Determine the second compressed data for the changed pixel data block in each difference marker sequence;

[0026] The sending end determines the differential compressed data packet based on the obtained first compressed data and / or second compressed data.

[0027] In one possible implementation, the transmitting end determines first compressed data of the length of the unchanged pixel segment in the difference marker sequence, including:

[0028] The transmitting end determines the first flag bit of the length of the unchanged pixel segment in the difference marker sequence;

[0029] The transmitting end determines the length information of the unchanged pixel segment in the difference marker sequence;

[0030] The transmitting end compresses the first flag bit and length information of the unchanged pixel segment in the difference marker sequence to determine the first compressed data of the unchanged pixel segment length in the difference marker sequence.

[0031] In one possible implementation, the transmitting end determines second compressed data for the changed pixel data block in each difference marker sequence, including:

[0032] The transmitting end determines the second flag bit of the changed pixel data block in the difference marker sequence;

[0033] The transmitting end determines the number of pixels in the changed pixel data block in the difference marker sequence and the color value of each pixel corresponding to the number of pixels;

[0034] The transmitting end compresses the second flag bit, the number of pixels, and the color value of each pixel corresponding to the number of pixels in each difference marker sequence of the changed pixel data block to determine the second compressed data of the changed pixel data block in the difference marker sequence.

[0035] In one possible implementation, the receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, including:

[0036] The receiving end reads the flag bit of the frame data of the current frame;

[0037] If the receiving end determines that the flag bit points to an unchanged pixel segment, it copies the number of pixels corresponding to the first length from the previously stored image to a specified position in the current frame buffer according to the first length carried by the frame data of the current frame, thereby determining the complete image of the current frame.

[0038] In one possible implementation, the receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, including:

[0039] If the receiving end determines that the flag bit points to the changing pixel segment, it reads the second length of the changing pixel segment and continuously reads the pixel values ​​of the second length, and fills the pixel values ​​of the second length into the specified position of the current frame buffer in sequence to determine the complete image of the current frame.

[0040] On the other hand, embodiments of the present invention provide a remote desktop transmission method applied to a remote desktop transmission system, the system including a sending end and a receiving end, the method including:

[0041] The sending end acquires the current screen image at a predetermined frame rate, obtains the frame image data stream, and performs frame difference processing on the frame image data stream to determine the difference marker sequence set;

[0042] The transmitting end encodes the differential marker sequence set to obtain differential compressed data; and encapsulates the differential compressed data into frame data packets, which are then sent to the receiving end via a Universal Serial Bus (USB) 2.0 interface.

[0043] The receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, and displays it on the receiving end.

[0044] The receiving end updates the cached reference image based on the complete image of the current frame, and continues to decode the frame data of the next frame in the frame data in combination with the reference image to determine the complete image of the next frame, and displays it on the receiving end, until the frame data of all frames in the frame data has been decoded.

[0045] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects:

[0046] In this embodiment of the invention, the remote desktop transmission system includes a sending end and a receiving end. The sending end acquires the current screen image at a predetermined frame rate to obtain a frame image data stream, and performs frame difference processing on the frame image data stream to determine a difference marker sequence set. The sending end encodes the difference marker sequence set to obtain differential compressed data and encapsulates the differential compressed data into frame data packets, which are then sent to the receiving end via a Universal Serial Bus (USB) 2.0 interface. The receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame and displays it on the receiving end. Furthermore, the receiving end updates the cached reference image based on the complete image of the current frame, and continues to decode the frame data of the next frame in the frame data based on the reference image to determine the complete image of the next frame and display it on the receiving end, until the frame data of all frames in the frame data has been decoded.

[0047] As can be seen, in this embodiment of the invention, due to the simple and direct encoding and decoding process, both encoding and decoding can be performed immediately after each frame is captured, resulting in extremely low end-to-end latency. The encoded output of each frame can be transmitted instantly, eliminating the need to accumulate multiple frames or wait for complex calculations, as is the case with video encoding schemes provided in related technologies, thus reducing latency accumulation. Simultaneously, because each frame contains relatively small amounts of data, the USB 2.0 interface can complete the transmission within the frame interval, preventing queue backlog and ensuring stable frame rate output. Furthermore, the simple and direct encoding and decoding process reduces the complexity of image encoding and decoding to a certain extent, and eliminates the need for hardware / chip support, thereby improving the versatility of remote desktop transmission solutions and reducing remote desktop transmission costs.

[0048] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practice. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0049] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention, but do not constitute an undue limitation of the invention.

[0051] Figure 1 A schematic diagram of a remote desktop transmission system provided in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a transmitting end provided in an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a receiving end provided in an embodiment of the present invention;

[0054] Figure 4 This is a flowchart illustrating a remote desktop transmission method provided in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. Without conflict, the embodiments and features in the embodiments of this invention can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here. The term "comprising" and any variations thereof in the specification and claims of this invention are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0056] In this embodiment of the invention, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0057] Furthermore, unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of the present invention are used to distinguish multiple objects and are not used to limit the size, content, order, timing, priority, or importance of multiple objects. For example, "first flag bit" and "second length" are only used to distinguish flag bits, and do not indicate that the size, priority, or importance of these two flag bits are different.

[0058] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These are to be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the embodiments of the present invention, references may be made to existing industry solutions such as software, components, and models. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of the present invention, and do not imply that the applicant has already used or necessarily used such solutions.

[0059] In the technical solution of this invention, the acquisition, transmission, and use of images all comply with the requirements of relevant national laws and regulations.

[0060] To facilitate understanding of the technical solutions provided in the embodiments of the present invention, some key terms used in the embodiments of the present invention will be explained here first:

[0061] 4K: refers to an ultra-high-definition display standard with a resolution of approximately 3840×2160 pixels, totaling about 8.3 million pixels.

[0062] H.264 video coding standard (H.264) / Advanced Video Coding (AVC): The current mainstream video compression standard, which achieves high compression ratios through complex motion estimation and transform quantization techniques.

[0063] Compression ratio: The ratio of the original data size to the compressed data size. The solution provided by this invention can achieve an effective compression ratio of over 10:1 in typical remote desktop static scenarios.

[0064] High Efficiency Video Coding (HEVC) / H.265: A next-generation video coding standard. HEVC-SCC (Screen Content Coding) is an extended version of HEVC, adding tools such as color wheel mode and intra-frame block copying.

[0065] Joint Photographic Experts Group (JPEG) / Dynamic JPEG (MJPEG): Each frame is independently encoded using JPEG compression, without utilizing inter-frame redundancy, resulting in high bandwidth consumption but low latency.

[0066] Central Processing Unit (CPU) utilization: The percentage of time a processor is used to execute a specific task, reflecting the computational resources consumed by the algorithm.

[0067] Frames Per Second (FPS): The number of frames transmitted or displayed per second. For example, 15 fps.

[0068] Group of Pictures (GOP): A method of organizing frame sequences in video coding, consisting of one keyframe (Intra-coded Picture, I-frame) and several predicted frames.

[0069] Key Frame: In this invention, a key frame is a frame that can be fully decoded without referencing the previous frame and is used for initialization or resynchronization after frame loss.

[0070] The preset encoding rule (RLX encoding) is the improved run-length encoding proposed in this invention. Specifically, it records the run-length for unchanged pixel segments and directly outputs the pixel value for changed pixel segments. The encoding format consists of a segment flag + length + data triplet, thus maintaining lossless encoding while ensuring extremely low CPU complexity.

[0071] Run-Length Encoding (RLE): A basic lossless compression method that uses consecutive identical data values ​​plus a counter to replace repeated data sequences. It is the core idea behind RLX encoding.

[0072] Universal Serial Bus 2.0 (USB 2.0): A widely used universal serial bus standard with a theoretical speed of 480 Mbit / s and an actual single-link effective throughput of approximately 280–320 Mbit / s (35–40 MB / s).

[0073] Inter-frame difference: The amount of change in pixel values ​​between two adjacent frames. The scheme provided in this embodiment of the invention first calculates the inter-frame difference to obtain a difference marker sequence, and then applies RLX coding to encode the difference marker sequence, thereby transmitting only the changed areas.

[0074] Currently, existing remote desktop image transmission methods typically compress and encode desktop images before sending them to reduce transmission bandwidth requirements.

[0075] In related technologies, image compression encoding typically involves simple image encoding on the server side followed by direct transmission. However, this method has limited compression rates at high resolutions such as 4K, easily leading to network congestion and increased transmission latency. In recent years, related technologies have optimized this simple image encoding and direct transmission approach. For example, they employ mainstream video encoding technologies (H.264 / AVC, H.265 / HEVC, etc.) to encode video streams for remote desktops, or utilize dedicated encoding standards tailored to screen content characteristics (such as HEVC-SCC).

[0076] However, while these newly provided video coding standards utilize advanced techniques such as intra / inter-frame prediction and entropy coding to deliver high-quality images at lower bitrates and perform excellently in bandwidth-constrained environments, these existing technologies either require high-performance processors to support complex encoding and decoding or require dedicated hardware / chips, thus limiting their versatility and cost.

[0077] It is evident that the video encoding and decoding technologies in these areas have the following shortcomings:

[0078] (1) High CPU resource usage:

[0079] Advanced video coding systems (such as H.264 / H.265) achieve high compression ratios through complex algorithms, requiring significant CPU computation or dedicated hardware support. H.264 encoding, in particular, utilizes inter-frame differences, motion estimation, and transform quantization to reduce redundancy, significantly increasing the computational load for encoding and decoding. Without hardware acceleration, implementing real-time H.264 encoding at 4K@15fps in software is nearly impossible or will consume a large amount of CPU, making it difficult for the system to perform other tasks in parallel.

[0080] (2) Complex implementation:

[0081] While advanced coding standards such as HEVC-SCC are optimized for screen content (e.g., adaptive color conversion, intra-frame block copying, palette mode, etc.), their implementation involves numerous coding tools and mode selections, resulting in high encoder complexity, long algorithm processes, and high development and maintenance costs. On resource-constrained platforms (such as embedded devices and older computers), it is difficult to fully implement and run these complex encoders in real time.

[0082] (3) Delay and delay jitter:

[0083] Complex encoding often requires buffering multiple frames (such as GOP grouping and frame reordering), which may introduce additional latency in interactive scenarios. In addition, high-compression encoding often outputs data in batches to improve efficiency, resulting in large instantaneous bit rate fluctuations. This may cause latency jitter on fixed-bandwidth channels such as USB, which is detrimental to the smoothness of interaction.

[0084] Furthermore, remote desktop systems in related technologies may also choose simple and fast compression methods, such as frame-by-frame JPEG compression (MJPEG) or uncompressed / simple RLE compression. MJPEG compresses each frame of the desktop image independently into a JPEG image. This method has low decoding overhead and low latency, making it suitable for low-power devices. However, MJPEG lacks inter-frame compression, resulting in a low compression ratio, often requiring a bitrate much higher than H.264 to maintain equivalent image quality. For 4K high resolution, the amount of data generated by MJPEG is extremely large, making it difficult to continuously transmit high frame rate video under USB 2.0 bandwidth. Simultaneously, each JPEG frame encoding also requires certain CPU resources for discrete cosine transform and entropy coding; when the frame rate and resolution are high, CPU usage is not entirely negligible.

[0085] It is evident that there is a dilemma in the relevant technologies for high-definition image transmission on remote desktops: either complex encoding is used to reduce bandwidth but results in high CPU overhead and implementation complexity, or simple encoding is used to reduce CPU usage but requires a large amount of bandwidth and cannot support ultra-high-definition real-time transmission.

[0086] In view of this, the present invention provides a remote desktop transmission system. In this system, the sending end acquires the current screen image at a predetermined frame rate to obtain a frame image data stream, and performs frame difference processing on the frame image data stream to determine a difference marker sequence set. The sending end encodes the difference marker sequence set to obtain differential compressed data and encapsulates the differential compressed data into frame data packets, which are then sent to the receiving end via a Universal Serial Bus (USB) 2.0 interface. Furthermore, the receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, and displays it on the receiving end. The receiving end updates the cached reference image based on the complete image of the current frame, and continues to decode the frame data of the next frame in the frame data based on the reference image to determine the complete image of the next frame, which is then displayed on the receiving end, until the frame data of all frames in the frame data has been decoded.

[0087] As can be seen, in this embodiment of the invention, due to the simple and direct encoding and decoding process, both encoding and decoding can be performed immediately after each frame is captured, resulting in extremely low end-to-end latency. The encoded output of each frame can be transmitted instantly, eliminating the need to accumulate multiple frames or wait for complex calculations, as is the case with video encoding schemes provided in related technologies, thus reducing latency accumulation. Simultaneously, because each frame contains relatively small amounts of data, the USB 2.0 interface can complete the transmission within the frame interval, preventing queue backlog and ensuring stable frame rate output. Furthermore, the simple and direct encoding and decoding process reduces the complexity of image encoding and decoding to a certain extent, and eliminates the need for hardware / chip support, thereby improving the versatility of remote desktop transmission solutions and reducing remote desktop transmission costs.

[0088] In this embodiment of the invention, the remote desktop transmission system can be applied to desktop sharing and remote collaboration scenarios, such as enterprise office and team collaboration business scenarios, technical support and operation and maintenance service business scenarios, education and training and knowledge transfer business scenarios, etc. This embodiment of the invention does not limit this.

[0089] For example, the remote desktop transmission system provided in this embodiment of the invention is suitable for transmission channels with limited bandwidth, such as USB 2.0 interfaces. In typical scenarios such as remote desktop preview and presentation (PPT) slide sharing, it can stably achieve a refresh rate of 15 frames per second at 4K resolution.

[0090] To further illustrate the remote desktop transmission system and method provided in the embodiments of the present invention, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present invention provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of the present invention. In actual processing or when the device executes the method, it may be executed in the order shown in the embodiments or drawings, or in parallel (e.g., in an application environment with parallel processors or multi-threaded processing).

[0091] The following combination Figures 1-3 The figure shown illustrates the remote desktop transmission system in an embodiment of the present invention.

[0092] See Figure 1 As shown, the remote desktop transmission system includes a sender 110 and a receiver 120.

[0093] In this embodiment of the invention, the remote desktop transmission system includes: a first device (e.g., a remote host computer) corresponding to the sending end 110 and a second device (e.g., a local client) corresponding to the receiving end 120.

[0094] The sending end 110 runs a remote desktop service program, which allows users to remotely access the desktop, applications, and files of another computer over a network. The receiving end 120 runs a remote desktop client program. Since the remote desktop client program is terminal software that connects to and controls the remote computer / server desktop via the network, it can work in conjunction with the sending end's remote desktop service program to overcome physical location limitations and enable visual operation of remote devices (such as viewing files, running software, and debugging the system).

[0095] In this embodiment of the invention, the transmitting end 110 acquires the current screen image at a predetermined frame rate, obtains the frame image data stream, performs frame difference processing on the frame image data stream, determines the difference marker sequence set, encodes the difference marker sequence set to obtain differential compressed data, and encapsulates the differential compressed data into a frame data packet, which is then sent to the receiving end 120 through the Universal Serial Bus (USB) 2.0 interface.

[0096] In one possible implementation, the transmitting end 110 determines the difference marker sequence of each frame image in the frame image data stream except for the first frame; and determines a difference marker sequence set based on the obtained difference marker sequence.

[0097] In one possible implementation, the transmitting end 110 performs the following operations on each frame image in the frame image data stream, excluding the first frame: determining the reference frame image of the current frame image and the reference frame of the current frame image; the current frame image is the frame image of any frame other than the first frame; and traversing the current frame image and the reference frame image pixel by pixel to determine the difference marker sequence.

[0098] In one possible implementation, the transmitting end 110 compares the current frame image and the reference frame image in a row-by-row, left-to-right, top-to-bottom order to determine the length of the unchanged pixel segment and / or the changed pixel data block corresponding to the current frame image; and, based on the length of the unchanged pixel segment and the changed pixel data block corresponding to the current frame image, determines the difference marker sequence of the current frame image.

[0099] In one possible implementation, the transmitter 110 performs the following operations on each differential marker sequence in the differential marker sequence set: determining first compressed data of the length of the unchanged pixel segment in the differential marker sequence; and / or determining second compressed data of the changed pixel data block in each differential marker sequence; and the transmitter 110 determines a differential compressed data packet based on the obtained first compressed data and / or second compressed data.

[0100] In one possible implementation, the transmitting end 110 determines a first flag bit of the length of the unchanged pixel segment in the difference marker sequence; determines the length information of the length of the unchanged pixel segment in the difference marker sequence; and compresses the first flag bit and the length information of the length of the unchanged pixel segment in the difference marker sequence to determine the first compressed data of the length of the unchanged pixel segment in the difference marker sequence.

[0101] In one possible implementation, the transmitting end 110 determines the second flag bit of the changed pixel data block in the difference marker sequence; determines the number of pixels in the changed pixel data block in the difference marker sequence and the color value of each pixel corresponding to the number of pixels; and performs compression processing on the second flag bit, the number of pixels, and the color value of each pixel corresponding to the number of pixels in each changed pixel data block in the difference marker sequence to determine the second compressed data of the changed pixel data block in the difference marker sequence.

[0102] In an embodiment of the present invention, see Figure 2 As shown, the specific implementation scheme of the transmitter 110 is introduced in conjunction with the specific modules included in the transmitter 110.

[0103] Please continue reading Figure 2 The transmitting end 110 includes a screen capture module 1101, an RLX encoding module 1102, and a USB transmission module 1103. The screen capture module 1101 is used to capture the current screen image at a predetermined frame rate to obtain a frame image data stream. The RLX encoding module 1102 is used to perform frame difference processing on the frame image data stream to determine the difference marker sequence set and to encode the difference marker sequence set to obtain differential compressed data. The USB transmission module 1103 is used to encapsulate the differential compressed data into frame data packets and send them to the receiving end 120 through the Universal Serial Bus (USB) 2.0 interface.

[0104] In this embodiment of the invention, the screen capture module 1101 can determine a preset frame rate and capture screenshots of the current desktop of the remote computer according to the preset frame rate to obtain a continuous frame image data stream.

[0105] In this embodiment of the invention, if the remote desktop service program running on the sending end 110 continues to run, the screen capture module 1101 can continuously capture data at a preset frame rate, thereby continuously acquiring a continuous stream of frame image data until the remote desktop service program running on the sending end 110 stops running, or the current session of the remote desktop service program running on the sending end 110 stops.

[0106] In this embodiment of the invention, a remote computer is connected via a network, and a predetermined frame rate (i.e., a set number of desktop screenshots per second) is used. The desktop of the remote computer is then automatically and continuously captured at this frame rate. If the remote computer uses a 4K monitor, each captured desktop screenshot will be a high-definition image of 3840×2160 pixels. These consecutive high-definition screenshots are sequentially arranged into a "video stream" (i.e., the aforementioned data stream). For example, with a preset frame rate of 15fps, the remote computer can acquire a 3840×2160 resolution image approximately every 66 milliseconds.

[0107] Optionally, embodiments of the present invention may also support dynamic adjustment of the preset frame rate based on bandwidth and processing capability. That is, the preset frame rate can be dynamically adjusted based on bandwidth and processing capability, and is updatable.

[0108] For example, under the USB 2.0 channel, the preset frame rate limit can be automatically adjusted to about 15fps, which can meet the smoothness requirements of PPT presentations and general remote operations.

[0109] For example, bandwidth permissions such as USB 3.0 or gigabit network can automatically adjust the preset frame rate to 30fps or even higher.

[0110] For example, when bandwidth is extremely low or the system is busy, the preset frame rate can be automatically adjusted to reduce the frame rate to ensure that each frame is transmitted completely.

[0111] As can be seen, the preset frame rate can be increased or decreased in the embodiments of the present invention, and there is no limitation on this in the embodiments of the present invention. Furthermore, since the encoding latency of the solution provided by the present invention is very low, the adjustment of the preset frame rate has little impact on the interaction, and therefore can improve the user experience to a certain extent.

[0112] In this embodiment of the invention, after obtaining the frame image data stream, the frame image data stream can be temporarily stored in the frame buffer for subsequent encoding processing.

[0113] In this embodiment of the invention, the RLX encoding module 1102 can perform frame difference comparison processing and RLX encoding packing processing.

[0114] In this embodiment of the invention, the frame difference comparison processing specifically involves: after determining the frame image data stream, determining whether a reference frame image exists for each frame image. Specifically, if all frame images in the frame image data stream, except for the first frame, contain the frame image of the previous frame, then it is determined that each frame image in the frame image data stream, except for the first frame (described below as the current frame image), contains a reference frame image.

[0115] Furthermore, the data of the current frame image and the reference frame image can be traversed pixel by pixel to compare whether the color values ​​of corresponding pixels are the same, thereby generating a difference marker sequence for the current frame image. Preferably, the comparison between the current frame image and the reference frame image can be performed row by row from left to right and from top to bottom, and the change information is recorded when a pixel change is detected. Also, when consecutive pixels remain unchanged, only the counter is incremented without recording them one by one. This process yields two types of basic information: (1) Length of unchanged pixel segment: indicating how many pixels are exactly the same as the previous frame starting from a certain starting pixel. (2) Change pixel data block: indicating that a segment of consecutive pixels has changed, and the new values ​​of these pixels need to be recorded.

[0116] Optionally, to improve image transmission efficiency, if the color repeats within the changed pixel data block, the changed pixel data block can be further compressed. Run-length compression can be used for compression, but this is not limited in this embodiment of the invention. It should be noted that, in principle, the changed pixel data block will provide new color values ​​pixel by pixel to ensure lossless restoration.

[0117] In this embodiment of the invention, after frame difference comparison processing, a “differential description” of the current frame image relative to the previous frame image (i.e., the reference frame image) is obtained – essentially an incremental update record, that is, skipping the unchanged areas and only capturing the changed content.

[0118] Optionally, if the current frame image is set as a keyframe (e.g., due to periodic refresh or significant scene changes), the frame difference comparison process can be skipped, and all data of the current frame can be directly used as the changed pixel data block. That is, the difference marker sequence of the current frame only includes the changed pixel data block.

[0119] As can be seen, some frame images do not have pixel changes compared to the previous frame image (i.e., the reference frame image). In this case, the difference marker sequence corresponding to the frame image of that frame only includes the length of the unchanged pixel segment. For example, if the current frame is a periodically refreshed frame image or a frame image that has encountered a significant scene change, meaning that all the pixel data of the current frame are changed pixels, then frame difference comparison processing can be skipped for this current frame, and all the data of the current frame can be directly treated as changed pixel data blocks. That is, the difference marker sequence only includes changed pixel data blocks. Therefore, the difference marker sequence of the current frame image includes the length of the unchanged pixel segment and / or changed pixel data blocks.

[0120] In this embodiment of the invention, the RLX encoding and packing process is as follows: a simple and efficient RLX code stream format is defined, for example, the aforementioned difference marker sequence is encoded using a flag bit + length + data method.

[0121] Specifically, a flag can be used to distinguish between "unchanged pixel segments" and "changing pixel segments." For example, 0 indicates that it is followed by an unchanged segment length value, and 1 indicates that it is followed by changing segment data. For unchanged pixel segments, the segment length (number of pixels) can be stored directly. Since USB transmission is suitable for byte alignment, the segment length can be stored with one or two bytes, which is sufficient to represent a fairly long identical area. For changing pixel segments, the number of pixels in the segment can be stored first, and then the new color value of each pixel in the segment can be stored sequentially (for example, represented by two bytes of RGB565).

[0122] Alternatively, if there are many pixels within the variable pixel segment, compression can be performed before storage. For example, a simple lossless algorithm (such as the RLE algorithm, the Lempel-Ziv (LZ) algorithm, etc.) can be used to further compress the color data to reduce its length.

[0123] In this embodiment of the invention, the overall processing logic of the RLX encoding module 1102 can be implemented based on the following pseudocode:

[0124] Input: CurrFrame (the pixel array of the current frame, i.e., the image of the current frame), PrevFrame (the pixel array of the previous frame, i.e., the image of the reference frame)

[0125] Output: EncodedStream (a byte stream encoded with RLX)

[0126] Initialize EncodedStream to empty

[0127] Set index i = 0

[0128] while i < total number of pixels:

[0129] If CurrFrame[i] is the same as PrevFrame[i]:

[0130] / / Enter the unchanged area

[0131] count = 0

[0132] while i < total number of pixels AND CurrFrame[i] == PrevFrame[i] AND count < maximum count value:

[0133] count+=1

[0134] i+=1

[0135] Output flag 0 to EncodedStream

[0136] Output count to EncodedStream / / This indicates that count pixels have not changed else:

[0137] / / Entering the changed area

[0138] change_data=[]

[0139] while i < total number of pixels AND CurrFrame[i]! = PrevFrame[i]:

[0140] Add the pixel value of CurrFrame[i] to the change_data list.

[0141] i+=1

[0142] Output flag 1 to EncodedStream

[0143] Output length(change_data) to EncodedStream / / Number of changed pixels Output all pixel values ​​in the change_data list to EncodedStream

[0144] End of while

[0145] The pseudocode above describes the process of comparing pixels one by one and outputting RLX compressed data (i.e., the aforementioned differential compressed data).

[0146] Optionally, adjacent unchanged pixel segments and changed pixel segments can be cyclically merged to effectively avoid sending redundant information pixel by pixel. It should be noted that, in order to prevent long continuous segments from causing the length field to be too large in extreme cases, a "maximum count value" can be set to make the block encoding of an unchanged pixel segment or changed pixel segment controllable, for example, encoding a maximum of 65535 pixels, and then splitting the encoded pixels into multiple segments.

[0147] In this embodiment of the disclosure, the amount of frame data ultimately output by the RLX encoding module 1102 depends on the degree of image change. Specifically, for static areas of the image (i.e., unchanged pixel segments), only a small flag and count bytes are generated; for changed areas of the image (i.e., changed pixel segments), new values ​​are output pixel by pixel.

[0148] In this embodiment of the invention, when the entire frame changes completely, the encoding result is equivalent to adding a small amount of identification overhead before each pixel, and the data volume is slightly larger than that of the uncompressed image. However, in general desktop sharing scenarios, it is rare for images to change completely, and a large number of adjacent pixels often have the same or similar content, which can be merged into the changed segment for further compression.

[0149] It can be seen that the RLX encoding module 1102 provided by the embodiments of the present invention can provide a considerable compression ratio with low complexity.

[0150] In this embodiment of the invention, the USB transmission module 1103 sends the frame data packets to the receiving end 120 via USB 2.0. Optionally, in addition to the encoded data, each frame image data packet may also include a small amount of header information such as frame sequence number and timestamp to support frame reassembly and packet loss detection.

[0151] In this embodiment of the invention, considering the limited bandwidth of USB 2.0, a frame flow control strategy can be adopted. This strategy may include ensuring that the next frame is sent only after the previous frame has been completely transmitted, and discarding or merging overly dense frames if necessary to avoid congestion.

[0152] In the specific implementation process, experiments were conducted at a preset frame rate of 15fps. The amount of data generated by the scheme provided in this embodiment of the invention, after compression, can usually be transmitted by USB 2.0 within two frame intervals, thereby ensuring a stable frame rate.

[0153] As can be seen, the transmitter 110 and the receiver 120 establish a data channel through a USB 2.0 interface (e.g., through a USB virtual network card or a USB straight-through cable).

[0154] In this embodiment of the invention, the receiving end 120 decodes the frame data of the current frame in the frame data to determine the complete image of the current frame and displays it on the receiving end 120; and updates the cached reference image according to the complete image of the current frame, and continues to decode the frame data of the next frame in the frame data in conjunction with the reference image to determine the complete image of the next frame and display it on the receiving end 120, until the frame data of all frames in the frame data has been decoded.

[0155] In one possible implementation, the receiving end 120 reads the flag bit of the frame data of the current frame; if it is determined that the flag bit points to an unchanged pixel segment, then according to the first length carried by the frame data of the current frame, it copies the number of pixels corresponding to the first length from the previously stored image to the specified position of the current frame buffer to determine the complete image of the current frame.

[0156] In one possible implementation, if the receiving end 120 determines that the flag bit points to the changed pixel segment, it reads the second length of the changed pixel segment and continuously reads the pixel values ​​of the second length, and fills the pixel values ​​of the second length into the specified positions of the current frame buffer in sequence to determine the complete image of the current frame.

[0157] In an embodiment of the present invention, see Figure 3 As shown, the specific implementation scheme of receiver 120 is introduced in conjunction with the specific modules included in receiver 120.

[0158] See Figure 3 As shown, the receiver 120 includes a USB receiver module 1201, an RLX decoding module 1202, and a display output module 1203. The USB receiver module 1201 receives frame data packets. The RLX decoding module 1202 decodes the frame data packets to obtain a complete frame image, and the display output module 1203 displays the obtained complete frame image.

[0159] In this embodiment of the invention, after the USB receiving module 1201 of the receiving end 120 receives the frame data packet, it is handed over to the RLX decoding module 1202 for parsing.

[0160] In this embodiment of the invention, the RLX decoding module 1202 reads the bitstream according to the opposite logic to the encoding. Specifically, it can read the flag bits sequentially: if it is 0, it indicates that the following is the length of an unchanged pixel segment; if it is 1, it indicates that the following is data of a changed pixel segment. Optionally, for unchanged pixel segments, the corresponding number of pixels can be directly copied from the locally stored previous frame image to the specified position in the current frame buffer according to the length. Optionally, for changed pixel segments, the length n of the segment is read, and then n pixel values ​​are read consecutively and filled into the corresponding positions in the current frame buffer sequentially.

[0161] In this embodiment of the invention, the RLX decoding module 1202 also follows the frame-by-frame scanning order during the decoding process to ensure consistency with the encoding process of the RLX encoding module 1102. After completing the parsing of the entire bitstream, the pixel data of the current frame is reconstructed. The RLX decoding module 1202 can send the reconstructed frame image to the display output module 1203, thereby presenting a desktop screen consistent with the remote host on the local screen. At the same time, the frame image is saved as a local reference frame for use when decoding the next frame (processing pixel copying of unmodified segments).

[0162] In this embodiment of the invention, the remote desktop transmission system will continue the above-mentioned encoding, transmission, and decoding loop until the remote session ends.

[0163] In this embodiment of the invention, to ensure synchronization, the inventive end 110 and the receiving end 120 can verify frame continuity using frame sequence numbers or timestamps. Thus, in the event of frame loss or data packet corruption, the receiving end 120 can request the sending end 110 to retransmit key frames for resynchronization. Since the amount of encoded data in this invention is small, the resynchronization overhead is minimal, further improving robustness.

[0164] Based on the same concept, embodiments of the present invention also provide a remote desktop transmission method, applied to a remote desktop transmission system, the system including a sending end and a receiving end, see [link to relevant documentation]. Figure 4 As shown, the method includes:

[0165] Step 401: The sending end acquires the current screen image according to the predetermined frame rate, obtains the frame image data stream, and performs frame difference processing on the frame image data stream to determine the difference marker sequence set;

[0166] Step 402: The transmitting end encodes the differential marker sequence set to obtain differential compressed data; and encapsulates the differential compressed data into frame data packets, which are then sent to the receiving end via the Universal Serial Bus (USB 2.0) interface.

[0167] Step 403: The receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame and displays it on the receiving end;

[0168] Step 404: The receiving end updates the cached reference image based on the complete image of the current frame, and continues to decode the frame data of the next frame in the frame data in combination with the reference image to determine the complete image of the next frame, and displays it on the receiving end, until the frame data of all frames in the frame data has been decoded.

[0169] In this embodiment of the invention, the implementation of steps 401-402 can be found in the foregoing. Figure 2 For the implementation of the sending end 110, the implementation of steps 403-404 mentioned above can be found in the previous description. Figure 3 The implementation details of the transmitter 120 will not be elaborated here.

[0170] The solution provided by the embodiments of the present invention has the following advantages:

[0171] Low CPU usage: The RLX encoding algorithm is based on simple pixel comparison and run-length compression, requiring no complex transformations or motion searches, resulting in low computational complexity. The encoding process mainly involves memory operations (such as comparison, counting, and copying), placing extremely low demands on CPU capabilities, making it suitable for real-time operation on general processors and even low-power embedded chips. Therefore, the solution provided in this invention significantly reduces CPU utilization at the encoding end compared to video encoding methods such as H.264; compared to frame-by-frame JPEG compression, it also avoids frequent DCT and entropy coding operations. Consequently, the remote desktop transmission system can allocate more CPU resources to applications, improving overall response speed.

[0172] High bandwidth efficiency: Thanks to the inter-frame difference processing scheme, the solution provided in this embodiment of the invention only transmits necessary update information. For typical office desktops and PPT presentation scenarios, the differences between adjacent frames are small, allowing RLX encoding to achieve a high compression ratio. In particular, large areas of unchanged data are represented by only a small amount of counting information, making simple graphics such as text content or solid-color backgrounds highly suitable for compression. Although the compression ratio of the solution provided in this embodiment of the invention may be slightly lower than that of complex codecs such as H.264, in most remote desktop interaction scenarios, the compressed bitrate can be carried by USB 2.0, achieving smooth transmission. Furthermore, compared to MJPEG's frame-by-frame independent compression, this invention fully utilizes inter-frame redundancy, significantly reducing bandwidth requirements. Therefore, the solution provided in this embodiment of the invention has a clear advantage in bandwidth-constrained environments.

[0173] High real-time performance and low latency: The RLX encoding process is simple and direct, with encoding and decoding occurring immediately after each frame capture, resulting in extremely low end-to-end latency. The encoded output of each frame can be transmitted instantly, eliminating the need to accumulate multiple frames or wait for complex calculations, as is the case with some video encoding methods, thus reducing latency accumulation. Furthermore, due to the small size of each frame, the USB 2.0 interface can complete transmission within the frame interval, preventing queue backlog and ensuring stable frame rate output. Compared to H.264 encoding, which requires buffered frames and a longer processing pipeline, the solution provided in this invention improves the real-time response of interactive operations (such as mouse movement and window dragging).

[0174] Low implementation cost: The RLX algorithm is easy to implement in software and can be accelerated with minimal hardware resources. Its logic is clear, and the encoder is small, making it suitable for integration into existing remote desktop software or device firmware without requiring expensive dedicated codec chips. Furthermore, the decoding algorithm has low overhead, resulting in minimal computational and power consumption by the client. For scenarios requiring long-running remote desktop sessions (such as remote monitoring), low resource consumption also means lower heat generation and power consumption, leading to more stable system operation.

[0175] In summary, the solution provided by the embodiments of the present invention minimizes CPU and implementation complexity costs while maintaining acceptable compression efficiency. In particular, under transmission channels such as USB 2.0, the solution provided by the embodiments of the present invention can achieve stable transmission of 4K resolution desktops at a frame rate of 15fps, significantly improving the performance bottleneck of existing technologies in high-resolution remote desktop transmission.

[0176] Furthermore, the RLX encoding and inter-frame difference processing schemes proposed in this invention exhibit the following unique advantages compared to traditional schemes in remote desktop applications:

[0177] (1) Comparison with H.264 video coding scheme:

[0178] The H.264 video coding scheme achieves high compression through complex motion estimation and entropy coding, with low bit rate requirements. However, the encoding process is extremely CPU-intensive and often requires hardware acceleration. In a purely software environment, real-time H.264 encoding of 4K frames would consume a large amount of CPU resources and might not even be possible in real time.

[0179] The RLX encoding process of this invention does not perform motion search or frequency domain transformation, and only requires a very low amount of computation to complete the compression, ensuring real-time performance.

[0180] Furthermore, although H.264 is far more efficient than RLX in dynamic video (such as full-screen video playback) scenarios, in typical remote desktop scenarios (i.e., static background + a few local changes), the bitrate of the solution provided by this invention can be completely controlled within the USB 2.0 range after compression, while the CPU utilization is only a small fraction of that of the H.264 solution.

[0181] For example, in experiments, compressing a 4K desktop video at 15fps using software H.264 (x264 encoder) resulted in CPU usage approaching 100%, while encoding the same content using the solution provided in this invention resulted in CPU usage of less than 20%, a stable frame rate, and no need for any dedicated hardware. Clearly, this low CPU usage characteristic makes the solution provided in this invention more suitable for deployment in environments with limited CPU performance or those requiring parallel processing.

[0182] (2) Comparison with MJPEG intra-frame compression scheme:

[0183] MJPEG intra-frame compression compresses each frame as an independent JPEG image, which is simple to decode and has low latency. However, due to the lack of inter-frame utilization, its compression ratio is not ideal for screen content scenarios. Especially for text and graphical interfaces, JPEG often requires a high quality factor to avoid significant distortion, resulting in a large single-frame image size.

[0184] For example, in the case of a 4K desktop, a single high-quality JPEG image frame can reach hundreds of KB to several MB in size. Even at 15fps, it can generate hundreds of megabytes of data per second, exceeding the capacity of USB 2.0.

[0185] The solution provided in this embodiment of the invention significantly reduces the amount of data by utilizing inter-frame redundancy.

[0186] For example, in a static office environment, if most of the screen remains unchanged, the solution provided by this invention may only need to send less than 1% of the original image size to update the screen (e.g., mouse movement). Even when significant changes occur (e.g., interface switching), the instantaneous data peak is lower than when MJPEG sends a complete frame.

[0187] In addition, MJPEG requires JPEG compression for each frame, and its CPU overhead increases with the resolution. However, the solution provided by the present invention is always linear scanning, and the processing time for 4K images can be controlled within a short fixed range, which is friendly to CPU frequency and cache.

[0188] In actual testing, the MJPEG solution often consumes more than 300Mbps of bandwidth and has a moderate to high CPU utilization at 4K@15fps, while the solution provided by this invention only requires about 200Mbps or even less bandwidth to operate, with significantly lower CPU utilization and higher overall efficiency.

[0189] (3) Comparison with dedicated codec schemes such as HEVC-SCC:

[0190] The new generation HEVC-SCC standard offers higher compression performance for screen content, supporting tools such as color palettes and intra-frame block copying, achieving near-lossless high compression ratios. However, SCC encoders are extremely complex to implement, requiring advanced hardware to encode 4K content in real time.

[0191] The solution provided by this invention offers sufficient compression for most remote desktop applications, providing smooth and clear visuals even with limited bandwidth. Furthermore, it boasts low system overhead and low deployment costs. Moreover, the solution outputs a standard pixel image data stream, avoiding complex international encoding standards, making it suitable for free use and widespread adoption in products, thus enhancing the solution's versatility.

[0192] In actual implementation, the solution provided by this invention has been tested and verified in a variety of typical remote desktop transmission scenarios, and the results show that it performs well in 4K resolution remote desktop transmission.

[0193] The test environment was as follows: the sending end was a PC equipped with an 8th generation Intel i5 quad-core processor, with a resolution of 3840×2160 (4K UHD), connected to the receiving end via USB 2.0. The receiving end was a regular laptop. The preset frame rate was set to 15fps. The comparison schemes included the scheme provided in this embodiment of the invention, traditional MJPEG compression, and H.264 software encoding. All schemes were run in software on the sending end's CPU to compare their resource consumption and transmission performance.

[0194] Remote desktop transfer scenario 1: The transferred content consists of a static interface provided by the server, which is then operated by the client using a mouse. This may also involve demonstration scenarios, where static images are accompanied by mouse guidance. In other words, it's a scenario of a static desktop plus mouse movement.

[0195] In Scenario 1, the sending end displays a static, high-resolution image as the desktop background (filling the entire screen) and then simulates continuous mouse pointer movement. In this scenario, only the mouse pointer area (a few tens of pixels square) changes per frame. The solution provided in this embodiment only needs to send minimal differential data per frame (mainly pixels in the pointer shape area), with a frame size of approximately 10KB and bandwidth usage of about 1.2Mbps, far below the USB 2.0 limit. The sending end's CPU utilization during this process is only about 5%.

[0196] In contrast, the MJPEG scheme compresses the entire screen image for each frame, sending it repeatedly even if the background remains unchanged. A single JPEG frame is approximately 500KB, and at a frame rate of 15fps, the bandwidth is close to 60Mbps. Clearly, the bandwidth usage of the MJPEG scheme (i.e., 60Mbps) is significantly higher than that of the scheme provided in this embodiment (i.e., 1.2Mbps) by an order of magnitude.

[0197] In addition, although the H.264 scheme can reduce the bandwidth to about 0.5Mbps by utilizing the similarity between consecutive frames, the CPU usage on the encoding side is about 50%, and the processing latency is also slightly higher.

[0198] As can be seen, the solution provided by the embodiments of the present invention achieves the dual advantages of the lowest bandwidth and the lowest CPU usage in this scenario.

[0199] Remote desktop transfer scenario 2: The content to be transferred is: text editing / slide presentation.

[0200] In scenario 2, the sending end runs software to perform operations such as slideshow presentations or document editing. Considering that in this type of application, the screen content remains static most of the time, with occasional partial updates such as page turning or typing, a script containing multiple PPT page turning and text editing operations can be selected for testing.

[0201] For example, when the slideshow is viewed statically, the solution provided by this embodiment of the invention generates virtually no data transmission, only sending updates when animation is activated or when a page is turned. The average bitrate throughout the demonstration was less than 100 Mbps, with a peak bitrate of approximately 350 Mbps in the page-turning frame. However, since the effective throughput of USB 2.0 can approach the theoretical value, this instantaneous peak remained within an acceptable range and did not cause frame drops. The overall frame rate remained consistently at 15 fps, resulting in clear and error-free visuals.

[0202] In contrast, the MJPEG scheme requires transmitting the entire frame each time a page is turned, with peak speeds approaching 600Mbps. This exceeds the USB 2.0 bandwidth, causing a brief drop in frame rate and resulting in noticeable stuttering.

[0203] In addition, although H.264 encoding has the highest compression ratio in this scenario, with an average bitrate of only about 50Mbps, the CPU encoding thread is almost fully loaded for a long time, and occasionally frames are dropped due to insufficient calculation, causing some frames to be skipped.

[0204] Clearly, the smoothness and stability of the solution provided in this embodiment of the invention are demonstrated here—ensuring stable image quality and frame rate without placing an excessive burden on the system.

[0205] Remote desktop transfer scenario 3: The content to be transferred is: high-definition video content.

[0206] Scenario 3 tests an extreme case, where a full-screen high-definition video (1080p video enlarged to 4K full-screen display) is played on the remote desktop. This is the most unfavorable scenario for remote desktop system applications because each frame changes almost completely, equivalent to a high-speed moving video.

[0207] The solution provided in this embodiment of the invention degenerates into near-frame-by-frame compression in this scenario: each frame has a large number of pixel changes, and the average frame size after encoding reaches 2-3MB. At 15fps, the bandwidth requirement is close to 360Mbps, approaching the USB 2.0 limit, but it still barely maintains continuous frame output. Since the solution provided in this embodiment of the invention is lossless transmission pixel by pixel, the image quality maintains the clarity of the original video, but such a large data stream has already filled the channel.

[0208] In contrast, although the H.264 solution demonstrates advantages in this scenario, its mature video compression algorithm allows for smooth playback at a bitrate of approximately 100Mbps, which is far lower than the solution provided in this embodiment of the invention. However, the implementation of the H.264 solution requires real-time encoding support from GPU hardware, while pure CPU encoding cannot keep up with the frame rate.

[0209] Furthermore, the MJPEG solution is almost unsuitable for this scenario—4K frame-by-frame JPEG has a data rate exceeding 1Gbps per second, and USB 2.0 is completely unable to transmit such a high bitrate, resulting in extremely choppy playback.

[0210] As can be seen, the solution provided by the embodiments of the present invention is geared towards typical use cases of remote office work, and can cover desktop content ranging from static to moderately varied, with efficient transmission and excellent performance, and can even work with video content in full motion.

[0211] As can be seen from the above tests, the solution provided by this invention achieves a balance between performance and overhead in remote desktop applications. In common remote interaction scenarios, it consumes less CPU resources than video encoding methods such as H.264, while its compression efficiency is significantly better than simple methods like MJPEG. It can still provide a good user experience even with limited bandwidth like USB 2.0. This is of great value for applications that need to implement high-definition remote desktops on various terminals (including older computers, embedded devices, and even single-board computers with Advanced RISC Machine (ARM) architecture).

[0212] As can be seen, based on the solution provided by the embodiments of the present invention, users do not need to add a special compression card or upgrade the network; they can improve remote desktop performance simply by updating the software.

[0213] For example, in remote teaching and remote conference presentations, 4K PPT images can be stably transmitted through the technical solutions provided in the embodiments of this invention, allowing participants in remote locations to see clear and smooth slides; in operation and maintenance monitoring, engineers can view high-resolution server desktops with low latency; in scenarios where portable mobile devices are connected to remote screens via USB, the technical solutions provided in the embodiments of this invention can also play a role, realizing "plug and play" high-definition desktop extension.

[0214] Furthermore, the desktop transmission system and method provided in this invention are easily implemented in software and can be accelerated with minimal hardware resources. Their logic is clear, the encoder is small, and they are suitable for integration into existing shared software or device firmware, reducing image encoding and transmission costs.

[0215] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A remote desktop transmission system, characterized in that, The system includes a transmitter and a receiver, wherein... The sending end acquires the current screen image at a predetermined frame rate, obtains the frame image data stream, and performs frame difference processing on the frame image data stream to determine the difference marker sequence set; The transmitting end encodes the differential marker sequence set to obtain differential compressed data; and encapsulates the differential compressed data into frame data packets, which are then sent to the receiving end via a Universal Serial Bus (USB) 2.0 interface. The receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, and displays it on the receiving end. The receiving end updates the cached reference image based on the complete image of the current frame, and continues to decode the frame data of the next frame in the frame data in combination with the reference image to determine the complete image of the next frame, and displays it on the receiving end, until the frame data of all frames in the frame data has been decoded.

2. The system according to claim 1, characterized in that, The sending end performs frame difference processing on the frame image data stream to determine a set of difference marker sequences, including: The transmitting end determines the difference marker sequence of each frame image in the frame image data stream, excluding the first frame; The sending end determines the differential marker sequence set based on the obtained differential marker sequence.

3. The system according to claim 2, characterized in that, The sending end determines the difference marker sequence of each frame image in the frame image data stream, excluding the first frame, including: The sending end performs the following operations on each frame image in the frame image data stream, excluding the first frame: The transmitting end determines the reference frame image of the current frame image and the reference frame of the current frame image; the current frame image is the frame image of any frame other than the first frame; The transmitting end traverses the current frame image and the reference frame image pixel by pixel to determine the difference marker sequence.

4. The system according to claim 3, characterized in that, The transmitting end traverses the current frame image and the reference frame image pixel by pixel to determine the difference marker sequence, including: The transmitting end performs pixel comparison between the current frame image and the reference frame image in a row-by-row, left-to-right, top-to-bottom order to determine the length of the unchanged pixel segment and / or the changed pixel data block corresponding to the current frame image. The transmitting end determines the difference marker sequence of the current frame image based on the length of the unchanged pixel segment and the changed pixel data block corresponding to the current frame image.

5. The system according to claim 4, characterized in that, The transmitting end encodes the differential marker sequence set to obtain differentially compressed data, including: The sending end performs the following operations on each differentially labeled sequence in the differentially labeled sequence set: First compressed data to determine the length of the unchanged pixel segment in the difference marker sequence; and / or, Determine the second compressed data for the changed pixel data block in each difference marker sequence; The sending end determines the differential compressed data packet based on the obtained first compressed data and / or second compressed data.

6. The system according to claim 5, characterized in that, The first compressed data for determining the length of the unchanged pixel segment in the difference marker sequence at the transmitting end includes: The transmitting end determines the first flag bit of the length of the unchanged pixel segment in the difference marker sequence; The transmitting end determines the length information of the unchanged pixel segment in the difference marker sequence; The transmitting end compresses the first flag bit and length information of the unchanged pixel segment in the difference marker sequence to determine the first compressed data of the unchanged pixel segment length in the difference marker sequence.

7. The system according to claim 5, characterized in that, The sending end determines the second compressed data of the changed pixel data block in each difference marker sequence, including: The transmitting end determines the second flag bit of the changed pixel data block in the difference marker sequence; The transmitting end determines the number of pixels in the changed pixel data block in the difference marker sequence and the color value of each pixel corresponding to the number of pixels; The transmitting end compresses the second flag bit, the number of pixels, and the color value of each pixel corresponding to the number of pixels in each difference marker sequence of the changed pixel data block to determine the second compressed data of the changed pixel data block in the difference marker sequence.

8. The system according to any one of claims 1-7, characterized in that, The receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, including: The receiving end reads the flag bit of the frame data of the current frame; If the receiving end determines that the flag bit points to an unchanged pixel segment, it copies the number of pixels corresponding to the first length from the previously stored image to a specified position in the current frame buffer according to the first length carried by the frame data of the current frame, thereby determining the complete image of the current frame.

9. The system according to claim 8, characterized in that, The receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, including: If the receiving end determines that the flag bit points to the changing pixel segment, it reads the second length of the changing pixel segment and continuously reads the pixel values ​​of the second length, and fills the pixel values ​​of the second length into the specified position of the current frame buffer in sequence to determine the complete image of the current frame.

10. A remote desktop transmission method, characterized in that, Applied to a remote desktop transmission system, the system including a sending end and a receiving end, the method includes: The sending end acquires the current screen image at a predetermined frame rate, obtains the frame image data stream, and performs frame difference processing on the frame image data stream to determine the difference marker sequence set; The transmitting end encodes the differential marker sequence set to obtain differential compressed data; and encapsulates the differential compressed data into frame data packets, which are then sent to the receiving end via a Universal Serial Bus (USB) 2.0 interface. The receiving end decodes the frame data of the current frame in the frame data to determine the complete image of the current frame, and displays it on the receiving end. The receiving end updates the cached reference image based on the complete image of the current frame, and continues to decode the frame data of the next frame in the frame data in combination with the reference image to determine the complete image of the next frame, and displays it on the receiving end, until the frame data of all frames in the frame data has been decoded.

Images