A remote desktop bidirectional mouse style synchronization method and system based on a dynamic style library

Abstract

The application belongs to the technical field of cross-region virtual network connection, and particularly relates to a remote desktop bidirectional mouse style synchronization method and system based on a dynamic style library, which comprises the following steps: a layered dynamic style library is constructed, the dynamic style library comprises a basic style layer for storing a default cursor of an operating system and an extended style layer for storing a custom cursor of an application; when a mouse style change event is captured by a controlled end, a style identifier, vector path data and screen DPI parameters are extracted; the style identifier, the vector path data and the screen DPI parameters are encapsulated into binary data packets and transmitted to a control end through a compression channel based on a WebSocket protocol and Protobuf coding. The SVG vector format is used to replace a bitmap, and a sawtooth problem under high resolution is eliminated; an incremental request mechanism avoids repeated transmission of known styles, and the known styles are stored according to operating system versions, so that a compatibility problem of new and old systems is solved.

Description

Technical Field

[0001] This invention belongs to the field of cross-regional virtual network connection technology, specifically relating to a method and system for synchronizing bidirectional mouse styles on a remote desktop based on a dynamic style library. Background Technology

[0002] Current remote desktop technologies primarily employ two cursor synchronization schemes: One relies on the operating system's native cursor library. When the controlling and controlled operating systems differ, incompatibility in cursor specifications between systems leads to abnormal display of basic cursors such as arrows and wait circles, or a degradation to the default style. The second scheme achieves cross-platform compatibility through bitmap transmission, rendering the cursor as a PNG image for real-time transmission. While this solves the problem of style uniformity, it suffers from the following drawbacks: 1. Scaling on high-resolution screens produces jagged edges, affecting precision operations such as CAD drawing; 2. The animated cursor requires the bitmap to be transmitted frame by frame, and the bandwidth occupied by a single synchronization exceeds 50KB. When the latency is greater than 100ms in cross-border networks, the operation lag is significant. 3. The complete cursor library occupies more than 20MB of storage, which cannot be deployed on IoT devices with less than 128MB of RAM, thus restricting the application of touch interaction in industrial settings. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention provides a remote desktop bidirectional mouse style synchronization method and system based on a dynamic style library. This addresses issues such as jagged edges caused by scaling on high-resolution screens, which affect precise operations like CAD drawing; animated cursors requiring frame-by-frame bitmap transmission, with single synchronization consuming over 50KB of bandwidth; significant operational lag when cross-border network latency exceeds 100ms; and a complete cursor library requiring over 20MB of storage, making it unsuitable for deployment on IoT devices with less than 128MB of RAM, thus restricting touch interaction applications in industrial settings.

[0004] One embodiment of the present invention provides a method for synchronizing two-way mouse styles on a remote desktop based on a dynamic style library, comprising: Construct a hierarchical dynamic style library, which includes a base style layer that stores the operating system's default cursor and an extended style layer that stores application-defined cursors; When the controlled device captures mouse style change events, it extracts the style identifier, vector path data, and screen DPI parameters. The style identifier, vector path data, and screen DPI parameters are encapsulated into binary data packets and transmitted to the control terminal via a compression channel based on the WebSocket protocol and Protobuf encoding. The control terminal loads the corresponding style from the dynamic style library based on the received style identifier, and then renders the cursor size after interpolating the cursor size based on the control terminal's DPI value. The control end sends the style hash value generated by the rendering back to the controlled end; The controlled end compares the returned hash value with the local hash value. If they do not match, it triggers a retransmission mechanism or switches to the default cursor style.

[0005] In one embodiment, the construction of the dynamic style library includes: The basic style layer stores default cursors categorized by operating system version and uses SVG vector graphics format; The extended style layer dynamically loads application-customized cursors using hash index values; When an unstored style identifier is detected, incremental vector path data is requested only from the control terminal.

[0006] In one embodiment, the vector path data is graphic path coordinates in SVG format, and the size of a single style file is reduced by more than 70% compared to PNG format.

[0007] In one embodiment, data transmission in the compressed channel includes: Keyframe compression technology is used for the animated cursor, transmitting only the first frame and the path coordinate difference frame with a time interval of 10ms; Use a 32-bit floating-point array to store the compressed vector path coordinates; Secure transmission is ensured by a combination of RSA2048 asymmetric encryption and AES256 symmetric encryption.

[0008] In one embodiment, the retransmission mechanism includes: If the hash check fails three times in a row, the default cursor style on the control terminal will be used instead. During retransmission, path data with high-frequency usage patterns predicted by the LSTM neural network are sent first.

[0009] In one embodiment, the interpolation calculation satisfies the formula: The scaled size is calculated as follows: (physical pixel size of the controlled cursor ÷ DPI value of the controlled cursor) × DPI value of the control cursor. Bilinear interpolation algorithm is used to optimize the scaling smoothness.

[0010] In one embodiment, a resource optimization step is also included: The LRU caching strategy is used to keep frequently used patterns in memory; Differential encoding is used to handle color attribute changes, transmitting only the changed RGB values.

[0011] One embodiment of the present invention provides a remote desktop bidirectional mouse style synchronization system for implementing the method described in any of the above embodiments, comprising: The dynamic style library module stores the operating system's default cursor and application-defined cursors. It adopts a layered architecture, where the base layer stores cross-platform compatible cursor files in SVG vector format, and the extension layer associates application-specific cursors through SHA-256 hash indexes. The bidirectional synchronization engine, integrated into the remote desktop protocol stack, is configured to: capture cursor style change events on the controlled end and transmit style identifiers and vector path coordinates through the WebSocket+Protobuf compressed channel; The intelligent rendering module contains a lightweight LSTM model, and the output style uses probability prediction values ​​to feed into the dynamic style library module.

[0012] In one embodiment, the optimization of the dynamic style library module in IoT devices includes: Perform 8-bit integer quantization compression on the vector path coordinates; The LSTM model was parameterized to INT8 precision, with a total parameter size of ≤50KB. In one embodiment, the cross-platform parsing unit parses the SVG vector path and calls the operating system's native graphics interface to render the cursor; The packet loss and retransmission unit locates the lost path coordinate data segment and initiates retransmission when the consecutive packet loss count is ≥3. Preload execution units that respond to style identifiers with a probability value ≥ 80% output by the intelligent rendering module and preload them into the memory buffer.

[0013] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: 1. By using a cross-platform parsing mechanism based on vector graphics format, the cursor style distortion problem between different operating systems (such as Windows / macOS / Linux) is completely eliminated, ensuring that arrows, waiting circles, text input cursors, etc. present a consistent visual effect on all platforms. The synchronization process based on bidirectional hash verification effectively solves the style disorder problem caused by system differences, and users can obtain a native operating experience without manual adaptation.

[0014] 2. Keyframe compression technology is used for animated cursors to significantly reduce the transmission load of dynamic cursors such as progress circles and brush effects, ensuring a high frame rate rendering effect of 60fps, making remote operation and local operation feel consistent. The intelligent preloading mechanism predicts high-frequency styles based on user behavior, eliminating waiting delays when switching cursors, and especially optimizing the smoothness of scenarios where designers and programmers frequently switch cursors.

[0015] 3. The layered dynamic style library design, combined with the on-demand loading strategy, enables the system to occupy only a small amount of storage resources in low-memory devices (such as industrial IoT terminals), solving the device lag problem caused by the large size of the cursor library in traditional solutions. Integer quantization compression technology avoids the consumption of hardware resources by floating-point operations, significantly improving the compatibility of low-computing-power terminals such as embedded devices.

[0016] 4. The differential retransmission mechanism automatically locates lost data segments in response to network packet loss, and only retransmits the necessary path coordinates instead of the complete file, effectively maintaining the stability of style synchronization in high-latency and high-jitter networks. The encrypted transmission channel uses a combination of asymmetric and symmetric encryption to prevent the risk of cursor tampering caused by man-in-the-middle attacks, enhancing the security of enterprise-level remote control scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0018] Figure 1 This is a diagram showing the hierarchical structure of the dynamic style library of the present invention; Figure 2 This is a flowchart of the bidirectional synchronization process of the present invention; Figure 3 This is the anti-packet loss and retransmission control diagram of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] like Figure 1-3 As shown, one embodiment of the present invention provides a method for synchronizing bidirectional mouse styles on a remote desktop based on a dynamic style library, including: Construct a hierarchical dynamic style library, which includes a base style layer that stores the operating system's default cursor and an extended style layer that stores application-defined cursors; When the controlled device captures mouse style change events, it extracts the style identifier, vector path data, and screen DPI parameters. The style identifier, vector path data, and screen DPI parameters are encapsulated into binary data packets and transmitted to the control terminal via a compression channel based on the WebSocket protocol and Protobuf encoding. The control terminal loads the corresponding style from the dynamic style library based on the received style identifier, and then renders the cursor size after interpolating the cursor size based on the control terminal's DPI value. The control end sends the style hash value generated by the rendering back to the controlled end; The controlled end compares the returned hash value with the local hash value. If they do not match, it triggers a retransmission mechanism or switches to the default cursor style.

[0023] In this embodiment of the invention, a two-way hash verification mechanism is used to ensure that the cursor rendering results of the controlled end (such as macOS) and the control end (Windows) are completely consistent, eliminating problems such as arrows turning into squares due to system differences; Trigger a retransmission or switch the default style mechanism to avoid synchronization lag caused by network jitter and maintain operation continuity.

[0024] In one embodiment, the construction of the dynamic style library includes: The basic style layer stores default cursors categorized by operating system version and uses SVG vector graphics format; The extended style layer dynamically loads application-customized cursors using hash index values; When an unstored style identifier is detected, incremental vector path data is requested only from the control terminal.

[0025] In this embodiment of the invention, SVG vector format replaces bitmap to eliminate the jagged edges at high resolution; incremental request mechanism avoids repeated transmission of known styles; and storage is categorized by operating system version to solve compatibility issues between new and old systems such as Windows 11 and macOS Sonoma.

[0026] In one embodiment, the vector path data is graphic path coordinates in SVG format, and the size of a single style file is reduced by more than 70% compared to PNG format.

[0027] In this embodiment of the invention, the volume advantage of SVG path coordinates (more than 70% smaller than PNG) significantly reduces the load of cross-border network transmission, and vector data supports real-time modification of color / transparency, meeting the special cursor requirements of design software (such as semi-transparent brushes).

[0028] In one embodiment, data transmission in the compressed channel includes: Keyframe compression technology is used for the animated cursor, transmitting only the first frame and the path coordinate difference frame with a time interval of 10ms; Use a 32-bit floating-point array to store the compressed vector path coordinates; Secure transmission is ensured by a combination of RSA2048 asymmetric encryption and AES256 symmetric encryption.

[0029] In this embodiment of the invention, the keyframe compression technology reduces the amount of animated cursor transmission to 20% of the traditional solution, achieving smooth rendering at 60fps. The RSA2048 / AES256 combination encryption prevents cursor tampering caused by man-in-the-middle attacks (such as in financial remote operation scenarios).

[0030] In one embodiment, the retransmission mechanism includes: If the hash check fails three times in a row, the default cursor style on the control terminal will be used instead. During retransmission, path data with high-frequency usage patterns predicted by the LSTM neural network are sent first.

[0031] In this embodiment of the invention, the three-failure degradation mechanism avoids infinite retransmission blocking, LSTM prediction prioritizes retransmission of high-frequency patterns (such as the pen cursor commonly used by designers), reduces the synchronization interruption rate, and preloads high-frequency patterns into memory to reduce disk I / O consumption.

[0032] In one embodiment, the interpolation calculation satisfies the formula: The scaled size is calculated as follows: (physical pixel size of the controlled cursor ÷ DPI value of the controlled cursor) × DPI value of the control cursor. Bilinear interpolation algorithm is used to optimize the scaling smoothness.

[0033] In this embodiment of the invention, the cursor scaling imbalance under 4K / 1080P hybrid screens is solved by precise conversion using the above formula, and bilinear interpolation eliminates scaling jagged edges, ensuring a smooth experience for precise operations such as CAD drawing.

[0034] In one embodiment, a resource optimization step is also included: The LRU caching strategy is used to keep frequently used patterns in memory; Differential encoding is used to handle color attribute changes, transmitting only the changed RGB values.

[0035] In this embodiment of the invention, the LRU caching strategy improves the hit rate of high-frequency styles and reduces style loading latency, while the differential encoding technology saves invalid data transmission for color change scenarios (such as cursor hover color change).

[0036] One embodiment of the present invention provides a remote desktop bidirectional mouse style synchronization system for implementing the method described in any of the above embodiments, comprising: The dynamic style library module stores the operating system's default cursor and application-defined cursors. It adopts a layered architecture, where the base layer stores cross-platform compatible cursor files in SVG vector format, and the extension layer associates application-specific cursors through SHA-256 hash indexes. The bidirectional synchronization engine, integrated into the remote desktop protocol stack, is configured to: capture cursor style change events on the controlled end and transmit style identifiers and vector path coordinates through the WebSocket+Protobuf compressed channel; The intelligent rendering module contains a lightweight LSTM model, and the output style uses probability prediction values ​​to feed into the dynamic style library module.

[0037] In this embodiment of the invention, a bidirectional synchronization engine is embedded in the underlying remote desktop protocol to reduce rendering latency, and the LSTM model preloads styles based on user habits (such as programmers switching text cursors more frequently than designers) to achieve "zero-wait" interaction. In one embodiment, the optimization of the dynamic style library module in IoT devices includes: Perform 8-bit integer quantization compression on the vector path coordinates; The LSTM model was parameterized to INT8 precision, with a total parameter size of ≤50KB. In this embodiment of the invention, 8-bit integer quantization makes the single-style memory <5KB (originally 12KB), supports industrial control equipment with RAM=64MB, and the INT8 quantized LSTM model size <50KB (originally 210KB), thus solving the computing power bottleneck of embedded devices.

[0038] In one embodiment, the cross-platform parsing unit parses the SVG vector path and calls the operating system's native graphics interface to render the cursor; The packet loss and retransmission unit locates the lost path coordinate data segment and initiates retransmission when the consecutive packet loss count is ≥3. Preload execution units that respond to style identifiers with a probability value ≥ 80% output by the intelligent rendering module and preload them into the memory buffer.

[0039] In this embodiment of the invention, native rendering across all platforms is achieved by calling the OS graphics interface (Windows DirectX / macOS Metal) to eliminate cross-platform rendering differences. Packet loss resilience: The packet loss location and retransmission mechanism maintains seamless operation even during network fluctuations; Preload precise triggering: A probability threshold of ≥80% ensures the effectiveness of preloading and avoids memory waste.

[0040] It should be noted that the CAD use case is only for understanding the technical effect, and it can also be used for other high-precision operation scenarios, which are explained here.

[0041] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0042] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0043] Finally, it should be noted that the remote desktop bidirectional mouse style synchronization method and system based on a dynamic style library disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for synchronizing bidirectional mouse styles on a remote desktop based on a dynamic style library, characterized in that, include: Construct a hierarchical dynamic style library, which includes a base style layer that stores the operating system's default cursor and an extended style layer that stores application-defined cursors; When the controlled terminal captures a mouse style change event, it extracts the style identifier, vector path data, and screen DPI parameters; the vector path data is SVG format graphic path coordinates, and the size of a single style file is smaller than that of PNG format; The style identifier, vector path data, and screen DPI parameters are encapsulated into binary data packets and transmitted to the control terminal via a compression channel based on the WebSocket protocol and Protobuf encoding. The control terminal loads the corresponding style from the dynamic style library based on the received style identifier, and then renders the cursor after interpolating the cursor size according to the control terminal's DPI value; the construction of the dynamic style library includes: The basic style layer stores default cursors categorized by operating system version and uses SVG vector graphics format; The extended style layer dynamically loads application-customized cursors using hash index values; When an unstored style identifier is detected, only incremental vector path data is requested from the control terminal; The control end sends the style hash value generated by the rendering back to the controlled end; The controlled end compares the returned hash value with the local hash value. If they do not match, it triggers a retransmission mechanism or switches to the default cursor style.

2. The method for synchronizing two-way mouse styles on a remote desktop based on a dynamic style library according to claim 1, characterized in that, The data transmission in the compressed channel includes: Keyframe compression technology is used for the animated cursor, transmitting only the first frame and the path coordinate difference frame with a time interval of 10ms; Use a 32-bit floating-point array to store the compressed vector path coordinates; Secure transmission is ensured by a combination of RSA2048 asymmetric encryption and AES256 symmetric encryption.

3. The method for synchronizing two-way mouse styles on a remote desktop based on a dynamic style library according to claim 1, characterized in that, The retransmission mechanism includes: If the hash check fails three times in a row, the default cursor style on the control terminal will be used instead. During retransmission, path data with high-frequency usage patterns predicted by the LSTM neural network are sent first.

4. The method for synchronizing two-way mouse styles on a remote desktop based on a dynamic style library according to claim 1, characterized in that, The interpolation calculation satisfies the following formula: The scaled size is calculated as follows: (physical pixel size of the controlled cursor ÷ DPI value of the controlled cursor) × DPI value of the control cursor. Bilinear interpolation algorithm is used to optimize the scaling smoothness.

5. The method for synchronizing two-way mouse styles on a remote desktop based on a dynamic style library according to claim 1, characterized in that, It also includes resource optimization steps: The LRU caching strategy is used to keep frequently used patterns in memory; Differential encoding is used to handle color attribute changes, transmitting only the changed RGB values.

6. A remote desktop bidirectional mouse style synchronization system for implementing the method of any one of claims 1-5, characterized in that, include: The dynamic style library module stores the operating system's default cursor and application-defined cursors. It adopts a layered architecture, where the base layer stores cross-platform compatible cursor files in SVG vector format, and the extension layer associates application-specific cursors through SHA-256 hash indexes. The bidirectional synchronization engine, integrated into the remote desktop protocol stack, is configured to: capture cursor style change events on the controlled end and transmit style identifiers and vector path coordinates through the WebSocket+Protobuf compressed channel; The intelligent rendering module contains a lightweight LSTM model, and the output style uses probability prediction values ​​to feed into the dynamic style library module.

7. The system according to claim 6, characterized in that, The optimization of the dynamic style library module in IoT devices includes: Perform 8-bit integer quantization compression on the vector path coordinates; The LSTM model was parameterized to INT8 precision, with a total parameter size of ≤50KB.

8. The system according to claim 7, characterized in that, Also includes: A cross-platform parsing unit parses SVG vector paths and calls the operating system's native graphics interface to render the cursor; The packet loss and retransmission unit locates the lost path coordinate data segment and initiates retransmission when the consecutive packet loss count is ≥3. Preload execution units that respond to style identifiers with a probability value ≥ 80% output by the intelligent rendering module and preload them into the memory buffer.

Images