Method and system for remotely controlling multi-screen computer

Abstract

The invention relates to the technical field of multi-screen computers, in particular to a method and system for remotely controlling a multi-screen computer, and the method comprises the following steps: S1, operation environment quantization modeling: collecting physical parameters and environment noise data of multi-screen equipment in real time, and outputting an environment feature vector set; s2, operation intention prediction and error pre-compensation: predicting a user operation track, calculating a quantization error accumulation value on a cross-screen path, and generating an optimization operation instruction including a pre-compensation coordinate offset; and S3, dynamic resource cooperative execution: according to the optimization operation instruction, pre-loading operation area resources in a video memory of a target screen, synchronously adjusting a network transmission bandwidth allocation strategy, and executing control operation subjected to error compensation. According to the method, high-precision prediction, low-delay response and error self-adaptive compensation of remote control in a multi-screen heterogeneous environment are realized, and the accuracy of cross-screen operation and the system stability are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-screen computers, and in particular to a method and system for remotely controlling a multi-screen computer. Background Art

[0002] With the rapid development of application scenarios such as remote office, industrial monitoring, intelligent interaction and distributed collaboration, remote control of multi-screen computer systems has become a key technical means to improve operational efficiency and user experience. Especially in the context of widespread deployment of high-resolution multi-screen terminals, users need to achieve seamless switching and precise control between different screens during remote control to meet the actual needs of cross-screen data interaction, task allocation and interface synchronization. Therefore, building a remote control method that supports cross-screen intelligent prediction and precise control has become a research hotspot in the field of human-computer interaction systems.

[0003] Existing multi-screen remote control methods generally rely on static screen mapping rules and fixed coordinate conversion methods, making it difficult to perceive dynamic factors such as physical differences, environmental interference, and network jitter in real time across multiple screen devices. This leads to problems such as decreased accuracy, response delays, or target misalignment in operation trajectories in cross-screen scenarios. In addition, although some solutions introduce image stream compression or frame synchronization mechanisms to improve transmission efficiency, they often lack dynamic prediction mechanisms and error pre-compensation strategies for user intentions, and are unable to achieve high-precision, low-error operation control under complex heterogeneous screen layouts. In particular, when the operation process involves multi-device switching, heterogeneous resolution transitions, or weak network transmission, operation reliability and system adaptability are difficult to guarantee. Summary of the Invention

[0004] The present invention provides a method and system for remotely controlling a multi-screen computer, which realizes video memory preloading, bandwidth optimization and precise instruction placement, thereby effectively improving the operational accuracy, stability and user experience of the multi-screen remote control system in complex environments, and meeting the high-performance human-computer interaction requirements in multiple terminals and multiple scenarios.

[0005] A method for remotely controlling a multi-screen computer comprises the following steps: S1, Quantum modeling of the operating environment: This involves collecting physical parameters and environmental noise data of multiple screen devices in real time, building an environmental quantum model that includes screen physical coordinates, pixel density, and network jitter coefficient, and outputting a set of environmental feature vectors. S2, Operation Intention Prediction and Error Precompensation: Based on the set of environmental feature vectors, a spatiotemporal convolutional neural network is used to predict the user's operation trajectory, calculate the cumulative value of the quantized error on the cross-screen path, and generate optimized operation instructions including pre-compensated coordinate offsets; S3, dynamic resource collaborative execution: According to the optimized operation instructions, the operation area resources are preloaded in the video memory of the target screen, the network transmission bandwidth allocation strategy is adjusted synchronously, and the error-compensated control operation is executed.

[0006] Optionally, the quantized modeling of the operating environment in S1 includes: S11, real-time collection of multi-screen device parameters: real-time collection of the physical parameters of all connected screens, including screen size, resolution, pixel density (PPI), arrangement and relative physical coordinate position; S12, Environmental Noise and Network Status Perception: Utilizes environmental sensors to obtain current network transmission status parameters (jitter, packet loss rate, latency) and interference factors in the surrounding environment that affect control responses (light changes, electromagnetic interference), generating environmental noise data. S13, environmental quantum model construction and feature vector generation: normalize and quantize the collected physical parameters and environmental noise data, build a unified environmental quantum model, and output a set of environmental feature vectors.

[0007] Optionally, the real-time collection of multi-screen device parameters in S11 includes: S111, screen connection status scanning: scanning all currently connected display devices in real time through the provided display device interface or graphics card driver to obtain their unique identifiers, connection port types, and online status information; S112, basic parameter call and extraction: call the system API or graphics interface to obtain the basic physical parameters of each screen, including resolution, screen size, refresh rate and pixel density (PPI); S113, arrangement and coordinate mapping analysis: Based on the recorded display topology, analyze the arrangement (such as horizontal splicing, vertical stacking) and relative physical coordinates of each screen in a multi-screen environment, and calculate the relative position information of each screen in the overall display area through the boundary alignment method.

[0008] Optionally, the environmental noise and network status perception in S12 includes: S121, network transmission status parameter collection: call the network diagnosis module in real time through the network interface to obtain the average jitter in the current network transmission process , packet loss rate , average delay ; S122, interference factor detection: collecting interference factor data through environmental sensors, including light intensity fluctuation rate and electromagnetic interference index; S123, environmental noise index vector generation: normalize the network transmission status parameters and interference factors to form environmental noise data .

[0009] Optionally, the environmental quantum model construction and feature vector generation in S13 include: S131, parameter normalization processing: linear normalization is performed on the collected screen physical parameters and environmental noise data respectively; S132, parameter quantization encoding: mapping the linearly normalized screen physical parameters and environmental noise data to a discrete interval set for quantization processing; S133, Environmental feature vector construction: All quantized screen physical parameters and environmental noise data are combined to form a unified environmental feature vector set .

[0010] Optionally, the operation intention prediction and error pre-compensation in S2 include: S21, constructing operation trajectory input features: jointly encode the original operation signal input by the user (mouse movement trajectory, touch point sequence) and the environmental feature vector set to construct an operation-environment fusion feature sequence; S22, predicting operation intentions and operation instructions: Introducing a spatiotemporal convolutional neural network (ST-CNN) model to jointly model the spatial position and temporal evolution of the operation-environment fusion feature sequence, outputting the predicted results of the user's operation trajectory, and combining the pixel quantization granularity of each screen to calculate the cumulative error caused by coordinate conversion and display differences during the cross-screen process through a discrete deviation function, and outputting optimized control instructions including pre-compensated coordinate offsets.

[0011] Optionally, constructing the operation trajectory input feature in S21 includes: S211, operation signal time series encoding: convert the original operation signal input by the user, including the mouse movement trajectory or touch point sequence, into a two-dimensional position vector set in the form of a time series. ; S212, time alignment and expansion of environmental features: Time alignment processing is performed on the set of environmental feature vectors, and the static vector is copied and expanded in the time dimension to a time synchronization matrix with the same length as the operation sequence. ; S213, Joint Feature Splicing Coding: Splice the two-dimensional position vector set and the time synchronization matrix frame by frame to construct the operation-environment fusion feature sequence .

[0012] Optionally, the predicted operation intention and operation instruction in S22 include: S221, Spatiotemporal Convolutional Neural Network Trajectory Prediction Modeling: Fusion of Operation-Environment Feature Sequences The input is fed into the spatiotemporal convolutional neural network (ST-CNN) model, which extracts the dynamic evolution pattern through one-dimensional temporal convolution and combines it with two-dimensional spatial convolution to encode the trajectory position information and output the operation prediction trajectory. S222, screen pixel quantization error modeling: Based on the pixel quantization granularity of the predicted trajectory point and its corresponding target screen, a discrete deviation function is used to calculate the cumulative error caused by the display parameter difference. ; S223, generating an optimization control instruction: based on the calculated cumulative error and offset trends, generating pre-compensated coordinate offsets 、 Optimized control instructions .

[0013] Optionally, the dynamic resource collaborative execution in S3 includes: S31, pre-fetching and loading resources in the operation area: Based on the target coordinates and pre-compensated coordinate offsets included in the optimization control instruction, the screen area that the user is about to operate is predicted, and relevant graphics resources and interface components, including high-frequency interactive elements, regional cache tiles, and cursor layers, are pre-loaded into the video memory corresponding to the target screen; S32, dynamic allocation and scheduling of network bandwidth: Based on the resource size of the operating area, data transmission priority and current network status, a scheduling algorithm based on priority queue and token bucket mechanism is used to dynamically adjust the bandwidth allocation strategy between the master terminal and the target screen; S33, control instruction execution: sending the optimized control instruction to the target screen execution end, triggering the remote operation behavior matching the predicted position, and performing coordinate correction based on the pre-compensated coordinate offset.

[0014] A system for remotely controlling a multi-screen computer, for implementing the aforementioned method for remotely controlling a multi-screen computer, comprises the following modules: Environmental modeling module: This module collects physical parameters and environmental noise data of multiple screen devices in real time, constructs an environmental quantum model including screen physical coordinates, pixel density, and network jitter coefficient, and outputs a set of environmental feature vectors. Operation prediction and compensation module: Based on the set of environmental feature vectors, it uses a spatiotemporal convolutional neural network to predict the user's operation trajectory, calculate the cumulative value of the quantized error on the cross-screen path, and generate optimized operation instructions including pre-compensated coordinate offsets; Resource collaborative execution module: According to the optimized operation instructions, the operation area resources are preloaded in the video memory of the target screen, and the network transmission bandwidth allocation strategy is adjusted synchronously to execute the control operation after error compensation.

[0015] Beneficial effects of the present invention: The present invention introduces a quantum modeling mechanism for the operating environment, comprehensively collects the physical parameters and environmental interference data of multi-screen devices, constructs a unified environmental quantum model, and generates a set of environmental feature vectors in a normalized and quantized manner. This mechanism enables the system to fully perceive the device status and communication quality in a multi-screen heterogeneous environment, thereby improving the adaptability and robustness of the remote control system to complex environmental changes.

[0016] The present invention jointly models the operation trajectory and environmental characteristics by adopting a spatiotemporal convolutional neural network, extracts the dynamic characteristics of user behavior through one-dimensional time convolution, and combines the multi-screen coordinate mapping relationship of two-dimensional spatial convolution encoding to achieve accurate prediction of user operation intentions. The system further combines the pixel quantization granularity of each screen, models and accumulates cross-screen errors based on a discrete deviation function, and generates optimized control instructions including pre-compensation offsets, reducing the risk of trajectory offset and target point misalignment in remote control, especially in complex scenarios such as multi-screen splicing and mixing of different resolutions, showing strong stability and accuracy.

[0017] The present invention proactively pre-fetches the area resources to be operated in the video memory of the target screen in combination with the prediction results, and adopts the priority queue and token bucket mechanism to dynamically allocate network bandwidth, giving priority to the data transmission requirements of the low-latency control channel, and effectively avoiding the interference of resource loading on the operation response. Through the precise control of instruction execution and offset correction mechanism, the system can stably complete operation synchronization under cross-network and multi-terminal conditions, thereby improving the real-time, interactivity and user experience of the remote control system in application scenarios such as industrial monitoring, remote office, and virtual collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic flow chart of a method for controlling a multi-screen computer according to an embodiment of the present invention; Figure 2 Schematic diagram of system function modules according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0021] like Figure 1 As shown, a method for remotely controlling a multi-screen computer includes the following steps: S1, Quantum modeling of the operating environment: This involves collecting physical parameters and environmental noise data of multiple screen devices in real time, building an environmental quantum model that includes screen physical coordinates, pixel density, and network jitter coefficient, and outputting a set of environmental feature vectors. S2, Operation Intention Prediction and Error Precompensation: Based on the set of environmental feature vectors, a spatiotemporal convolutional neural network is used to predict the user's operation trajectory, calculate the cumulative value of the quantized error on the cross-screen path, and generate optimized operation instructions including pre-compensated coordinate offsets; S3, dynamic resource collaborative execution: According to the optimized operation instructions, the operation area resources are preloaded in the video memory of the target screen, the network transmission bandwidth allocation strategy is adjusted synchronously, and the error-compensated control operation is executed.

[0022] The quantized modeling of the operating environment in S1 includes: S11, real-time collection of multi-screen device parameters: real-time collection of the physical parameters of all connected screens, including screen size, resolution, pixel density (PPI), arrangement and relative physical coordinate position; S12, Environmental Noise and Network Status Perception: Utilizes environmental sensors to obtain current network transmission status parameters (jitter, packet loss rate, latency) and interference factors in the surrounding environment that affect control responses (light changes, electromagnetic interference), generating environmental noise data. S13, environmental quantum model construction and feature vector generation: normalize and quantize the collected physical parameters and environmental noise data, build a unified environmental quantum model, and output a set of environmental feature vectors.

[0023] Real-time collection of multi-screen device parameters in S11 includes: S111, screen connection status scanning: scanning all currently connected display devices in real time through the provided display device interface or graphics card driver to obtain their unique identifiers, connection port types, and online status information; S112, basic parameter call and extraction: call the system API or graphics interface to obtain the basic physical parameters of each screen, including resolution, screen size, refresh rate and pixel density (PPI); S113, Arrangement and Coordinate Mapping Analysis: Based on the recorded display topology, analyze the arrangement of each screen in a multi-screen environment (such as horizontal splicing, vertical stacking) and relative physical coordinates. Calculate the relative position information of each screen in the overall display area using the boundary alignment method, expressed as: Horizontal arrangement (right alignment): If the coordinates of the upper left corner of the i-th screen are , with a width of , then the position of the i+1th screen is: ; Vertical arrangement (bottom alignment): ; in, is the height of the i-th screen; Mixed arrangement (grid type): If the starting reference point is known , then the i-th screen (in the r-th row and c-th column) is expressed as: ; in, 、 The screen width and height of the front column and row respectively.

[0024] Environmental noise and network status awareness in S12 include: S121, network transmission status parameter collection: call the network diagnosis module in real time through the network interface to obtain the average jitter in the current network transmission process , packet loss rate , average delay , expressed as: ; ; ; in, is the transmission delay of the i-th data packet, is the transmission delay of the i-th data packet, N is the number of sampled data packets, is the number of packets lost in the time window, is the total number of data packets sent within the time window; S122, interference factor detection: collecting interference factor data through environmental sensors, including light intensity fluctuation rate and electromagnetic interference index, where; Light intensity fluctuation rate By calculating the rate of change of the light sensor reading per unit time, it can be expressed as: ; in, is the jth illumination sampling value, M is the number of sampling times, is the j+1th illumination sampling value; Electromagnetic interference index The average field strength per unit time is obtained through the integrated electromagnetic noise sensing module; S123, environmental noise index vector generation: normalize the network transmission status parameters and interference factors to form environmental noise data , expressed as: ; in, are the normalized network transmission state parameters and interference factors respectively.

[0025] The construction of the environmental quantum model and the generation of feature vectors in S13 include: S131, parameter normalization processing: linear normalization is performed on the collected screen physical parameters and environmental noise data, respectively, which can be expressed as: ; in, is the normalized value, is the kth original parameter value, 、 The minimum and maximum values of the parameter; S132, parameter quantization encoding: Map the linearly normalized screen physical parameters and environmental noise data to a discrete interval set for quantization processing, which is expressed as: ; in, is the quantization result, Q is the quantum precision level (Q=256 means 8-bit quantization); S133, Environmental feature vector construction: All quantized screen physical parameters and environmental noise data are combined to form a unified environmental feature vector set , expressed as: ; Where K is the total number of parameters.

[0026] Operation intention prediction and error pre-compensation in S2 include: S21, constructing operation trajectory input features: jointly encode the original operation signal input by the user (mouse movement trajectory, touch point sequence) and the environmental feature vector set to construct an operation-environment fusion feature sequence; S22, predicting operation intentions and operation instructions: Introducing a spatiotemporal convolutional neural network (ST-CNN) model to jointly model the spatial position and temporal evolution of the operation-environment fusion feature sequence, outputting the predicted results of the user's operation trajectory, and combining the pixel quantization granularity of each screen to calculate the cumulative error caused by coordinate conversion and display differences during the cross-screen process through a discrete deviation function, and outputting optimized control instructions including pre-compensated coordinate offsets.

[0027] The operation trajectory input feature construction in S21 includes: S211, operation signal time series encoding: convert the original operation signal input by the user, including the mouse movement trajectory or touch point sequence, into a two-dimensional position vector set in the form of a time series. , expressed as: ; in, is the screen operation coordinate collected at the t-th frame, where T is the total number of frames; S212, time alignment and expansion of environmental features: Time alignment processing is performed on the set of environmental feature vectors, and the static vector is copied and expanded in the time dimension to a time synchronization matrix with the same length as the operation sequence. , expressed as: ; in, is the length of the operation signal time series; S213, Joint Feature Splicing Coding: Splice the two-dimensional position vector set and the time synchronization matrix frame by frame to construct the operation-environment fusion feature sequence , expressed as: ; in, for The replicated values of each dimension feature in the t-th frame.

[0028] The predicted operation intention and operation instructions in S22 include: S221, Spatiotemporal Convolutional Neural Network Trajectory Prediction Modeling: Fusion of Operation-Environment Feature Sequences The input is fed into the spatiotemporal convolutional neural network (ST-CNN) model, which extracts the dynamic evolution pattern through one-dimensional temporal convolution and combines it with two-dimensional spatial convolution to encode the trajectory position information, outputting the predicted operation trajectory. Specifically, it includes: (1) Temporal convolution modeling: Assume The input of the layer temporal convolution is , the convolution kernel is , the output is: ; in, is the time step, is the weight vector of the convolution kernel at the jth position, is the bias term, is the ReLU activation function, is the temporal convolution kernel size; (2) Spatial coordinate embedding and convolution modeling: the position dimension in the temporal convolution output Mapped into a two-dimensional spatial feature map, and then two-dimensional convolution is performed. Let the embedded spatial feature map be , then the spatial convolution operation is expressed as: ; in, is the spatial coordinate index, is the spatial convolution kernel, 、 are the sizes of the convolution kernel in height and width respectively, is bias; (3) Trajectory point prediction output: The output of the last layer of ST-CNN is mapped into a predicted coordinate sequence through a fully connected regression layer, which is expressed as: ; in, is the final spatiotemporal eigenvector at the sth moment, is the weight of the fully connected layer, is the bias term, is the coordinate of the s-th prediction point, S is the prediction time step; S222, screen pixel quantization error modeling: Based on the pixel quantization granularity of the predicted trajectory point and its corresponding target screen, a discrete deviation function is used to calculate the cumulative error caused by the display parameter difference. , expressed as: ; in, 、 is the sth ideal operation coordinate, 、 The unit pixel granularity of the screen where the prediction point is located; S223, generating an optimization control instruction: based on the calculated cumulative error and offset trends, generating pre-compensated coordinate offsets 、 Optimized control instructions , expressed as: ; ; ; Dynamic resource coordination in S3 includes: S31, pre-fetching and loading resources in the operation area: Based on the target coordinates and pre-compensated coordinate offsets included in the optimization control instruction, the screen area that the user is about to operate is predicted, and relevant graphics resources and interface components, including high-frequency interactive elements, regional cache tiles, and cursor layers, are loaded into the video memory corresponding to the target screen in advance, as shown in the following example: ; ; in, , is the final trajectory point predicted by ST-CNN, , is the target operation coordinate after compensation, , Prefetch boundary range for resources, The screen area where the user is about to operate; S32, dynamic allocation and scheduling of network bandwidth: Based on the resource size of the operating area, data transmission priority, and current network status, a scheduling algorithm based on priority queues and token bucket mechanisms is used to dynamically adjust the bandwidth allocation strategy between the master terminal and the target screen, giving priority to ensuring low-latency interactive data channels and avoiding interference with the control flow during the resource loading process. It is expressed as: ; in, is the allocation rate of the i-th channel at time t, is the current number of available tokens (initialized by bandwidth level), For the refresh cycle, The maximum bandwidth limit (device configuration constraint); ; in, is the channel priority weight, is the resource usage priority, is the center coordinate of the resource block, is the Euclidean distance function, , is the priority adjustment coefficient; S33, control instruction execution: sending the optimized control instruction to the target screen execution end, triggering the remote operation behavior matching the predicted position, and performing coordinate correction based on the pre-compensated coordinate offset.

[0029] like Figure 2 As shown, a system for remotely controlling a multi-screen computer is used to implement the above-mentioned method for remotely controlling a multi-screen computer, including the following modules: Environmental modeling module: This module collects physical parameters and environmental noise data of multiple screen devices in real time, constructs an environmental quantum model including screen physical coordinates, pixel density, and network jitter coefficient, and outputs a set of environmental feature vectors. Operation prediction and compensation module: Based on the set of environmental feature vectors, it uses a spatiotemporal convolutional neural network to predict the user's operation trajectory, calculate the cumulative value of the quantized error on the cross-screen path, and generate optimized operation instructions including pre-compensated coordinate offsets; Resource collaborative execution module: According to the optimized operation instructions, the operation area resources are preloaded in the video memory of the target screen, and the network transmission bandwidth allocation strategy is adjusted synchronously to execute the control operation after error compensation.

[0030] The present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention. To provide a thorough understanding of the present invention, specific details are described in detail below in connection with the preferred embodiments of the present invention, but those skilled in the art will be able to fully understand the present invention without these detailed descriptions. Furthermore, to avoid unnecessary confusion regarding the essence of the present invention, well-known methods, processes, procedures, components, and circuits have not been described in detail.

[0031] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for remotely controlling a multi-screen computer, characterized in that: The following steps are involved: S1, Quantum modeling of the operating environment: This involves collecting physical parameters and environmental noise data of multiple screen devices in real time, building an environmental quantum model that includes screen physical coordinates, pixel density, and network jitter coefficient, and outputting a set of environmental feature vectors. S2, Operation Intention Prediction and Error Precompensation: Based on the set of environmental feature vectors, a spatiotemporal convolutional neural network is used to predict the user's operation trajectory, calculate the cumulative value of the quantized error on the cross-screen path, and generate optimized operation instructions including pre-compensated coordinate offsets; S3, dynamic resource collaborative execution: According to the optimized operation instructions, the operation area resources are preloaded in the video memory of the target screen, the network transmission bandwidth allocation strategy is adjusted synchronously, and the error-compensated control operation is executed.

2. A method for remotely controlling a multi-screen computer according to claim 1, characterized in that: The quantized modeling of the operating environment in S1 includes: S11, real-time acquisition of multi-screen device parameters: real-time acquisition of the physical parameters of all connected screens, including screen size, resolution, pixel density, arrangement, and relative physical coordinate position; S12, Environmental Noise and Network Status Perception: Utilizes environmental sensors to obtain current network transmission status parameters and interference factors in the surrounding environment that affect control responses, generating environmental noise data. S13, environmental quantum model construction and feature vector generation: normalize and quantize the collected physical parameters and environmental noise data, build a unified environmental quantum model, and output a set of environmental feature vectors.

3. A method for remotely controlling a multi-screen computer according to claim 2, characterized in that: The real-time acquisition of multi-screen device parameters in S11 includes: S111, screen connection status scanning: scanning all currently connected display devices in real time through the provided display device interface or graphics card driver to obtain their unique identifiers, connection port types, and online status information; S112, basic parameter call and extraction: call the system API or graphics interface to obtain the basic physical parameters of each screen, including resolution, screen size, refresh rate and pixel density; S113, arrangement and coordinate mapping analysis: Based on the recorded display topology, analyze the arrangement and relative physical coordinates of each screen in the multi-screen environment, and calculate the relative position information of each screen in the overall display area through the boundary alignment method.

4. A method for remotely controlling a multi-screen computer according to claim 3, characterized in that: The environmental noise and network status perception in S12 includes: S121, network transmission status parameter collection: call the network diagnosis module in real time through the network interface to obtain the average jitter in the current network transmission process , packet loss rate , average delay ; S122, interference factor detection: collecting interference factor data through environmental sensors, including light intensity fluctuation rate and electromagnetic interference index; S123, environmental noise index vector generation: normalize the network transmission status parameters and interference factors to form environmental noise data .

5. The method for remotely controlling a multi-screen computer according to claim 4, wherein: The environmental quantum model construction and feature vector generation in S13 include: S131, parameter normalization processing: linear normalization is performed on the collected screen physical parameters and environmental noise data respectively; S132, parameter quantization encoding: mapping the linearly normalized screen physical parameters and environmental noise data to a discrete interval set for quantization processing; S133, Environmental feature vector construction: All quantized screen physical parameters and environmental noise data are combined to form a unified environmental feature vector set .

6. The method for remotely controlling a multi-screen computer according to claim 1, wherein: The operation intention prediction and error pre-compensation in S2 include: S21, constructing operation trajectory input features: jointly encoding the original operation signal input by the user and the set of environmental feature vectors to construct an operation-environment fusion feature sequence; S22, predicting operation intentions and operation instructions: Introducing a spatiotemporal convolutional neural network model, jointly modeling the spatial position and temporal evolution of the operation-environment fusion feature sequence, outputting the predicted results of the user's operation trajectory, and combining the pixel quantization granularity of each screen. The cumulative error caused by coordinate conversion and display differences during the cross-screen process is calculated through a discrete deviation function, and the output includes optimized control instructions including pre-compensation of coordinate offsets.

7. A method for remotely controlling a multi-screen computer according to claim 6, characterized in that: The operation trajectory input feature construction in S21 includes: S211, operation signal time series encoding: convert the original operation signal input by the user, including the mouse movement trajectory or touch point sequence, into a two-dimensional position vector set in the form of a time series. ; S212, time alignment and expansion of environmental features: Time alignment processing is performed on the set of environmental feature vectors, and the static vector is copied and expanded in the time dimension to a time synchronization matrix with the same length as the operation sequence. ; S213, Joint Feature Splicing Coding: Splice the two-dimensional position vector set and the time synchronization matrix frame by frame to construct the operation-environment fusion feature sequence .

8. The method for remotely controlling a multi-screen computer according to claim 7, wherein: The predicted operation intention and operation instruction in S22 include: S221, Spatiotemporal Convolutional Neural Network Trajectory Prediction Modeling: Fusion of Operation-Environment Feature Sequences The input is fed into the spatiotemporal convolutional neural network model, which extracts the dynamic evolution pattern through one-dimensional temporal convolution and combines it with two-dimensional spatial convolution to encode the trajectory position information and output the operation prediction trajectory. S222, screen pixel quantization error modeling: Based on the pixel quantization granularity of the predicted trajectory point and its corresponding target screen, a discrete deviation function is used to calculate the cumulative error caused by the display parameter difference. ; S223, generating an optimization control instruction: based on the calculated cumulative error and offset trends, generating pre-compensated coordinate offsets 、 Optimized control instructions .

9. The method for remotely controlling a multi-screen computer according to claim 8, wherein: The dynamic resource collaborative execution in S3 includes: S31, pre-fetching and loading resources in the operation area: Based on the target coordinates and pre-compensated coordinate offsets included in the optimization control instruction, the screen area that the user is about to operate is predicted, and relevant graphics resources and interface components, including high-frequency interactive elements, regional cache tiles, and cursor layers, are pre-loaded into the video memory corresponding to the target screen; S32, dynamic allocation and scheduling of network bandwidth: Based on the resource size of the operating area, data transmission priority and current network status, a scheduling algorithm based on priority queue and token bucket mechanism is used to dynamically adjust the bandwidth allocation strategy between the master terminal and the target screen; S33, control instruction execution: sending the optimized control instruction to the target screen execution end, triggering the remote operation behavior matching the predicted position, and performing coordinate correction based on the pre-compensated coordinate offset.

10. A system for remotely controlling a multi-screen computer, for implementing a method for remotely controlling a multi-screen computer according to any one of claims 1 to 9, characterized in that: Includes the following modules: Environmental modeling module: This module collects physical parameters and environmental noise data of multiple screen devices in real time, constructs an environmental quantum model including screen physical coordinates, pixel density, and network jitter coefficient, and outputs a set of environmental feature vectors. Operation prediction and compensation module: Based on the set of environmental feature vectors, it uses a spatiotemporal convolutional neural network to predict the user's operation trajectory, calculate the cumulative value of the quantized error on the cross-screen path, and generate optimized operation instructions including pre-compensated coordinate offsets; Resource collaborative execution module: According to the optimized operation instructions, the operation area resources are preloaded in the video memory of the target screen, and the network transmission bandwidth allocation strategy is adjusted synchronously to execute the control operation after error compensation.

Images