A method and system for multi-screen cooperative control based on USB

Abstract

The application relates to the technical field of intelligent device interaction, and discloses a multi-screen cooperative control method and system based on USB. The method comprises the following steps: acquiring the task characteristics of each device screen, and analyzing the screen visual composition to determine the task type according to the task characteristics; if the task type is a high real-time category and the transmission link performance is poor, assigning a high priority label to the task and adjusting the transmission resource; monitoring the execution state of the task with the high priority label to determine the supplementary instruction; fusing the supplementary instruction and context awareness data to generate a final cooperative instruction set; and distributing and executing the cooperative instruction set, and feeding back and optimizing by comparing the execution result with the preset task state. The method dynamically perceives the task and adaptively adjusts the instruction, and solves the problem that the efficiency is not high due to the rigidity of the cooperative strategy in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent device interaction and computer technology, and in particular to a multi-screen cooperative control method and system based on USB. BACKGROUND

[0002] At present, multi-screen cooperative control technology is one of the core research directions in the field of modern intelligent device interaction, which can significantly improve the operation efficiency and experience of users in multi-device scenarios. With the popularization of intelligent terminals, users expect to seamlessly switch and synchronize tasks between multiple screens such as mobile phones, tablets, and computers.

[0003] In one prior art, multi-screen cooperative systems mostly rely on preset instruction distribution mechanisms and fixed priority strategies. These systems lack dynamic perception ability of the current task state of each screen, and when the user runs multiple tasks with different properties on different screens at the same time, the system often cannot flexibly adjust the processing order of instructions and resource allocation according to the actual needs of the tasks (such as the high real-time requirement of video conference and the low real-time requirement of document editing). This static instruction processing method often leads to key instruction response delay, reducing the cooperative efficiency between devices and user experience.

[0004] In summary, the prior art has the problem of low multi-screen cooperative control efficiency. SUMMARY

[0005] The present application provides a multi-screen cooperative control method and system based on USB to solve the problem of low multi-screen cooperative control efficiency.

[0006] In a first aspect, to solve the above technical problems, the present application provides a multi-screen cooperative control method based on USB, comprising:

[0007] Obtaining running data and user interaction records of applications on each device screen, as well as delay indicators and load data of transmission links, and analyzing and classifying to obtain task characteristics;

[0008] According to the task characteristics, analyzing the visual composition features of the screen content to determine the specific category of the task type;

[0009] If the task type belongs to the category with high real-time requirement, according to the analysis results of the delay indicators and the load data, a high priority mark is assigned to the task, and the transmission resource allocation is adjusted according to the high priority mark;

[0010] For tasks with the high priority mark, obtain the synchronization signal and feedback response of instruction execution, and analyze the consistency of the synchronization signal and the feedback response to determine the supplementary instruction;

[0011] acquire context-aware data, and fuse the context-aware data to generate an instruction adaptation version according to the supplementary instruction, and adjust the instruction sequence to obtain a final collaborative instruction set;

[0012] distribute and execute the final collaborative instruction set, and obtain an optimized multi-screen collaborative control output by comparing the execution result with a preset task state evaluation to match the degree.

[0013] Preferably, the running data and user interaction records of the applications on the screens of the devices, and the delay indicators and load data of the transmission links are acquired and analyzed and classified to obtain task characteristics, including:

[0014] The running data and user interaction records of the applications on the screens of the devices, and the delay indicators and load data of the transmission links are acquired in real time to form multi-dimensional state data;

[0015] The process occupancy and operation frequency in the multi-dimensional state data are analyzed to determine the running state of the device;

[0016] The delay indicators and the load data are subjected to time series analysis to extract the change trend and fluctuation characteristics;

[0017] The running state, change trend and fluctuation characteristics are comprehensively analyzed to classify the task and determine the task characteristics.

[0018] Preferably, the visual composition characteristics of the screen content are analyzed according to the task characteristics to determine the specific category of the task type, including:

[0019] The edge features and color distribution of the display content on the screens of the devices are analyzed according to the task characteristics to form a visual feature set;

[0020] The dynamic change of the visual feature set in time series is analyzed to obtain task dynamic characteristics;

[0021] The visual feature set and the task dynamic characteristics are combined to classify and identify the task to determine the specific category of the task type.

[0022] Preferably, if the task type belongs to the category with high real-time requirement, a high-priority label is assigned to the task according to the analysis result of the delay indicators and the load data, and the transmission resource allocation is adjusted according to the high-priority label, including:

[0023] The delay indicators and the load data are subjected to statistical processing to calculate the delay mean and load variance for the task whose task type belongs to the category with high real-time requirement;

[0024] if the average delay exceeds a preset delay threshold or the load variance exceeds a preset stability threshold, assigning the task with the high priority mark;

[0025] allocating transmission link resources preferentially for the task with the high priority mark.

[0026] Preferably, for the task with the high priority mark, acquiring a synchronization signal and a feedback response of instruction execution, and analyzing consistency of the synchronization signal and the feedback response to determine a supplementary instruction, comprising:

[0027] acquiring the synchronization signal and the feedback response of instruction execution from a transmission link and parsing to obtain signal integrity and timing characteristics;

[0028] comparing an actual execution priority of an instruction queue with the high priority mark according to the signal integrity and the timing characteristics, and if the actual execution priority is inconsistent with the high priority mark, extracting a deviation feature from the feedback response to determine an abnormal point of an execution state;

[0029] generating a supplementary instruction candidate set containing instruction types and parameters according to the abnormal point of the execution state;

[0030] determining the supplementary instruction from the supplementary instruction candidate set.

[0031] Preferably, acquiring context-aware data, and fusing the context-aware data to generate an instruction adaptation version according to the supplementary instruction, adjusting an instruction sequence to obtain a final collaborative instruction set, comprising:

[0032] acquiring the context-aware data containing user operation habits and device performance differences;

[0033] extracting a user behavior pattern from the context-aware data to generate a reference instruction sequence;

[0034] inserting the supplementary instruction into a preset high priority position of the reference instruction sequence to form an initial instruction sequence;

[0035] adjusting a priority of an instruction in the initial instruction sequence and optimizing instruction conflicts according to the device performance differences to obtain an optimized instruction sequence;

[0036] for a heterogeneous device, acquiring an adaptation parameter to generate a device instruction compatible configuration corresponding to the optimized instruction sequence;

[0037] combining the optimized instruction sequence and the device instruction compatible configuration to generate the final collaborative instruction set.

[0038] Preferably, the process of allocating and executing the final collaborative instruction set, and evaluating the matching degree by comparing the execution results with preset task states to obtain optimized multi-screen collaborative control output, includes:

[0039] The final collaborative instruction set is parsed to generate an instruction allocation table, and control instructions are distributed to the execution modules of each device according to the instruction allocation table.

[0040] Obtain the execution feedback returned by the execution module;

[0041] The execution feedback is compared with the preset task status to obtain the comparison result, and the matching degree is calculated based on the comparison result;

[0042] If the matching degree exceeds the preset matching degree threshold, then optimization parameters are extracted based on the comparison results to generate preliminary collaborative control output;

[0043] Based on the initial collaborative control output, adjust the operating parameters of the execution module;

[0044] The adjusted running status is obtained, and the optimized multi-screen collaborative control output is obtained by comparing the adjusted running status with the task status a second time.

[0045] Secondly, the present invention provides a USB-based multi-screen collaborative control system, comprising:

[0046] The task feature acquisition module is used to acquire the application running data and user interaction records on the screen of each device, as well as the latency indicators and load data of the transmission link, and to analyze and classify them to obtain task features.

[0047] The task type determination module is used to analyze the visual composition features of the screen content based on the task characteristics and determine the specific category of the task type.

[0048] The priority processing module is used to assign a high priority tag to the task based on the analysis results of the latency index and the load data if the task type belongs to the category with high real-time requirements, and to adjust the transmission resource allocation according to the high priority tag.

[0049] The supplementary instruction generation module is used to obtain the synchronization signal and feedback response of instruction execution for tasks with the high priority mark, and analyze the consistency between the synchronization signal and the feedback response to determine the supplementary instruction;

[0050] The collaborative instruction set generation module is used to acquire context-aware data, and based on the supplementary instructions, fuse the context-aware data to generate an instruction adaptation version, and adjust the instruction sequence to obtain the final collaborative instruction set;

[0051] The collaborative execution and optimization module is used to allocate and execute the final collaborative instruction set, and to obtain optimized multi-screen collaborative control output by comparing the execution results with the preset task status to evaluate the matching degree.

[0052] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the USB-based multi-screen collaborative control method described in any one of the above.

[0053] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described USB-based multi-screen collaborative control methods.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] (1) This invention acquires multi-dimensional data such as application operation, user interaction, and transmission links on the screens of various devices in real time, and performs comprehensive analysis and classification to accurately identify the current task characteristics and specific types. This method transforms the traditional static strategy that relies on fixed rules into a dynamic perception process based on real-time data, solving the problem of insufficient decision-making basis caused by the lack of dynamic perception of task status in existing technologies, and providing accurate and reliable data input for subsequent intelligent scheduling.

[0056] (2) This invention establishes a dynamic priority decision and error correction mechanism for high real-time tasks. After identifying high-priority tasks, it not only adjusts transmission resources for them, but also monitors their instruction execution status in real time and actively generates supplementary instructions for error correction when anomalies occur. This mechanism changes the existing technology's fixed priority mode, which cannot cope with sudden execution anomalies, and solves the problems of critical instruction response delay and poor coordination smoothness, significantly improving the system's stability and real-time response capability under high load and complex network environments.

[0057] (3) This invention generates a final collaborative instruction set adapted to heterogeneous devices by integrating context-aware data such as user operating habits and device performance differences, and performs closed-loop optimization through execution feedback. This forms a complete intelligent closed-loop control system from perception and decision-making to execution and optimization, solving the problems of rigid instruction strategies and inability to adapt to the differences of multiple devices in the prior art. Ultimately, it realizes dynamic optimization of task allocation among multi-screen devices, and comprehensively improves collaborative efficiency and user experience. Attached Figure Description

[0058] Figure 1This is a schematic flowchart of a USB-based multi-screen collaborative control method provided in the first embodiment of the present invention;

[0059] Figure 2 This is a schematic diagram of a USB-based multi-screen collaborative control system provided in the second embodiment of the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Reference Figure 1 The first embodiment of the present invention provides a USB-based multi-screen collaborative control method, comprising the following steps:

[0062] S11: Obtain the running data and user interaction records of the applications on the screens of each device, as well as the latency indicators and load data of the transmission link, and analyze and classify them to obtain task characteristics.

[0063] S12, Based on the task characteristics, analyze the visual composition characteristics of the screen content to determine the specific category of the task type;

[0064] S13, if the task type belongs to the category with high real-time requirements, then according to the analysis results of the latency index and the load data, a high priority tag is assigned to the task, and the transmission resource allocation is adjusted according to the high priority tag;

[0065] S14, for the task with the high priority mark, obtain the synchronization signal and feedback response for instruction execution, analyze the consistency between the synchronization signal and the feedback response, and determine the supplementary instruction;

[0066] S15, acquire context-aware data, and generate an instruction adaptation version by fusing the context-aware data according to the supplementary instructions, and adjust the instruction sequence to obtain the final collaborative instruction set;

[0067] S16, allocate and execute the final collaborative instruction set, and evaluate the matching degree by comparing the execution results with the preset task status to obtain optimized multi-screen collaborative control output.

[0068] In step S11, the running data and user interaction records of the applications on each device screen, as well as the latency indicators and load data of the transmission link, are acquired, analyzed, and classified to obtain task characteristics, including:

[0069] Real-time collection of application operation data and user interaction records on the screens of various devices, as well as latency indicators and load data of transmission links, to form multi-dimensional status data;

[0070] The process occupancy rate and operation frequency in the multidimensional status data are analyzed to determine the operating status of the device;

[0071] Time series analysis is performed on the latency index and the load data to extract the changing trends and fluctuation characteristics;

[0072] Based on the aforementioned operating status, changing trends, and fluctuation characteristics, tasks are classified to determine their characteristics.

[0073] In one implementation, monitoring modules deployed at the operating system level of each device collect real-time operational data of the current foreground application and user interaction records. Simultaneously, a network interface monitoring module acquires transmission link performance data during device-to-external network interactions, primarily including latency metrics and load data. These heterogeneous data types are integrated into a timestamped structured data record, namely the multidimensional state data. The multidimensional state data can be in the form of a data vector, with an example structure of: {timestamp, process utilization, operation frequency, latency metric, bandwidth utilization}.

[0074] It should be noted that the processing of the multidimensional state data first involves determining the internal operating state of the device. This embodiment first extracts two key indicators from the multidimensional state data: process utilization rate and operation frequency. The process utilization rate is obtained by querying the device's operating system's process management interface to obtain the percentage of CPU time slices occupied by the target application within a unit time window; the operation frequency is obtained by counting user interaction events within a preset time window. The length of the preset time window (e.g., 60 seconds) is determined based on statistical analysis of a large number of user interaction habits.

[0075] Furthermore, the acquired process utilization rate is compared with a preset CPU threshold (e.g., 70%), and the operation frequency is compared with a preset interaction threshold (e.g., 4 times per minute). If the process utilization rate exceeds the CPU threshold, or the operation frequency exceeds the interaction threshold, the device's operating state is determined to be a high-load interaction state. It should be noted that the preset CPU threshold and interaction threshold are determined based on statistical modeling using historical data. By collecting CPU utilization and operation frequency data from public datasets (such as device usage logs from the ULB Machine Learning Group) and over 1000 devices in a laboratory environment, their probability distribution is analyzed, and the value corresponding to the 95th percentile is selected as the threshold. The effectiveness is verified on the test set using k-fold cross-validation. The threshold set by this method represents the statistical upper limit of normal operating intensity; exceeding this threshold is considered a high-load state. This method is not sensitive to occasional data spikes and can reliably identify changes in device operating state. For example, through statistical analysis, the 95th percentile of CPU utilization was determined to be 70%, and the 95th percentile of operation frequency was 4 times per minute. Therefore, these two values ​​were set as the CPU threshold and interaction threshold, respectively.

[0076] It is worth noting that the link data in the multidimensional state data is processed to extract its dynamic features. This processing is performed within a sliding time window. The size of the time window (e.g., containing the most recent 100 data points) is determined experimentally based on the data acquisition frequency and response speed requirements. The trend of latency index changes is extracted, specifically by applying least squares linear regression analysis to the latency index data points within the window and calculating the slope of the regression line; this slope value serves as the quantification result of the trend. The fluctuation characteristics of load data are extracted, specifically by calculating the standard deviation of the bandwidth occupancy data points within the window; this standard deviation value serves as the quantification result of the fluctuation characteristics.

[0077] In another implementation, the equipment operating status, change trend, and fluctuation characteristics determined in the aforementioned steps are processed through standardized feature engineering and then used as input features to provide a pre-trained decision tree classification model. The output of this model is the task feature.

[0078] It should be noted that before inputting the data into the model, an effective feature vector needs to be constructed. Since the device operating status is a categorical feature, while the trend (slope) and fluctuation feature (standard deviation) are continuous values ​​with different ranges and dimensions, the processing method is as follows: First, the device operating status is numerically encoded (e.g., 1 for high load status, 0 otherwise); second, to eliminate the influence of dimensions, the Z-score standardization method is used to process the trend and fluctuation features so that they follow a distribution with a mean of 0 and a standard deviation of 1.

[0079] It should be noted that the pre-training process of the model is as follows: A training set containing thousands of historical data samples is obtained, where each sample contains a set of input features (running status, trend, fluctuation features) and correct task feature labels (e.g., "high-load real-time interaction under network constraints") labeled by technical experts according to the actual business scenario. This training set is used to train a decision tree classifier. The CART algorithm can be used to construct the decision tree model, with Gini impurity as the splitting criterion, a maximum tree depth of 10, and a minimum number of leaf node samples of 20. Hyperparameters are determined through grid search and 5-fold cross-validation optimization. When the model's classification accuracy reaches a preset standard (e.g., 95%) on an independent validation set, training is complete, and the decision tree classification model is obtained. It is worth noting that the maximum tree depth is not simply preset, but rather selected using grid search combined with k-fold cross-validation to achieve the highest model classification accuracy on the validation set as the final configuration. In practical use, the input features acquired in real time are input from the root node of the decision tree, and the tree is traversed downwards according to the splitting rules of each node until a leaf node is reached. The category represented by the leaf node is the final output task feature.

[0080] It is worth noting that the system uses an initial pre-trained decision tree classification model and has an online update mechanism: after every 1000 new sample data, an incremental learning process is triggered to update the model using the Hoeffding tree algorithm to adapt to the dynamically changing task environment.

[0081] In step S12, based on the task characteristics, the visual composition features of the screen content are analyzed to determine the specific category of the task type, including:

[0082] Based on the task characteristics, analyze the edge features and color distribution of the content displayed on the device screen to form a visual feature set;

[0083] Analyze the dynamic changes of the visual feature set over time to obtain the task's dynamic features;

[0084] By combining the visual feature set and the task dynamic features, the task is classified and identified to determine the specific category of the task type.

[0085] In one implementation, this embodiment triggers and guides subsequent analysis based on the task features obtained in S11. First, the real-time image of the screen is processed to extract static visual features. The specific implementation of this process is guided by the task features, and corresponding preset optimal processing parameters are adopted according to different task features.

[0086] It should be noted that a mapping relationship between task features and image processing parameters is pre-established within the system. The construction process of this mapping relationship is as follows: First, various typical task features (such as "high load" and "low load") in historical data are classified; then, for each type of task feature, through extensive experimental testing, the parameter values ​​that can achieve the best processing effect on the image in the corresponding scene are found (for example, the gradient threshold that can most clearly distinguish between edges and background noise); finally, the category of task feature is bound to its corresponding optimal parameter value and stored in this mapping relationship.

[0087] In one implementation, the system first obtains the current task features, and then retrieves the optimal parameters corresponding to those features from the aforementioned mapping relationship. For example, if the current task features include the description of "high load," the system dynamically sets the gradient threshold used for edge detection to a lower value (such as 0.1) based on the mapping relationship to identify more subtle textures.

[0088] It should be noted that the extraction of edge features can be achieved by applying an edge detection algorithm (such as the Sobel operator) to the screen image. This involves calculating the pixel gradients in the horizontal and vertical directions of the image and identifying pixels with gradient magnitudes higher than the aforementioned dynamically set gradient threshold as edge points. It is also worth noting that the extraction of color distribution can be achieved by calculating the image's color histogram. The color histogram counts the number of pixels within each preset color interval in the image and is itself a color feature vector. The number of preset color intervals is determined based on the color resolution accuracy requirements of the application scenario and computational efficiency considerations. In the RGB color space, the 256 gray levels of each color channel are typically divided into 16 or 32 equally wide intervals.

[0089] In one implementation, the extracted set of edge points is integrated with the color histogram data to form the visual feature set. To address the model bias problem caused by differences in numerical ranges that may result from directly concatenating different features, the specific implementation of this integration is as follows: The set of edge points is statistically analyzed to calculate a series of edge statistical feature values ​​such as total number and density, and a color feature vector representing the color distribution is obtained; Feature scaling technology is used to preprocess the two sets of features to eliminate the dimensional differences between features; a preferred approach is to use min-max normalization to linearly scale all edge statistical feature values ​​and each value in the color histogram to a closed interval of [0,1]; the normalized edge statistical feature vector is then concatenated with the color feature vector to form a combined feature vector with higher dimensions and more comprehensive information.

[0090] It is worth noting that, to further reduce the dimensionality of the feature vectors, eliminate information redundancy, and improve the computational efficiency of subsequent models, Principal Component Analysis (PCA) can optionally be used to reduce the dimensionality of the combined feature vectors. Principal components with a cumulative contribution rate exceeding a preset information retention threshold (e.g., 95%) are selected to form the final visual feature set. It should be noted that the information retention threshold is determined through quantitative experimental analysis. For example, a set of candidate thresholds (e.g., 80%, 85%, 90%, 95%, 99%) is set. On the training set, PCA is applied to reduce the dimensionality at each candidate threshold, and subsequent classification models are trained. The performance of each model on classification accuracy and average time per recognition is evaluated on an independent validation set. Experimental results show that when the information retention threshold is reduced from 99% to 95%, the model's classification accuracy does not change significantly, but the average time decreases by 30%; however, when further reduced to 90%, the classification accuracy declines significantly. Therefore, 95% is selected as the information retention threshold.

[0091] For example, if the screen is currently displaying a text document, its edge features will mainly focus on the outline of the characters' strokes, presenting regular and sharp edges; its color distribution will be highly concentrated in two ranges: background color (such as white) and foreground color (such as black).

[0092] It is worth noting that, in order to distinguish the dynamic nature of the task, the system performs temporal analysis on the continuously acquired visual feature sets to obtain the task's dynamic features. This temporal analysis is performed within a sliding time window, quantifying the degree of dynamics by calculating the change in the visual feature sets between two consecutive frames within the window. The size of the time window (e.g., including the most recent 30 frames) is determined through experimental testing based on the typical rate of change in the task's dynamics; the change is obtained by calculating the Euclidean distance between the visual feature set vectors at two consecutive moments, and the calculated Euclidean distance value is the task's dynamic feature.

[0093] In another implementation, the visual feature set and the task dynamic features are ultimately used as input features and provided to a pre-trained Support Vector Machine (SVM) classification model. The output of this model is the specific category of the task type. The pre-training process of the model is as follows: A training set containing thousands of labeled screen images is obtained, where each image contains the extracted visual feature set, task dynamic features, and the correct task type label (e.g., "video transmission" or "document editing") labeled by technical experts. This training set is used to train an SVM classifier using a radial basis function (RBF) kernel. When the model's classification accuracy reaches a preset standard (e.g., 98%) determined by business requirements on an independent validation set, training is complete, and the SVM classification model is obtained. In practical use, the real-time extracted combined feature vectors are input into the trained SVM model, and the model's output classification result is the specific category of the task type. The preset standard for classification accuracy (e.g., 98%) is determined based on the business scenario's requirements for classification accuracy.

[0094] In step S13, if the task type belongs to the category with high real-time requirements, then based on the analysis results of the latency index and the load data, a high-priority tag is assigned to the task, and the transmission resource allocation is adjusted according to the high-priority tag, including:

[0095] For tasks that fall into the category of high real-time requirements, statistical processing is performed on the latency indicators and the load data to calculate the mean latency and the load variance.

[0096] If the mean latency exceeds a preset latency threshold or the load variance exceeds a preset stability threshold, then the high priority label is assigned to the task.

[0097] For tasks marked with the high priority tag, transmission link resources are allocated preferentially.

[0098] In one implementation, if the task type determined in S12 belongs to a predefined category with high real-time requirements, such as "video transmission" or "online game", then a high-priority tag is assigned to the task based on the analysis results of the latency index and the load data, and the transmission resource allocation is adjusted according to the high-priority tag.

[0099] It should be noted that for tasks with high real-time requirements, the system first extracts the transmission link latency index and load data sequence from the multi-dimensional state data generated in S11, and performs statistical processing on the latency index and load data to calculate the mean latency and load variance within a sliding time window. The size of the sliding time window (e.g., containing the most recent 50 data points) is determined based on experimental testing of historical data.

[0100] Subsequently, the calculated mean latency is compared with a preset latency threshold, and the load variance is compared with a preset stability threshold. The latency threshold and stability threshold are determined based on statistical analysis of network performance data during smooth system operation. Latency metrics and load data are collected from the system within multiple time windows with no lag or latency reports, and the 90th percentile of each value is calculated and set as the threshold.

[0101] For example, through statistical analysis, the determined latency threshold is 25 milliseconds, and the stability threshold is 10 (variance value). If the calculated average latency of a current video transmission task is 30 milliseconds, and the load variance is 8, since its average latency (30ms) exceeds the latency threshold (25ms), satisfying the condition that the average latency exceeds the latency threshold or the load variance exceeds the stability threshold, the system assigns the high-priority tag to this video transmission task.

[0102] It is worth noting that for tasks marked with the high priority tag, the system will immediately allocate resources to them. Specifically, this resource allocation includes: the system increasing the priority of the task's data packets in the network transmission queue, or dynamically increasing the reserved bandwidth quota for it, thereby achieving priority allocation of transmission link resources.

[0103] In step S14, for tasks with the high-priority marker, the synchronization signal and feedback response for instruction execution are obtained, and the consistency between the synchronization signal and the feedback response is analyzed to determine supplementary instructions, including:

[0104] The synchronization signal and feedback response of the instruction execution are obtained from the transmission link and parsed to obtain signal integrity and timing characteristics;

[0105] Based on the signal integrity and timing characteristics, the actual execution priority of the instruction queue is compared with the high priority marker. If the actual execution priority is inconsistent with the high priority marker, deviation characteristics are extracted from the feedback response to determine the abnormal point of the execution state.

[0106] Based on the abnormal points in the execution state, a supplementary instruction candidate set containing instruction types and parameters is generated;

[0107] The supplementary instruction is determined from the supplementary instruction candidate set.

[0108] In one implementation, this step involves real-time monitoring and dynamic error correction of the execution status of tasks assigned high-priority tags in S13 within the instruction queue.

[0109] It should be noted that the system first captures in real time the synchronization signal sent by the control device and the feedback response returned by the target device from the transmission link between devices. Then, the acquired signals are analyzed to obtain the signal integrity and timing characteristics. The signal integrity is quantified by checking the checksum of the data packets to obtain a value reflecting the degree of error or loss during transmission; the timing characteristics are obtained by calculating the time difference between the timestamp of the synchronization signal transmission and the timestamp of the corresponding feedback response received.

[0110] It is worth noting that the system determines whether the actual execution priority of the instruction queue matches the expected high-priority marker based on the parsed signal integrity and timing characteristics. This determination process is performed by a pre-trained support vector machine (SVM) classification model.

[0111] It should be noted that feature engineering is required before inputting features into the SVM model. Since the signal integrity quantization value and the time difference of the temporal feature have different dimensions and numerical ranges, to avoid the model performance being affected by the difference in feature scale, these two input features must first be Z-score normalized, so that their mean is normalized to 0 and their standard deviation is normalized to 1. The two processed feature values ​​constitute a two-dimensional feature vector, which serves as the input to the SVM model.

[0112] The pre-training process of the model is as follows: A training set containing thousands of historical instruction interaction samples is acquired, where each sample contains a two-dimensional feature vector that has undergone the above-mentioned standardization process and a label indicating the actual priority level of the instruction at that time (e.g., "high", "medium", "low") obtained from system log analysis; a support vector machine classifier with a radial basis function (RBF) kernel is used for training. To ensure optimal model performance, its key hyperparameters, namely the penalty coefficient and kernel function parameters, are optimized using a grid search combined with k-fold cross-validation. Training is complete when the model's classification accuracy under optimal hyperparameters reaches a preset standard on an independent validation set. The preset standard (e.g., 99%) is determined according to the reliability requirements (SLA, Service-Level Agreement) of this collaborative control system. During actual operation, the model takes the feature vectors acquired in real time and subjected to the same standardization process as input and outputs an evaluation result of the actual priority of the current instruction. If the evaluation result is "medium" or "low", it is considered inconsistent.

[0113] For example, if a volume adjustment command with a high-priority label has a response time of up to 50 milliseconds, exceeding the preset 20-millisecond high-priority task response threshold, the system determines that its actual execution priority is inconsistent with the high-priority label and extracts the "30-millisecond increase in response time" as a deviation feature from the feedback response. This deviation feature is recorded as an anomaly in the execution state. It should be noted that the high-priority task response threshold is determined based on the technical specifications of real-time tasks and well-known conclusions of Human Factors Engineering. For example, for real-time tasks such as video conferencing, to ensure a synchronized audio-visual user experience, the command response time must be lower than the latency range perceptible to the human ear (typically 20-30 milliseconds). Therefore, this threshold is preset to 20 milliseconds to ensure that the execution of critical commands meets the basic requirements of real-time interaction.

[0114] In another implementation, based on the anomalies in the execution state, the system generates a supplementary set of instruction candidates. This generation process is performed by a pre-trained decision tree classification model.

[0115] It should be noted that the pre-training process of the decision tree classification model is based on a labeled dataset containing a large number of historical outliers and their corresponding best solutions. The construction process of the labeled dataset includes: automatically mining and extracting all execution state outliers and their contextual features from the system's historical operation logs; analyzing subsequent system logs to identify the error correction instructions used to successfully resolve each outlier; and having technical experts review and confirm these automatically matched results to form a highly reliable set of data pairs consisting of outliers and labels of the best error correction instructions, i.e., the labeled dataset. For example, each record in the labeled dataset contains an outlier, i.e., the deviation feature (e.g., "response time increased by 30 milliseconds"), and a label of the most effective error correction instruction (e.g., "reissue instruction"). The C4.5 algorithm is used to construct the decision tree model. This algorithm recursively selects features that provide the maximum information gain to segment the dataset, thereby learning the rules between outliers and solutions. The training process ends when any of the following preset stopping conditions are met, resulting in the decision tree classification model:

[0116] All samples in the current node belong to the same error correction instruction label;

[0117] The number of samples in the current node is less than a preset minimum number of split samples. The minimum number of split samples (e.g., 20) is set to ensure that each split has sufficient statistical significance and to avoid the model generating invalid rules based on a few random samples.

[0118] The decision tree reaches a preset maximum depth, which is set to control the complexity of the model and prevent overfitting caused by excessively long rule chains.

[0119] In one implementation, the currently identified anomaly is taken as input, and the decision tree outputs one or more supplementary instructions and their parameters that are most likely to solve the problem, forming a candidate set. For example, the output candidate set might be: {Instruction 1: [Type: Retransmission, Parameter: Interval 10ms], Instruction 2: [Type: Switch to Backup Link, Parameter: Bandwidth 50Mbps]}. The system selects and determines the final supplementary instruction from the candidate supplementary instructions according to preset queue optimization rules.

[0120] It should be noted that the queue optimization rules are determined based on statistical analysis of historical instruction execution data and system simulation results. Association rule mining algorithms (such as the Apriori algorithm) are used to analyze massive amounts of "anomaly feature vector - successfully resolved instruction" data pairs to uncover high-confidence rule patterns. For example, strong association rules such as "IF (deviation type = response timeout AND network load > 80%) THEN (execute switchover to backup link instruction)" are discovered and stored in the rule base after review by technical experts. It is worth noting that to ensure the efficiency and rationality of decision-making, the rules in the rule base are sorted according to a clear priority. The priority setting mainly follows these principles:

[0121] The success rate priority principle means that rules with a higher probability of successfully solving problems in historical data (i.e., the confidence level of the rule) have a higher priority.

[0122] The principle of low cost priority applies, meaning rules that require less system resources (such as CPU and bandwidth) to execute error correction instructions or have the least potential impact on user experience have higher priority. For example, the cost of "retransmitting instructions" is lower than that of "switching to a backup link".

[0123] When matching the candidate set, the system traverses the rules in descending order of priority and selects the first instruction that meets the conditions as the final supplementary instruction.

[0124] In step S15, context-aware data is acquired, and based on the supplementary instructions, the context-aware data is fused to generate an instruction adaptation version. The instruction sequence is then adjusted to obtain the final collaborative instruction set, including:

[0125] Acquire the context-aware data, which includes differences in user operating habits and device performance;

[0126] Extract user behavior patterns from the context-aware data to generate a baseline instruction sequence;

[0127] The supplementary instruction is inserted into a preset high-priority position in the baseline instruction sequence to form an initial instruction sequence;

[0128] Based on the differences in device performance, the priority of instructions in the initial instruction sequence is adjusted and instruction conflicts are optimized to obtain an optimized instruction sequence;

[0129] For heterogeneous devices, obtain adaptation parameters and generate a device instruction compatible configuration corresponding to the optimized instruction sequence;

[0130] The final cooperative instruction set is generated by combining the optimized instruction sequence and the device instruction compatibility configuration.

[0131] In one implementation, this embodiment combines the error correction instructions determined in S14 with broader context information to generate an optimal final set of cooperative instructions that can be executed by all devices.

[0132] It should be noted that the system first acquires the context-aware data, which includes user operating habits and device performance differences. The user operating habits are obtained through statistical analysis of historical user behavior logs. This statistical analysis specifically includes: filtering relevant historical user behavior logs based on the current task scenario; applying a sequence pattern mining algorithm (such as the PrefixSpan algorithm) to the filtered logs to calculate the instruction sequence with the highest support, and determining this sequence as the user's behavior pattern. The system uses this determined user behavior pattern as the baseline instruction sequence. The device performance differences are obtained by reading and normalizing the device hardware configuration files (such as CPU model, memory size, and GPU performance index).

[0133] For example, if the current task is "document editing", the system discovers through sequence pattern mining that "1. Open file -> 2. Input text -> 3. Save file" is the sequence with the highest support. This sequence is then identified as the user behavior pattern and directly used as the baseline instruction sequence.

[0134] It is worth noting that the system inserts the supplementary instruction determined in S14 into a preset high-priority position in the base instruction sequence to form an initial instruction sequence. The preset high-priority position is determined according to the preemptive scheduling principle of the real-time operating system (RTOS), and is usually the head of the instruction queue to ensure that error correction instructions can be processed immediately.

[0135] In another implementation, the initial instruction sequence is optimized based on the differences in device performance. This optimization process includes two stages: dynamic priority adjustment and resource conflict resolution.

[0136] First, dynamic priority adjustment is performed. A pre-established instruction cost database records the typical resource consumption (such as CPU cycles and memory usage) of each instruction type on standard hardware. The system iterates through each instruction in the initial instruction sequence, queries its cost, and, combined with the performance parameters of the target execution device (obtained from the device configuration file library), calculates the expected resource utilization rate of that instruction on that device. If the expected resource utilization rate exceeds a preset performance threshold (e.g., 80%), the system lowers the execution priority of that instruction in the device's task queue. It should be noted that the performance threshold is based on statistical analysis of the target device's historical performance data, calculating the average resource utilization rate of the device during stable operation. ) and standard deviation ), and set the threshold at The location. According to statistical principles, resource usage exceeding this threshold can be considered a low-probability event, indicating that the equipment is entering a high-load state. For most equipment, this calculated value is around 80%.

[0137] Secondly, resource conflict resolution is performed. The system simulates the execution of the adjusted instruction sequence, marking the critical resources (such as GPU, specific I / O ports) that each instruction needs to exclusively occupy on the timeline. By checking the timeline, if two or more instructions are found to request the same exclusive resource within overlapping time periods, it is determined to be a resource contention conflict. At this time, the system will schedule the lower-priority instructions backward according to their priorities until there are no more resource conflicts on the timeline, ultimately obtaining the optimized instruction sequence.

[0138] It should be noted that for heterogeneous devices, the system will obtain their adaptation parameters (such as instruction format JSON or XML) and generate a version that can be correctly parsed by the target device for each instruction in the optimized instruction sequence. All these adapted instruction versions together constitute the device instruction compatibility configuration.

[0139] It is worth noting that the system combines the optimized instruction sequence (which defines the execution order) with the device instruction compatibility configuration (which defines the instruction content) to generate the final collaborative instruction set that can be executed efficiently in sequence and can be correctly understood by all heterogeneous devices in terms of content.

[0140] In step S16, the final collaborative instruction set is allocated and executed, and the matching degree is evaluated by comparing the execution results with the preset task state to obtain optimized multi-screen collaborative control output, including:

[0141] The final collaborative instruction set is parsed to generate an instruction allocation table, and control instructions are distributed to the execution modules of each device according to the instruction allocation table.

[0142] Obtain the execution feedback returned by the execution module;

[0143] The execution feedback is compared with the preset task status to obtain the comparison result, and the matching degree is calculated based on the comparison result;

[0144] If the matching degree exceeds the preset matching degree threshold, then optimization parameters are extracted based on the comparison results to generate preliminary collaborative control output;

[0145] Based on the initial collaborative control output, adjust the operating parameters of the execution module;

[0146] The adjusted running status is obtained, and the optimized multi-screen collaborative control output is obtained by comparing the adjusted running status with the task status a second time.

[0147] In one implementation, this embodiment sends out the final cooperative instruction set generated in S15 for execution, and performs closed-loop feedback optimization based on the execution results.

[0148] It should be noted that the system first parses the final collaborative instruction set to generate an instruction allocation table. This table is structured data that clearly defines the target device, execution module, instruction content, parameters, and execution order for each instruction. Based on this instruction allocation table, the system distributes control instructions to the specific execution modules (such as audio modules and display modules) of each device via the USB bus.

[0149] It is worth noting that after distributing instructions, the system receives and aggregates the execution feedback returned by each execution module. This execution feedback includes the execution result of the instruction, such as execution time, success or failure status code, etc. The system compares the acquired execution feedback with a preset task status in real time and calculates the matching degree. The preset task status is a set of key performance indicators (KPIs) and their target values ​​predefined for each task type.

[0150] It should be noted that the specific process of comparing and calculating the matching degree is as follows: The system iterates through each KPI defined in the preset task state; finds the actual value corresponding to the current KPI from the execution feedback and compares it with the target value of the KPI; based on the comparison result, a standardized single-item score is calculated. The calculation rule for this single-item score is predefined according to the nature of the KPI, for example:

[0151] For Boolean indicators such as "success rate", if the execution feedback is "success", the score for a single item is 1.0, otherwise it is 0;

[0152] For numerical metrics such as "response time," if the actual value is better than or equal to the target value, the score for that metric is 1.0; if it is worse than the target value, the score decreases according to a preset linear penalty function. For example, the linear penalty function is calculated as follows: a "completely unacceptable" failure point is set for the KPI, where the actual value is worse than the target value; the score starts at 1.0 (at the target value) and linearly decreases to 0 as the actual value approaches this failure point.

[0153] Based on the preset weights for each KPI, the scores of all individual items are weighted and summed to obtain the final total score, which is the matching degree. It should be noted that the preset weights are determined by business requirements and technical experts through simulation experiments, based on the importance of different KPIs to specific task types.

[0154] For example, for the "video transmission" task, its preset task status is {KPI_1:[Name: Response Time, Target: <20ms, Weight: 0.7], KPI_2:[Name: Success Rate, Target: Success, Weight: 0.3]}. If the obtained execution feedback is {Execution Time: 18ms, Status: Success}, the individual score is calculated as follows: a response time of 18ms is better than the target of 20ms, so the score is 1.0; a successful status also earns a score of 1.0. The final matching degree is 1.0, which is 100%.

[0155] In another implementation, if the calculated matching degree is not lower than a preset matching degree threshold, the system initiates an optimization process. This matching degree threshold is a business indicator determined according to the system's preset Service Level Agreement (SLA), rather than a technical statistical value. This threshold (e.g., 90%) defines the minimum performance standard that the system must achieve to ensure service quality and user satisfaction, set according to the minimum acceptable performance specified in the SLA. When the matching degree between the execution result and the preset task state is lower than this threshold, it indicates that the system performance has not met its business objectives, and therefore the optimization process must be initiated.

[0156] It should be noted that the optimization rule base is built based on regression analysis of historical performance data. It stores a series of IF-THEN rules used to map specific comparison results to specific parameter adjustment actions. The specific process of extracting optimization parameters is as follows: the system takes the comparison results as input and performs matching in the optimization rule base.

[0157] For example, the rule base contains a rule: IF(response time comparison result < target value - 2ms) AND (success rate comparison result = success) THEN(output optimization parameter: {target task, bandwidth quota, adjustment value: -5%}). If the current comparison result shows a response time of 18ms, which is 2ms faster than the target value of 20ms, and the execution is successful, then this rule is matched. The system then extracts "reduce the bandwidth quota allocated to this task by 5%" as the optimization parameter and encapsulates it into a preliminary collaborative control output.

[0158] It should be noted that the system adjusts the operating parameters of the corresponding execution modules based on the initial collaborative control output. Subsequently, the system obtains the adjusted operating status and compares it a second time with the preset task status. If the second comparison result shows that the adjusted performance (e.g., the response time becomes 19ms) is still within the range required by the task status, the optimization is confirmed to be effective, and the system outputs an updated parameter configuration as the optimized multi-screen collaborative control output, which is then applied to subsequent collaborative control processes.

[0159] To facilitate understanding of the present invention, some preferred embodiments of the present invention will be described in further detail below.

[0160] In one implementation, the method of the present invention can be applied to a complete, closed-loop intelligent office scenario. Taking a user conducting a video conference on a computer while simultaneously accessing information on a tablet connected via USB as an example, the workflow is as follows:

[0161] Step 1: The system acquires application data (video conferencing software CPU utilization 85%) and transmission link data (latency 30ms) from the computer screen in real time. After analysis and classification, its task characteristics are identified as "high-load real-time interaction". At the same time, it acquires application data from the tablet screen (PDF reader CPU utilization 15%), and its task characteristics are identified as "low-load static".

[0162] Step two: Based on the "high-load real-time interaction" task characteristics of the computer screen, the system further analyzes the visual composition of the screen content (color and edge features change drastically and continuously), and finally determines the task type of the screen as "video transmission".

[0163] Step 3: Since "video transmission" belongs to the category with high real-time requirements, and the analysis found that its latency (30ms) exceeded the preset performance threshold (e.g., 25ms), the system immediately assigned a high priority tag to the task and initially increased its bandwidth allocation on the USB bus based on the tag.

[0164] Step four: When monitoring the execution status of the instruction stream for video transmission tasks with high priority tags, the system detects a timeout in the feedback response of a critical control signaling. The system determines that an inconsistency in the execution status has occurred and generates a supplementary instruction to "retransmit the critical signaling" based on the pre-trained decision tree model.

[0165] Step five: The system acquires context-aware data (e.g., it learns that the computer has superior performance while the tablet is an older model) and inserts a supplementary instruction to "retransmit critical signaling" at the head of the instruction queue. Simultaneously, based on the device performance difference, the priority of synchronization instructions sent to the tablet is slightly reduced, generating the complete final set of collaborative instructions.

[0166] Step six: The system issues and executes the final set of collaborative instructions. After the instructions are executed, feedback shows that the key signaling response time of the video conference has returned to 18ms. The system confirms that the execution result meets the preset task status requirements by comparison, and records the successful parameter adjustment as the optimized multi-screen collaborative control output for subsequent dynamic adjustments.

[0167] In another implementation, the USB connection method between devices in this invention can be configured differently. The connection can be a high-speed wired direct connection via USB-C or Thunderbolt. The wired method offers the highest bandwidth and lowest latency, suitable for scenarios with extremely high collaborative performance requirements, such as real-time video editing collaboration between a computer and a professional display. Alternatively, the connection can also be made through a docking station supporting wireless USB technology or Wi-Fi Display technology. The wireless method offers greater deployment flexibility, suitable for mobile office or home entertainment scenarios, ensuring that users can freely move devices within a certain range while maintaining stable multi-screen collaboration. The method of this invention can adapt to the different link characteristics under these two connection methods and dynamically adjust its collaborative control strategy.

[0168] Those skilled in the art should understand that the algorithm parameters involved in the above embodiments (such as the C value, γ value, and decision tree depth of SVM) can be determined through the default settings of publicly available toolkits (such as Scikit-learn) or grid search optimization, and do not affect the implementation of the present invention. The threshold values ​​can be recalibrated according to the actual application scenario using the above statistical methods, and still fall within the protection scope of the present invention.

[0169] In summary, this invention discloses a USB-based multi-screen collaborative control method, comprising: acquiring application running data and user interaction records, as well as latency indicators and load data of transmission links on each device screen, and analyzing and classifying them to obtain task characteristics; analyzing the visual composition features of screen content based on the task characteristics to determine the specific category of the task type; if the task type belongs to a category with high real-time requirements, assigning a high-priority tag to the task based on the analysis results of the latency indicators and load data, and adjusting the transmission resource allocation based on the high-priority tag; for tasks with the high-priority tag, acquiring the synchronization signal and feedback response of instruction execution, and analyzing the consistency between the synchronization signal and the feedback response to determine supplementary instructions; acquiring context-aware data, and generating an instruction adaptation version by fusing the context-aware data based on the supplementary instructions, and adjusting the instruction sequence to obtain the final collaborative instruction set; allocating and executing the final collaborative instruction set, and evaluating the matching degree by comparing the execution results with the preset task state to obtain optimized multi-screen collaborative control output. This invention solves the problem of low collaboration efficiency caused by rigid collaboration strategies and lack of awareness of task status in the prior art by dynamically sensing and identifying the task types of each screen and adaptively adjusting instruction priority and resource allocation.

[0170] Reference Figure 2 The second embodiment of the present invention provides a USB-based multi-screen collaborative control system, comprising:

[0171] The task feature acquisition module is used to acquire the application running data and user interaction records on the screen of each device, as well as the latency indicators and load data of the transmission link, and to analyze and classify them to obtain task features.

[0172] The task type determination module is used to analyze the visual composition features of the screen content based on the task characteristics and determine the specific category of the task type.

[0173] The priority processing module is used to assign a high priority tag to the task based on the analysis results of the latency index and the load data if the task type belongs to the category with high real-time requirements, and to adjust the transmission resource allocation according to the high priority tag.

[0174] The supplementary instruction generation module is used to obtain the synchronization signal and feedback response of instruction execution for tasks with the high priority mark, and analyze the consistency between the synchronization signal and the feedback response to determine the supplementary instruction;

[0175] The collaborative instruction set generation module is used to acquire context-aware data, and based on the supplementary instructions, fuse the context-aware data to generate an instruction adaptation version, and adjust the instruction sequence to obtain the final collaborative instruction set;

[0176] The collaborative execution and optimization module is used to allocate and execute the final collaborative instruction set, and to obtain optimized multi-screen collaborative control output by comparing the execution results with the preset task status to evaluate the matching degree.

[0177] It should be noted that the USB-based multi-screen collaborative control system provided in this embodiment of the invention is used to execute all the process steps of the USB-based multi-screen collaborative control method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0178] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a USB-based multi-screen collaboration control program. When the processor executes the computer program, it implements the steps described in the various USB-based multi-screen collaboration control method embodiments above, for example... Figure 1 The step S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, such as the task feature acquisition module.

[0179] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0180] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0181] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0182] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0183] Wherein, if the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0184] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0185] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A multi-screen collaborative control method based on USB, characterized in that, include: The system acquires and analyzes the application's running data and user interaction records on each device's screen, as well as the latency and load data of the transmission link, to obtain task characteristics. Based on the task characteristics, analyze the visual composition features of the screen content to determine the specific category of the task type; If the task type belongs to the category with high real-time requirements, then based on the analysis results of the latency index and the load data, a high-priority tag is assigned to the task, and the transmission resource allocation is adjusted according to the high-priority tag; For tasks marked with the high priority tag, obtain the synchronization signal and feedback response for instruction execution, analyze the consistency between the synchronization signal and the feedback response, and determine supplementary instructions; Acquire context-aware data, and based on the supplementary instructions, fuse the context-aware data to generate an instruction adaptation version, and adjust the instruction sequence to obtain the final collaborative instruction set; The final collaborative instruction set is allocated and executed, and the matching degree is evaluated by comparing the execution results with the preset task status to obtain optimized multi-screen collaborative control output; The step of analyzing the visual composition features of the screen content based on the task characteristics to determine the specific category of the task type includes: Based on the task characteristics, analyze the edge features and color distribution of the content displayed on the device screen to form a visual feature set; Analyze the dynamic changes of the visual feature set over time to obtain the task's dynamic features; By combining the visual feature set and the task dynamic features, the task is classified and identified to determine the specific category of the task type; Specifically, for tasks with the high-priority marker, obtaining the synchronization signal and feedback response for instruction execution, analyzing the consistency between the synchronization signal and the feedback response, and determining supplementary instructions includes: The synchronization signal and feedback response of the instruction execution are obtained from the transmission link and parsed to obtain signal integrity and timing characteristics; Based on the signal integrity and timing characteristics, the actual execution priority of the instruction queue is compared with the high priority marker. If the actual execution priority is inconsistent with the high priority marker, deviation characteristics are extracted from the feedback response to determine the abnormal point of the execution state. Based on the abnormal points in the execution state, a supplementary instruction candidate set containing instruction types and parameters is generated; The supplementary instruction is determined from the candidate set of supplementary instructions; The step of acquiring context-aware data, generating an instruction adaptation version by fusing the context-aware data according to the supplementary instructions, and adjusting the instruction sequence to obtain the final collaborative instruction set includes: Acquire the context-aware data, which includes differences in user operating habits and device performance; Extract user behavior patterns from the context-aware data to generate a baseline instruction sequence; The supplementary instruction is inserted into a preset high-priority position in the baseline instruction sequence to form an initial instruction sequence; Based on the differences in device performance, the priority of instructions in the initial instruction sequence is adjusted and instruction conflicts are optimized to obtain an optimized instruction sequence; For heterogeneous devices, obtain adaptation parameters and generate a device instruction compatible configuration corresponding to the optimized instruction sequence; The final cooperative instruction set is generated by combining the optimized instruction sequence and the device instruction compatibility configuration.

2. The USB-based multi-screen collaborative control method according to claim 1, characterized in that, The process involves acquiring and analyzing the application's runtime data and user interaction records on each device screen, as well as the latency and load data of the transmission link, to obtain task characteristics, including: Real-time collection of application operation data and user interaction records on the screens of various devices, as well as latency indicators and load data of transmission links, to form multi-dimensional status data; The process occupancy rate and operation frequency in the multidimensional status data are analyzed to determine the operating status of the device; Time series analysis is performed on the latency index and the load data to extract the changing trends and fluctuation characteristics; Based on the aforementioned operating status, changing trends, and fluctuation characteristics, tasks are classified to determine their characteristics.

3. The USB-based multi-screen collaborative control method according to claim 1, characterized in that, If the task type belongs to the category with high real-time requirements, then based on the analysis results of the latency index and the load data, a high-priority tag is assigned to the task, and the transmission resource allocation is adjusted according to the high-priority tag, including: For tasks that fall into the category of high real-time requirements, statistical processing is performed on the latency indicators and the load data to calculate the mean latency and the load variance. If the mean latency exceeds a preset latency threshold or the load variance exceeds a preset stability threshold, then the high priority label is assigned to the task. For tasks marked with the high priority tag, transmission link resources are allocated preferentially.

4. The USB-based multi-screen collaborative control method according to claim 1, characterized in that, The process of allocating and executing the final collaborative instruction set, and evaluating the matching degree by comparing the execution results with preset task states to obtain optimized multi-screen collaborative control output, includes: The final collaborative instruction set is parsed to generate an instruction allocation table, and control instructions are distributed to the execution modules of each device according to the instruction allocation table. Obtain the execution feedback returned by the execution module; The execution feedback is compared with the preset task status to obtain the comparison result, and the matching degree is calculated based on the comparison result; If the matching degree exceeds the preset matching degree threshold, then optimization parameters are extracted based on the comparison results to generate preliminary collaborative control output; Based on the initial collaborative control output, adjust the operating parameters of the execution module; The adjusted running status is obtained, and the optimized multi-screen collaborative control output is obtained by comparing the adjusted running status with the task status a second time.

5. A USB-based multi-screen collaborative control system, characterized in that, To implement the USB-based multi-screen collaborative control method as described in any one of claims 1 to 4, the method includes: The task feature acquisition module is used to acquire the running data and user interaction records of the application on the screen of each device, as well as the latency indicators and load data of the transmission link, and to analyze and classify them to obtain task features. The task type determination module is used to analyze the visual composition features of the screen content based on the task characteristics and determine the specific category of the task type. The priority processing module is used to assign a high priority tag to the task based on the analysis results of the latency index and the load data if the task type belongs to the category with high real-time requirements, and to adjust the transmission resource allocation according to the high priority tag. The supplementary instruction generation module is used to obtain the synchronization signal and feedback response of instruction execution for tasks with the high priority mark, and analyze the consistency between the synchronization signal and the feedback response to determine the supplementary instruction; The collaborative instruction set generation module is used to acquire context-aware data, and based on the supplementary instructions, fuse the context-aware data to generate an instruction adaptation version, and adjust the instruction sequence to obtain the final collaborative instruction set; The collaborative execution and optimization module is used to allocate and execute the final collaborative instruction set, and to obtain optimized multi-screen collaborative control output by comparing the execution results with the preset task status to evaluate the matching degree.

Images