Interaction control method of double-screen organic display system

Abstract

The invention relates to the technical field of data processing of organic displays, in particular to an interaction control method of a double-screen organic display system, which acquires touch control, deformation and ambient light data in real time through a multi-mode sensor array, and realizes cross-screen heterogeneous data space-time alignment by adopting an improved clock synchronization protocol and a sliding window mechanism. And generating a time sequence feature vector set with weight. And based on an LSTM-Transform hybrid model, fusing historical communication delay data, analyzing space-time relevance by using a multi-head attention mechanism, and outputting a cross-screen communication delay probability distribution diagram. And generating a multi-target task allocation strategy in combination with a deep reinforcement learning model, and iteratively calculating a shortest path allocation scheme of a rendering task through an improved ant colony optimization algorithm. The space-time synchronization precision of double-screen interaction is effectively improved, signal misalignment caused by communication delay is reduced, the distribution efficiency of distributed rendering resources is optimized, and the environmental adaptability and operation robustness of a flexible display system are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of organic display data processing, and particularly to an interaction control method for a dual-screen organic display system. Background Art

[0002] The dual-screen organic display system is a multi-screen interaction solution based on flexible organic light-emitting diode technology. Its core consists of two independent or integrated organic light-emitting display panels, and realizes the coordination of physical form and function through a high-precision optical bonding process. This system uses a low-temperature polysilicon thin-film transistor backplane to drive the organic light-emitting layer, and combines an ultra-thin encapsulation technology to inhibit the penetration of water and oxygen, so as to achieve the reliability of the display unit in a bent or folded state. The dual-screen architecture supports split-screen content mapping, cross-screen interaction and dynamic information linkage, and optimizes the multi-channel signal synchronization efficiency through a distributed image processing algorithm.

[0003] The dual-screen organic display system faces technical challenges in real-time collaborative scheduling of multi-source heterogeneous data streams at the data processing level, which is mainly reflected in the timing deviation between cross-screen interaction instructions and dynamic content rendering under a distributed architecture. Due to the physical separation characteristics and independent driving mechanisms of the dual screens, the communication delay between distributed image processing nodes is likely to cause multi-channel signal synchronization misalignment, resulting in pixel-level misalignment during the split-screen content mapping process or cross-screen interaction logic faults. It is necessary to reconstruct the data synchronization strategy through a dynamic compensation mechanism to maintain interaction consistency. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides an interaction control method for a dual-screen organic display system, which solves the problem of multi-channel signal synchronization misalignment caused by the timing deviation between cross-screen interaction instructions and dynamic content rendering in the dual-screen organic display system, so as to maintain cross-screen interaction consistency.

[0005] To solve the above technical problems, the specific technical solutions of the present invention are as follows: The interaction control method for a dual-screen organic display system provided by the present invention includes: Real-time collecting touch coordinates, deformation curvature and ambient light intensity data through a multi-modal sensor array deployed at the edge nodes of the dual screens, performing cross-screen timing alignment processing on the touch coordinates, deformation curvature and ambient light intensity data, and generating a weighted timing feature vector set; Inputting the timing feature vector set and cross-screen communication delay historical data into an LSTM-Transformer hybrid model, parsing the spatio-temporal correlation through a multi-head attention mechanism, and outputting a cross-screen communication delay probability distribution map for future multi-frame periods; According to the cross-screen communication delay probability distribution map, real-time rendering queue status and cross-screen interaction instructions, a deep reinforcement learning model is used to generate a dynamic task allocation strategy, and the shortest path allocation scheme for multi-channel rendering tasks is iteratively calculated based on an improved ant colony optimization algorithm; Inputting the cross-screen communication delay probability distribution map and the original frame sequence with timing deviation into a conditional generative adversarial network to generate a cross-screen pixel compensation frame, wherein the conditional generative adversarial network optimizes adversarial training parameters through Wasserstein distance; Input the system buffer queue depth, synchronization error and the compensation effect of the cross-screen pixel compensation frame into the model-independent meta-learning framework, and dynamically adjust the learning rate of the LSTM-Transformer hybrid model, the adversarial loss weight of the conditional generative adversarial network and the exploration rate parameter of the deep reinforcement learning model; The real-time data of the multimodal sensor array and the prediction results of the LSTM-Transformer hybrid model are fused through a dual-channel Kalman filter to generate an error correction signal, which is fed back to the input end of the cross-screen timing alignment processing to adjust the sensor sampling frequency, and trigger the federated learning framework to perform global aggregation updates on the weights of the LSTM-Transformer hybrid model, forming a closed-loop control link of data acquisition, prediction compensation and parameter optimization.

[0006] Furthermore, in the interactive control method of the dual-screen organic display system of the present invention, the real-time acquisition of cross-screen timing alignment and feature extraction includes: An improved clock synchronization protocol is used to mark multi-source sensor data with a unified timestamp to generate a sensor data stream with synchronized timestamps. Inputting the timestamp-synchronized sensor data stream into a sliding window mechanism for buffer alignment, removing abnormal data according to a preset mapping rule between touch points and screen areas, and generating a spatiotemporally aligned intermediate data set; The time-space aligned intermediate data set is input into a lightweight convolutional network to extract touch trajectory continuity features and deformation dynamic response features, and output a weighted temporal feature vector set.

[0007] Furthermore, in the interactive control method of the dual-screen organic display system of the present invention, the step of constructing a hybrid prediction model to generate a delay probability distribution comprises: Input the weighted time series feature vector set and the historical data of cross-screen communication delay into the LSTM-Transformer hybrid model, parse the spatiotemporal correlation through the multi-head attention mechanism of the LSTM-Transformer hybrid model, and output a probability distribution map of cross-screen communication delay in future multi-frame periods; Based on the sliding window data stream, an online incremental learning strategy is adopted to dynamically update the network weights of the LSTM-Transformer hybrid model to adapt to the dynamic fluctuations of communication delays.

[0008] Further, in the interactive control method of the dual-screen organic display system of the present invention, the dynamic allocation of multi-channel rendering tasks includes: Taking the cross-screen communication delay probability distribution map, the real-time rendering queue status, and the cross-screen interaction instruction as inputs, a multi-objective deep reinforcement learning model is constructed to generate a dynamic task allocation strategy; Based on the screen space topological relationship and real-time load data, an improved ant colony optimization algorithm is used to iteratively calculate the shortest path allocation scheme of the multi-channel rendering tasks in the dynamic task allocation strategy.

[0009] Further, in the interactive control method of the dual-screen organic display system of the present invention, the execution of cross-screen pixel compensation includes: Inputting the cross-screen communication delay probability distribution map and the original frame sequence with timing deviation into a conditional generative adversarial network to generate cross-screen pixel compensation frames; Using the Wasserstein distance to optimize the adversarial training process of the conditional generative adversarial network, and compressing the network scale of the generator in the conditional generative adversarial network through knowledge distillation technology to adapt to the computing power limitation of the edge computing node.

[0010] Further, in the interactive control method of the dual-screen organic display system of the present invention, the dynamic collaborative optimization control parameters include: Mapping the buffer queue depth of the task allocation strategy, the synchronization error of the cross-screen pixel compensation frames, and the system power consumption index into a high-dimensional parameter space, and inputting them into a model-agnostic meta-learning framework to generate an optimal control strategy; Based on the Bayesian optimization algorithm, dynamically adjust the learning rate of the LSTM-Transformer hybrid model, the adversarial loss weight of the conditional generative adversarial network, and the exploration rate parameter of the deep reinforcement learning model.

[0011] Further, in the interactive control method of the dual-screen organic display system of the present invention, the generation of the error correction signal includes: Inputting the real-time data of the multi-modal sensor array and the prediction result of the LSTM-Transformer hybrid model into a two-channel Kalman filter to generate an error correction signal and feedback it to the delay prediction input end of the LSTM-Transformer hybrid model; When a communication interruption is detected, start the federated learning framework to aggregate the gradients of the dual-screen local models and reconstruct the global consistent network weights of the LSTM-Transformer hybrid model.

[0012] Furthermore, for the interactive control method of the dual-screen organic display system of the present invention, the reverse adjustment data acquisition strategy includes: Dynamically adjust the sampling frequency of the multi-modal sensor array and the feature weights of the lightweight convolutional network according to the error correction signal; Inject the adjusted sensor sampling parameters into the LSTM-Transformer hybrid model, and synchronize the updated feature weights to the conditional generative adversarial network and the deep reinforcement learning model to form a cross-model parameter collaborative update link.

[0013] Furthermore, for the interactive control method of the dual-screen organic display system of the present invention, the exception recovery mechanism includes: Re-initialize the network weights of the LSTM-Transformer hybrid model based on the model gradient aggregation result of the federated learning framework; Trigger the policy reset of the deep reinforcement learning model and the reconstruction of the adversarial loss function of the conditional generative adversarial network according to the real-time system state.

[0014] Furthermore, for the interactive control method of the dual-screen organic display system of the present invention, the data flow of the closed-loop control link includes: the error correction signal triggers the update of the data acquisition strategy of the multi-modal sensor array, and the updated sensor data is input into the LSTM-Transformer hybrid model; The optimized control parameters synchronously adjust the task allocation strategy of the deep reinforcement learning model and the pixel compensation parameters of the conditional generative adversarial network; The compensated frame data stream and the system state are fed back to the model-agnostic meta-learning framework to form a closed-loop link for cross-level parameter optimization and the exception recovery mechanism.

[0015] Advantages of the present invention; The interactive control method of the dual-screen organic display system provided by the present invention significantly improves the timing consistency and signal synchronization accuracy of dual-screen interaction through the technical collaboration of multi-modal data fusion, cross-level prediction compensation, and dynamic closed-loop optimization. Based on the improved clock synchronization protocol and sliding window mechanism, spatio-temporal alignment of cross-screen heterogeneous data is achieved. Combining the LSTM-Transformer hybrid model to analyze the spatio-temporal correlation between touch trajectories and deformation responses, accurately predict the characteristics of communication delay distribution, and provide a quantitative benchmark for task allocation and pixel compensation; the hierarchical decision-making mechanism of deep reinforcement learning and improved ant colony optimization dynamically adapts the rendering resource allocation path, reducing communication redundancy and computational conflicts between distributed nodes; the conditional generative adversarial network generates spatio-temporally consistent compensation frames based on the delay probability distribution, and combines knowledge distillation technology to achieve efficient inference on the edge side, effectively correcting pixel-level misalignments; the model-agnostic meta-learning framework drives the cross-model collaborative optimization of the learning rate of the prediction model, the adversarial loss weight, and the exploration rate of reinforcement learning. Combining the closed-loop feedback mechanism of federated learning and dual-channel Kalman filtering, dynamic adaptation of sensor parameters, model weights, and system strategies is realized. Through the full-link closed-loop control of data acquisition - prediction - compensation - optimization, this solution enables the dual-screen system to adapt to environmental interference and hardware drift, maintain spatio-temporal synchronization of cross-screen interaction while reducing communication energy consumption, and improve the interaction experience and system robustness in flexible display scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the present invention, the drawings required for the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the drawings.

[0017] Figure 1 It is a flowchart of the interactive control method of the dual-screen organic display system provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments and corresponding drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention. The following will describe in detail the technical solutions provided by each embodiment of the present invention in conjunction with the drawings. To better understand the objectives of the present invention, the present invention will be further described in detail below.

[0019] Please refer to Figure 1 , the interactive control method of the dual-screen organic display system provided by the present invention includes: Step S101: Real-time collect touch coordinates, deformation curvature, and ambient light intensity data through a multi-modal sensor array deployed at the dual-screen edge node, perform cross-screen temporal alignment processing on the touch coordinates, deformation curvature, and ambient light intensity data, and generate a weighted temporal feature vector set; In the interaction control method of the dual-screen organic display system of the present invention, the technical solution of step S101 realizes cross-screen heterogeneous data standardization through multi-level data processing. The multi-modal sensor array is deployed at the dual-screen edge node. Among them, the capacitive touch sensor captures touch coordinates in a matrix scanning manner, the flexible strain sensor measures the screen deformation curvature in real time through piezoelectric sensing units, and the ambient light sensor collects light intensity data based on a photodiode array. The improved clock synchronization protocol constructs a nanosecond-level clock synchronization link based on the physical separation characteristics of the master and slave screens, marks the multi-source sensor data with a unified timestamp, and eliminates the reference deviation caused by the dual-screen independent clock sources. The data stream after timestamp synchronization is input into the sliding window buffer alignment mechanism. The window size is dynamically adjusted according to the dual-screen interaction frequency. Combining the geometric mapping rule between the touch point and the screen display area, the out-of-bounds coordinate data is eliminated. The synchronization adopts a deformation curvature mutation detection algorithm to filter mechanical noise and generate a spatio-temporally aligned intermediate data set.

[0020] The lightweight convolutional network uses a depthwise separable convolutional architecture to process the intermediate data set. The shallow convolutional kernel extracts the local spatio-temporal features of the touch trajectory, and the deep network captures the global correlation pattern of the deformation response by dilated convolution to expand the receptive field. The network output layer integrates a channel attention mechanism, dynamically assigns weight coefficients according to the contribution of the features to the latency prediction, and generates a weighted temporal feature vector set. The weight assignment strategy is generated based on historical latency data, optimizes the feature sensitivity through backpropagation. The high-weight features characterize the coupling relationship between the touch intention intensity and the deformation response. The normalization module eliminates the dimensional differences of the sensors, and the interpolation compensation mechanism fills the temporal gaps caused by the elimination of abnormal data. The output vector set has spatio-temporal continuity and provides a standardized input for downstream latency prediction.

[0021] The sliding window mechanism adopts a dynamic capacity control strategy. When a high-frequency touch event is detected, the buffer capacity is expanded to improve the alignment accuracy, and the window is shrunk in the low-interaction state to reduce the computational overhead. The abnormal data elimination module integrates a touch trajectory coherence verification algorithm, judges the validity of the touch point through the coordinate displacement threshold of adjacent timestamps, and combines the change gradient of the deformation curvature to detect the difference between physical deformation and human touch. The data alignment status markers are synchronously recorded during the generation process of the intermediate data set, including the number of valid touch points, the deformation response amplitude, and the light intensity fluctuation range, providing metadata support for the compensation strategy in subsequent steps. The preprocessed feature vector set is transmitted to the edge computing node through the distributed data bus, completing the conversion of cross-screen heterogeneous data into standardized features.

[0022] Step S102: Input the temporal feature vector set and the cross-screen communication delay historical data into the LSTM-Transformer hybrid model, parse the spatio-temporal correlation through the multi-head attention mechanism, and output the cross-screen communication delay probability distribution map for multiple future frame periods. In the interactive control method of the dual-screen organic display system of the present invention, the technical solution of step S102 realizes accurate prediction of communication delay through spatio-temporal feature fusion and dynamic model optimization. The temporal feature vector set and the cross-screen communication delay historical data are input into the LSTM-Transformer hybrid model after being aligned by the feature concatenation module. Among them, the LSTM network layer captures the temporal dependence of the touch trajectory, and the Transformer encoder analyzes the spatial correlation between the touch event and the screen deformation through the multi-head attention mechanism, and outputs the spatio-temporal coupling feature vector. The multi-head attention layer calculates the cross-correlation weights of the touch coordinates, deformation curvature, and historical delay data, and filters out the key spatio-temporal features sensitive to delay prediction.

[0023] The delay probability distribution generation module constructs a probability map network based on the spatio-temporal coupling features. The fully connected layer maps the high-dimensional features to the multi-dimensional probability space and outputs the cross-screen communication delay probability distribution map for multiple future frame periods. This distribution map contains the confidence interval and peak distribution characteristics of delay events within each time window, and quantitatively characterizes the stability fluctuation of the communication link. The online incremental learning strategy dynamically updates the model weights using the sliding window data flow, fuses the new data and the historical gradient direction through the gradient memory replay mechanism, and triggers the reinforcement learning mode when the detected delay standard deviation exceeds the threshold, accelerating the model's adaptability to sudden delays.

[0024] The dynamic learning rate decay strategy is adopted in the model weight update process, and the parameter update step size is adjusted according to the communication delay fluctuation amplitude. The learning rate is increased in the high-fluctuation state to quickly converge, and the learning rate is reduced in the low-fluctuation stage to improve the prediction stability. The updated network weights are synchronized to the dual-screen edge nodes through the distributed parameter server. The model snapshot mechanism regularly saves the stage training results to prevent the loss of the model state caused by communication interruption. The weight change trajectory generated by incremental learning is input into the meta-learning framework to provide prior knowledge for cross-model parameter collaborative optimization, forming a closed-loop learning link from feature extraction, delay prediction to parameter update.

[0025] Step S103: According to the cross-screen communication delay probability distribution map, the real-time rendering queue status, and the cross-screen interaction instructions, use the deep reinforcement learning model to generate a dynamic task allocation strategy, and iteratively calculate the shortest path allocation scheme for multi-channel rendering tasks based on the improved ant colony optimization algorithm. In the interactive control method of the dual-screen organic display system described in the present invention, the technical solution of step S103 realizes the efficient scheduling of rendering tasks through hierarchical decision-making and path optimization. The deep reinforcement learning model takes the cross-screen communication delay probability distribution map, the real-time rendering queue status, and the cross-screen interaction instructions as inputs, constructs a multi-dimensional state space including the depth of the task buffer, the node load rate, and the user operation priority, and defines the action space as the assignable rendering channel combination strategy. The multi-objective reward function comprehensively considers the synchronization accuracy deviation, response delay, and power consumption metrics, and updates the policy network parameters through the temporal difference learning algorithm to generate a dynamic task allocation strategy. This strategy preferentially allocates the rendering tasks in high-delay-risk areas to low-load nodes and reserves redundant computing resources to cope with sudden delay fluctuations.

[0026] The improved ant colony optimization algorithm performs task path allocation at the physical layer, constructs a node connection graph based on the screen space topology relationship, and associates the node weights with the real-time load data and the communication link quality. In the initialization stage of the algorithm, the task allocation strategy output by the reinforcement learning model is transformed into a prior pheromone distribution to guide the search direction of the ant colony. During the iterative process, a dynamic evaporation factor is introduced to adaptively adjust the pheromone retention ratio according to the congestion degree of the rendering queue, balancing the global exploration and local exploitation capabilities. The path evaluation function combines the peak delay risk in the delay probability distribution map and the node computing margin to screen the shortest path scheme that meets the synchronization accuracy constraint.

[0027] The task allocation strategy and the path optimization result form a closed-loop feedback mechanism. The path allocation data generated in each round of iteration is input into the reinforcement learning model to update the synchronization accuracy evaluation parameter in the reward function and optimize the subsequent policy generation direction. The real-time load data is collected through the distributed monitoring module to dynamically correct the screen space topology weights, and preferentially allocate the rendering tasks in high-fluctuation areas to stable communication links. The task execution results are recorded in the rendering queue status database, triggering the parameter tuning process of the model-agnostic meta-learning framework, forming a complete decision-making link from policy generation, path optimization to effect feedback. The optimized allocation scheme is executed through the parallel rendering engine of the edge computing node, reducing the computing redundancy and communication conflicts of the distributed nodes.

[0028] Step S104, input the cross-screen communication delay probability distribution map and the original frame sequence with timing deviation into the conditional generative adversarial network to generate a cross-screen pixel compensation frame, where the conditional generative adversarial network optimizes the adversarial training parameters through the Wasserstein distance; In the interactive control method of the dual-screen organic display system described in the present invention, the technical solution of step S104 realizes pixel-level timing deviation correction through a generative adversarial network and a dynamic compensation mechanism. The generator of the conditional generative adversarial network receives the original frame sequence with timing deviation and the cross-screen communication delay probability distribution map, constructs an encoding and decoding structure using the U-Net architecture. The encoder extracts the spatio-temporal features of the original frame through multi-scale convolution, and the decoder combines the delay probability distribution information to generate a pixel compensation vector, which is superimposed on the original frame through residual connection to output a compensated frame. The discriminator is designed based on the dual-screen display consistency constraint, analyzes the pixel distribution difference between the compensated frame and the ideal synchronous frame through spatio-temporal convolutional layers, and generates an adversarial loss signal to guide the update of the generator parameters.

[0029] In the adversarial training optimization stage, the Wasserstein distance is used as the loss metric benchmark, and the training stability is enhanced through the gradient penalty strategy. The generator loss function combines the pixel reconstruction error and the adversarial loss, and the discriminator loss function introduces the conditional constraint of the delay probability distribution map to strengthen the optimization of the compensation accuracy in high-delay risk areas. The training data stream integrates a sliding window mechanism to dynamically load the timing deviation samples in real-time rendering tasks, constructs an enhanced data set in combination with historical delay scenarios, and improves the generalization ability of the model to complex delay patterns.

[0030] In the model inference stage, knowledge distillation technology is used to compress the scale of the generator network. The teacher network retains the complete U-Net structure to output a high-precision compensated frame, and the student network reconstructs the encoding and decoding modules through depthwise separable convolution, reducing the number of parameters while maintaining the compensation effect. The distillation process adopts a feature map alignment strategy to make the intermediate layer feature distribution of the student network consistent with the corresponding layer of the teacher network, and constrains the knowledge transfer process through the mean square error loss. The compressed generator optimizes the computational graph structure through operator fusion technology, combines the convolutional layer and the activation function into a single computational unit, and adapts to the parallel computing architecture of edge computing nodes.

[0031] The generation result of the compensated frame is transmitted to the rendering queue for execution in real-time, and the pixel difference data between the compensated frame and the original frame is recorded synchronously. The difference data includes the spatio-temporal domain error distribution and the edge sharpness loss index, which are fed back to the model-agnostic meta-learning framework through the data bus to drive the dynamic adjustment of the adversarial loss weight and the online fine-tuning of the generator parameters. The compensation effect evaluation module is associated with the status data of the rendering queue, and triggers the reinforcement training mode when it detects that the synchronization error continuously exceeds the standard, loads the incremental data set to iteratively optimize the generator, and forms a closed-loop balance mechanism for compensation accuracy and computational efficiency.

[0032] Step S105: Input the system buffer queue depth, synchronization error, and the compensation effect of the cross-screen pixel compensation frame into the model-agnostic meta-learning framework to dynamically adjust the learning rate of the LSTM-Transformer hybrid model, the adversarial loss weight of the conditional generative adversarial network, and the exploration rate parameter of the deep reinforcement learning model; In the interactive control method of the dual-screen organic display system of the present invention, the technical solution of step S105 realizes system-level performance tuning through cross-model parameter collaborative optimization. The model-agnostic meta-learning framework receives the system buffer queue depth, synchronization error, and pixel compensation effect data. The buffer queue depth is mapped to a node load balancing metric through a feature encoding module, the synchronization error is parsed into a spatio-temporal domain compensation deviation, and the compensation effect is quantitatively characterized as a pixel consistency score to construct a multi-dimensional parameter space. The framework generates a control strategy based on the historical optimization trajectory and real-time data, and outputs the learning rate adjustment coefficient of the LSTM-Transformer hybrid model, the adversarial loss weight allocation ratio of the conditional generative adversarial network, and the exploration rate correction parameter of the deep reinforcement learning model.

[0033] The Bayesian optimization algorithm analyzes the control strategy output by the meta-learning framework and constructs a Gaussian process surrogate model to predict the impact of parameter adjustment on system performance. For the LSTM-Transformer hybrid model, the dynamic learning rate adjustment strategy balances the memory of historical delay patterns and the adaptability to new data, and increases the learning rate in the high-delay fluctuation stage to accelerate model convergence. The adversarial loss weight allocation module of the conditional generative adversarial network adjusts the optimization weights of the generator reconstruction loss and the discriminator adversarial loss according to the pixel consistency score, and strengthens the compensation priority of the high-error region. The exploration rate parameter of the deep reinforcement learning model is dynamically corrected based on the node load index, reducing the exploration rate in the high-load state to maintain policy stability and increasing the exploration rate in the low-load stage to enhance policy diversity.

[0034] The parameter update module synchronously injects the optimized control parameters into each model through a distributed parameter server. The LSTM-Transformer hybrid model receives the new learning rate to reset the gradient update step size, and adopts a momentum acceleration strategy in the online incremental learning process to improve the convergence efficiency. The conditional generative adversarial network loads the adjusted adversarial loss weight, reconstructs the proportional relationship between the loss functions of the generator and the discriminator, and updates the network parameters through an online fine-tuning mechanism. The deep reinforcement learning model resets the sampling probability of the experience replay pool according to the corrected exploration rate and optimizes the generation efficiency of the task allocation strategy. The updated parameter status is fed back to the meta-learning framework in real time, triggering the reconstruction of the high-dimensional parameter space and policy iteration, forming a closed-loop collaborative mechanism from parameter adjustment, model optimization to effect feedback.

[0035] Step S106: Fuse the real-time data of the multi-modal sensor array and the prediction results of the LSTM-Transformer hybrid model through a dual-channel Kalman filter to generate an error correction signal. Feed back the error correction signal to the input end of the cross-screen time-series alignment process to adjust the sensor sampling frequency, and trigger the federated learning framework to globally aggregate and update the weights of the LSTM-Transformer hybrid model, forming a closed-loop control link for data acquisition, prediction compensation, and parameter optimization.

[0036] In the interactive control method of the dual-screen organic display system of the present invention, the technical solution of step S106 realizes the dynamic optimization of closed-loop control through data fusion and distributed learning. The main channel of the dual-channel Kalman filter constructs an observation equation based on the real-time data of the multi-modal sensor, and the secondary channel fuses the prediction results of the LSTM-Transformer hybrid model to construct a state equation. The spatio-temporal deviation is jointly estimated through covariance matrix iterative update, generating an error correction signal containing time-series compensation coefficients and spatial alignment parameters. The correction signal adjusts the sampling frequency of the sensor array through a feedback link. The capacitive touch sensor dynamically increases the sampling rate of high-frequency touch events according to the compensation coefficient, and the flexible strain sensor adjusts the sensitivity threshold according to the deformation response amplitude.

[0037] When the federated learning framework detects a communication interruption or a synchronization error exceeding the standard, it triggers a model recovery mechanism to calculate the gradient information of the LSTM-Transformer hybrid model through the incremental learning data stored locally by the dual-screen edge nodes. The gradient aggregation adopts a dynamic weighting strategy, allocating weight coefficients according to the node data magnitude and historical prediction accuracy to generate globally consistent network weights. The weight update module loads the pre-trained baseline model parameters, reconstructs the model weights in combination with the aggregated gradient direction, and synchronously resets the initial hidden state of the LSTM unit and the Transformer position encoding matrix to eliminate the cumulative prediction deviation in abnormal states.

[0038] The closed-loop control link realizes cross-module collaboration through multi-level data feedback. The spatial alignment parameters in the error correction signal are input into the cross-screen time-series alignment module to optimize the threshold of the touch point mapping rule; the time-series compensation coefficient drives the dynamic adjustment of sensor parameters, and the updated sensor data is input into the prediction model to generate a corrected delay distribution map. The weight update result of the federated learning framework is synchronized to the deep reinforcement learning model and the generative adversarial network, triggering the reset of the task allocation strategy and the reconstruction of the adversarial loss function. The compensated frame data stream and system state indicators are fed back to the meta-learning framework, driving the Bayesian optimization algorithm to update the parameter search space, forming a full-link closed-loop of data acquisition - prediction - compensation - optimization. The distributed parameter server monitors the parameter versions of each module in real time, and triggers the incremental learning mechanism when a version conflict is detected to maintain the consistency and environmental adaptability of the system control strategy.

[0039] The interactive control method of the dual-screen organic display system provided by the present invention forms a complete technical link through multimodal data synchronization processing, cross-screen communication delay prediction, dynamic task allocation, pixel compensation, parameter collaborative optimization, and closed-loop feedback mechanism. In the specific implementation process, the technical solutions and their logical relationships of each step are as follows: In the data acquisition and feature extraction stage, the capacitive touch sensor, flexible strain sensor, and ambient light sensor deployed at the dual-screen edge nodes respectively collect touch coordinates, deformation curvature, and light intensity data. The improved clock synchronization protocol is used to tag the multi-source sensor data with unified timestamps to eliminate the timing deviation caused by the physical separation of the dual screens. The sliding window mechanism is adopted to buffer and align the asynchronous data stream, and the abnormal data is eliminated based on the mapping rule between the touch point and the screen area to generate a spatio-temporally aligned intermediate data set. The lightweight convolutional network extracts the touch trajectory continuity feature and the deformation dynamic response feature from the intermediate data set, and outputs a weighted temporal feature vector set to provide a standardized input for subsequent delay prediction.

[0040] In the cross-screen communication delay prediction link, the temporal feature vector set and the historical communication delay data are input into the LSTM-Transformer hybrid model. This model captures the temporal dependence through the LSTM unit and uses the multi-head attention mechanism of the Transformer to analyze the spatial correlation between the touch trajectory and the screen deformation, and outputs the delay probability distribution map for the future multi-frame period. The online incremental learning strategy dynamically updates the model weights based on the sliding window data flow, enabling the prediction model to adapt to the dynamic fluctuations of the communication delay in real time. The generated delay probability distribution map will be used as the prior condition parameter for task allocation and pixel compensation.

[0041] In the dynamic task allocation process, the deep reinforcement learning model takes the delay probability distribution map, the real-time rendering queue status, and the cross-screen interaction instruction as inputs, constructs a multi-objective reward function including synchronization accuracy, response delay, and power consumption metrics, and generates a dynamic task allocation strategy. The physical layer adopts an improved ant colony optimization algorithm, and based on the screen space topology relationship and the real-time load data, iteratively calculates the shortest path allocation scheme for multi-channel rendering tasks to reduce the computational redundancy of distributed nodes. The optimized task allocation strategy will guide the parameter adjustment of the pixel compensation module.

[0042] In the cross-screen pixel compensation stage, the generator of the conditional generative adversarial network receives the original frame sequence with timing deviation and the delay probability distribution map, and outputs the pixel-level compensation frame. The discriminator conducts adversarial training based on the dual-screen display consistency constraint, and optimizes the generator parameters through the Wasserstein distance to improve the compensation accuracy. The knowledge distillation technology is adopted in the model inference stage to compress the scale of the generator network to adapt to the computing power limitation of the edge computing node. The compensated frame data stream carries the synchronization error information and will be fed back to the parameter optimization module.

[0043] The parameter collaborative optimization module maps the system buffer queue depth, synchronization error, and power consumption metrics into a high-dimensional parameter space and inputs it into the model-agnostic meta-learning framework. This framework quickly fits the optimal control strategy through a small amount of historical data and dynamically adjusts the learning rate of the LSTM-Transformer hybrid model, the adversarial loss weight of the generative adversarial network, and the exploration rate parameter of the deep reinforcement learning model in combination with the Bayesian optimization algorithm. The optimized parameters are reversely injected into the corresponding models to form a cross-model parameter collaborative update mechanism.

[0044] The closed-loop control link fuses the real-time sensor data and the model prediction results through a dual-channel Kalman filter, generates an error correction signal and feeds it back to the cross-screen timing alignment processing module, and dynamically adjusts the sensor sampling frequency and feature extraction weights. When a communication interruption is detected, the federated learning framework aggregates the gradient information of the local models on both screens and reconstructs the global consistency network weights of the LSTM-Transformer hybrid model. The correction signal synchronously triggers the strategy reset of the deep reinforcement learning model and the reconstruction of the loss function of the generative adversarial network, forming a full-link closed-loop control from data acquisition, prediction compensation to parameter optimization. The compensated frame data stream and system state information are continuously input into the meta-learning framework to drive the dynamic tuning of the parameters of each module and maintain the timing consistency and signal synchronization accuracy of the dual-screen interaction.

[0045] Specifically, for the interactive control method of the dual-screen organic display system described in the present invention, the real-time acquisition of cross-screen timing alignment and feature extraction includes: Using an improved clock synchronization protocol to tag the multi-source sensor data with a unified timestamp to generate a sensor data stream with timestamp synchronization; Inputting the sensor data stream with timestamp synchronization into a sliding window mechanism for buffer alignment, and removing abnormal data according to the preset mapping rules between the touch points and the screen areas to generate a spatio-temporal aligned intermediate data set; Inputting the spatio-temporal aligned intermediate data set into a lightweight convolutional network to extract the continuity features of the touch trajectory and the deformation dynamic response features, and output a weighted set of temporal feature vectors.

[0046] In the interactive control method of the dual-screen organic display system described in the present invention, the technical solution for real-time acquisition of cross-screen timing alignment and feature extraction realizes the standardization of multi-source heterogeneous data through a multi-level data processing link. In the data synchronization stage, an improved clock synchronization protocol constructs a nanosecond-level clock synchronization link based on the physical separation characteristics of the master and slave screens, stamps unified timestamps on touch coordinates, deformation curvature, and ambient light intensity data, and eliminates the baseline offset caused by the difference in the physical clocks of the two screens. The sensor data stream after timestamp synchronization enters the sliding window buffer alignment mechanism. According to the preset mapping rule of the effective area of the touch point, combined with the coherence threshold of the touch trajectory, abnormal data points are filtered to generate a spatio-temporally aligned intermediate data set, providing a standardized input for subsequent feature extraction.

[0047] In the feature extraction stage, the spatio-temporally aligned intermediate data set is input into a lightweight convolutional network for multi-dimensional feature analysis. The network architecture uses depthwise separable convolutional layers to extract the spatio-temporal continuity features of the touch trajectory, captures the dynamic response features of deformation through dilated convolutions, and adaptively weights and fuses the multi-modal feature vectors. The output weighted temporal feature vector set characterizes the correlation between the intention intensity of the touch behavior and the screen deformation. The weight coefficient reflects the sensitivity of the feature to the cross-screen interaction timing deviation, providing a normalized input for the delay prediction of the downstream model. The convolutional network uses a channel attention mechanism to dynamically adjust the feature weights and enhance the response ability to high-frequency touch events.

[0048] The sliding window mechanism adopts a dynamic window size adjustment strategy, adaptively expands or shrinks the buffer capacity according to the real-time changes of the sensor sampling frequency and cross-screen interaction instructions, and balances the data alignment accuracy and computational resource consumption. The abnormal data elimination module integrates a touch point spatial validity verification algorithm, identifies invalid contacts through the geometric mapping relationship between the touch coordinates and the screen display area, and filters mechanical deformation noise in combination with the deformation curvature mutation detection algorithm. The generation process of the intermediate data set synchronously records the data alignment status flag, providing metadata support for the compensation strategy in subsequent steps.

[0049] The feature extraction process of the lightweight convolutional network includes a multi-scale feature fusion module, which captures the local detailed features of the touch trajectory through shallow convolutional layers and extracts the global temporal correlation features of cross-screen interaction through deep networks. The network output layer adopts an adaptive weighting strategy, dynamically assigns weight coefficients according to the contribution degree of the feature vector in the historical delay prediction task, and forms a weighted temporal feature vector set. This vector set eliminates the dimensional difference of the sensors through normalization processing, and fills the temporal gaps caused by the elimination of abnormal data through a time series interpolation compensation mechanism to maintain the continuity of the feature vectors.

[0050] Specifically, for the interactive control method of the dual-screen organic display system described in the present invention, the construction of the hybrid prediction model to generate the delay probability distribution includes: Input the weighted time-series feature vector set and the cross-screen communication delay historical data into the LSTM-Transformer hybrid model, and parse the spatio-temporal correlation through the multi-head attention mechanism of the LSTM-Transformer hybrid model to output the cross-screen communication delay probability distribution map for multiple future frames; Based on the sliding window data stream, adopt an online incremental learning strategy to dynamically update the network weights of the LSTM-Transformer hybrid model to adapt to the dynamic fluctuations of communication delays.

[0051] In the interactive control method of the dual-screen organic display system described in the present invention, the construction of the hybrid prediction model and the generation of the delay probability distribution achieve accurate prediction of cross-screen communication delays through multi-modal data fusion and dynamic learning mechanisms. In the model input processing stage, the weighted time-series feature vector set and the cross-screen communication delay historical data are aligned in the spatio-temporal dimension through the feature concatenation module, where the weight coefficient of the time-series feature vector reflects the contribution degree of the touch behavior and deformation response to the delay. The LSTM-Transformer hybrid model adopts a cascaded architecture, where the LSTM network layer captures the time-series dependence of the touch trajectory, and the Transformer encoder layer analyzes the spatial correlation between the touch event and the screen deformation through the multi-head attention mechanism to output the spatio-temporal coupling feature vector of the cross-screen communication delay.

[0052] The delay probability distribution generation module constructs a probability map generation network based on the spatio-temporal coupling feature vector. This network maps the feature vector to a multi-dimensional probability space through a fully connected layer and outputs the cross-screen communication delay probability distribution map for multiple future frames. The probability distribution map contains the confidence interval and peak distribution characteristics of delay events within each time window, providing a quantitative basis for downstream task allocation and compensation. In the model training stage, a dynamic sample set is constructed using the sliding window data stream, and the window size is adaptively adjusted according to the dual-screen interaction frequency to balance the historical data coverage and real-time prediction requirements.

[0053] The online incremental learning strategy adopts a gradient memory replay mechanism to calculate the model weight gradient for the new data within the sliding window and updates the parameters by combining the historical gradient direction. This strategy sets a dynamic learning rate decay factor to adjust the model update intensity according to the communication delay fluctuation amplitude, and triggers the reinforcement learning mode when the detected delay standard deviation exceeds the threshold. The updated network weights are synchronized to the distributed computing nodes, and the staged training results are saved through the model snapshot mechanism to prevent the loss of the model state due to communication interruption. The weight change trajectory generated during the incremental learning process will be used as the input parameter of the meta-learning framework for collaborative parameter tuning in the cross-model optimization link.

[0054] Specifically, for the interactive control method of the dual-screen organic display system described in the present invention, the dynamic allocation of multi-channel rendering tasks includes: Taking the cross-screen communication delay probability distribution map, the real-time rendering queue status, and the cross-screen interaction instructions as inputs, a multi-objective deep reinforcement learning model is constructed to generate a dynamic task allocation strategy; Based on the screen space topological relationship and the real-time load data, an improved ant colony optimization algorithm is used to iteratively calculate the shortest path allocation scheme of multi-channel rendering tasks in the dynamic task allocation strategy.

[0055] In the interactive control method of the dual-screen organic display system described in the present invention, the dynamic allocation of multi-channel rendering tasks realizes the efficient scheduling of rendering resources through a hierarchical optimization mechanism. In the construction stage of the multi-objective deep reinforcement learning model, the cross-screen communication delay probability distribution map, the real-time rendering queue status, and the cross-screen interaction instructions are converted into a unified state vector by the feature encoding module. Among them, the delay probability distribution map provides the communication quality prediction of each time window, the rendering queue status includes the task buffer depth and the node load rate, and the cross-screen interaction instructions parse the user operation intention and the display content priority. The state space of the reinforcement learning model integrates the above multi-dimensional features, the action space is defined as the assignable rendering channel combination strategy, and the multi-objective reward function comprehensively considers the synchronization accuracy deviation, the task response delay, and the system power consumption index. The policy network parameters are iteratively optimized through the temporal difference learning algorithm to generate a dynamic task allocation strategy.

[0056] The improved ant colony optimization algorithm performs the shortest path allocation of rendering tasks at the physical layer. A node connection graph is constructed based on the screen space topological relationship, and the node weights are associated with the real-time load data and the communication link quality. In the initialization stage of the algorithm, the task allocation strategy output by the reinforcement learning model is transformed into a prior pheromone distribution to guide the search direction of the ant colony. During the iterative process, a dynamic evaporation factor is introduced to adaptively adjust the pheromone retention ratio according to the congestion degree of the rendering queue, balancing the global exploration and local exploitation capabilities. The path evaluation function integrates the peak delay risk in the communication delay probability distribution map and the node calculation margin to screen the shortest path scheme that meets the multi-objective constraints. The path allocation results generated in each round of iteration are fed back to the reinforcement learning model to update the synchronization accuracy evaluation parameters in the reward function, forming a policy optimization closed loop.

[0057] During the rendering task allocation process, the reinforcement learning model and the ant colony optimization algorithm achieve collaborative optimization through data exchange. The reinforcement learning model dynamically adjusts the feature weights of the state space according to the historical path allocation effect, and focuses on optimizing the task scheduling strategy within the high-delay risk window. The ant colony optimization algorithm dynamically corrects the screen space topological weights based on the real-time updated node load data, and combines the stability index of each region in the communication delay probability distribution map to preferentially allocate redundant rendering resources to the high-volatility region. The execution results of the task allocation strategy are synchronously recorded in the rendering queue status database for incrementally updating the training sample set of the reinforcement learning model, forming a complete scheduling link from policy generation, path optimization to effect feedback.

[0058] Specifically, in the interactive control method of the dual-screen organic display system of the present invention, the execution of cross-screen pixel compensation includes: Inputting the cross-screen communication delay probability distribution map and the original frame sequence with timing deviation into a conditional generative adversarial network to generate a cross-screen pixel compensation frame; Using the Wasserstein distance to optimize the adversarial training process of the conditional generative adversarial network, and compressing the network scale of the generator in the conditional generative adversarial network through knowledge distillation technology to adapt to the computing power limitation of the edge computing node.

[0059] In the interactive control method of the dual-screen organic display system of the present invention, cross-screen pixel compensation realizes dynamic repair of timing deviation through a generative adversarial network and edge computing adaptation technology. In the compensation frame generation stage, the generator of the conditional generative adversarial network receives the original frame sequence with timing deviation and the cross-screen communication delay probability distribution map, where the delay probability distribution map is used as a conditional input to guide the generator to learn the spatial distribution pattern of the timing deviation. The generator is constructed using a U-Net architecture. The encoder extracts multi-scale spatio-temporal features of the original frame, and the decoder combines the delay probability distribution information to generate a pixel compensation vector. The compensation vector is superimposed on the original frame through a residual connection to output a cross-screen pixel compensation frame. The discriminator is constructed based on the dual-screen display consistency constraint, and generates an adversarial loss signal by comparing the pixel distribution differences between the compensation frame and the ideal synchronous frame.

[0060] During the adversarial training optimization process, the Wasserstein distance is used as a loss function to measure the difference between the output distribution of the generator and the real data distribution, and the training stability is enhanced through a gradient penalty strategy. The training data stream adopts a sliding window mechanism to dynamically load the timing deviation samples in the real-time rendering task, and combines historical delay scenarios to construct an enhanced data set to improve the generalization ability of the model to complex delay patterns. The trained generator network is lightweight processed through knowledge distillation technology. The teacher network retains the complete U-Net structure, and the student network reconstructs the encoding and decoding modules using depthwise separable convolutions to reduce the number of network parameters while maintaining the compensation accuracy. During the distillation process, a feature map alignment strategy is adopted to make the intermediate layer features of the student network consistent with the corresponding layer features of the teacher network, and the knowledge transfer process is constrained by the mean square error loss.

[0061] During the edge computing node deployment phase, the compressed generator network optimizes the computational graph structure through operator fusion technology, combines the convolutional layer and the activation function into a single computational unit, and reduces the memory access latency. The network inference engine dynamically selects parallel computing strategies based on the hardware characteristics of the computing nodes, enables multi-stream parallel inference for GPU acceleration units, and adopts pipelined data processing for FPGA units. The generated result of the compensation frame is transmitted to the rendering queue in real time, the pixel difference data between the compensation frame and the original frame is synchronously recorded, and the compensation effect evaluation index is formed and fed back to the parameter optimization module. The compensation effect data includes the pixel error distribution in the spatio-temporal domain and the edge sharpness loss index, which are used to drive the online fine-tuning of the generative adversarial network and the collaborative parameter update of the meta-learning framework.

[0062] Specifically, in the interactive control method of the dual-screen organic display system according to the present invention, the dynamically collaborative optimization control parameters include: Mapping the buffer queue depth of the task allocation strategy, the synchronization error of the cross-screen pixel compensation frame, and the system power consumption index into a high-dimensional parameter space, and inputting them into a model-agnostic meta-learning framework to generate an optimal control strategy; Dynamically adjusting the learning rate of the LSTM-Transformer hybrid model, the adversarial loss weight of the conditional generative adversarial network, and the exploration rate parameter of the deep reinforcement learning model based on the Bayesian optimization algorithm.

[0063] In the interactive control method of the dual-screen organic display system according to the present invention, the dynamically collaborative optimization control parameters achieve system-level performance tuning through cross-model parameter coupling and intelligent optimization mechanisms. During the parameter space construction phase, the buffer queue depth of the task allocation strategy, the synchronization error of the cross-screen pixel compensation frame, and the system power consumption index are transformed into multi-dimensional vectors by the feature encoding module. Among them, the buffer queue depth reflects the computational load balancing state of the distributed nodes, the synchronization error quantifies the deviation degree between the pixel compensation effect and the ideal value, and the power consumption index includes the energy consumption distribution of each computational unit. The model-agnostic meta-learning framework maps the above multi-dimensional vectors into a high-dimensional parameter space, constructs a parameter optimization trajectory through a small number of historical data samples, and generates an optimal control strategy for multi-objective constraints. The strategy output includes the joint tuning directions of the LSTM-Transformer hybrid model, the conditional generative adversarial network, and the deep reinforcement learning model.

[0064] Based on the tuning direction output by the meta-learning framework, the Bayesian optimization algorithm constructs a Gaussian process surrogate model to predict the impact of parameter adjustment on system performance. For the LSTM-Transformer hybrid model, the optimization algorithm dynamically adjusts the learning rate to balance the memory of historical delay patterns and the adaptability to new data; for the conditional generative adversarial network, the training balance between the generator and the discriminator is adjusted through the dynamic allocation of adversarial loss weights; for the deep reinforcement learning model, the adjustment of the exploration rate parameter controls the trade-off between experience reuse and new policy development in the task allocation strategy. During the optimization process, a parallel parameter sampling strategy is adopted to synchronously evaluate the comprehensive impact of multiple groups of parameter combinations on system synchronization accuracy, response delay, and power consumption metrics, and to screen the Pareto optimal solution set.

[0065] In the parameter collaborative update stage, the optimized control parameters are synchronously injected into each model through a distributed parameter server. The LSTM-Transformer hybrid model receives the updated learning rate and resets the gradient update step size of online incremental learning; the conditional generative adversarial network loads the new adversarial loss weight and reconstructs the loss function ratio of the generator and the discriminator; the deep reinforcement learning model adopts the adjusted exploration rate and updates the data sampling strategy of the experience replay pool. The parameter update results are real-time fed back to the meta-learning framework, triggering the dynamic remapping of the high-dimensional parameter space. The buffer queue depth and synchronization error data are continuously collected during the execution of the rendering task, forming a closed-loop optimization data stream to drive the next round of parameter collaborative tuning. The system power consumption metric is real-time monitored through the hardware layer performance counter, providing a dynamic baseline reference for the multi-objective constraints of Bayesian optimization.

[0066] Specifically, for the interactive control method of the dual-screen organic display system described in the present invention, the generation of the error correction signal includes: Inputting the real-time data of the multi-modal sensor array and the prediction result of the LSTM-Transformer hybrid model into a dual-channel Kalman filter to generate an error correction signal and feedback it to the delay prediction input end of the LSTM-Transformer hybrid model; When a communication interruption is detected, the federated learning framework is started to aggregate the gradients of the dual-screen local models and reconstruct the global consistency network weights of the LSTM-Transformer hybrid model.

[0067] In the interactive control method of the dual-screen organic display system described in the present invention, the generation of the error correction signal and the model recovery mechanism achieve robust control of cross-screen interaction through data fusion and distributed learning. In the error correction stage, the real-time data of the multi-modal sensor array and the prediction results of the LSTM-Transformer hybrid model are input into the dual-channel Kalman filter after the time bases are unified by the data alignment module. The main channel constructs an observation equation based on the sensor data, and the secondary channel constructs a state transition equation by fusing the model prediction results. The spatio-temporal deviation is jointly estimated through the iterative update of the covariance matrix to generate an error correction signal. The correction signal includes a time series deviation compensation coefficient and a spatial alignment parameter, which are fed back to the delay prediction input end of the LSTM-Transformer hybrid model to adjust the feature weight allocation strategy of the multi-head attention layer and optimize the generation accuracy of the subsequent delay probability distribution map.

[0068] The federated learning framework triggers the global model recovery process in the communication interruption scenario, and calculates the gradient information of the LSTM-Transformer hybrid model through the incremental learning data locally stored by the dual-screen edge nodes. The gradient aggregation adopts a dynamic weighted average strategy, and the aggregation weights are allocated according to the node data volume and the amplitude of communication delay fluctuations to generate globally consistent network weights. The weight update module synchronizes the aggregated parameters to the dual-screen computing nodes, resets the historical gradient cache of online incremental learning, and reconstructs the initial hidden state of the LSTM unit. During the model recovery process, differential privacy technology is used to inject noise into the gradient data to prevent the leakage of sensitive information.

[0069] The error correction signal and the federated learning mechanism form a closed-loop control link. The time series deviation compensation coefficient in the correction signal acts on the sensor data acquisition module synchronously, dynamically adjusts the sampling frequencies of the capacitive touch and deformation sensors, and reduces the interference of high-frequency noise on the subsequent prediction model. The spatial alignment parameter is input into the cross-screen time series alignment processing module to optimize the mapping rule threshold between the touch point and the screen area. The weight update result of the federated learning framework is synchronously updated to the deep reinforcement learning model and the conditional generative adversarial network in real time, triggering the priority adjustment of the task allocation strategy and the reconstruction of the adversarial loss function of the pixel compensation network, forming a cross-module collaborative optimization mechanism. The generation frequency of the correction signal is dynamically adjusted according to the cumulative degree of the system synchronization error. When the error threshold exceeds the preset threshold, the enhanced correction mode is triggered to shorten the iteration period of the Kalman filter to improve the correction response speed.

[0070] Specifically, for the interactive control method of the dual-screen organic display system described in the present invention, the reverse adjustment data acquisition strategy includes: Dynamically adjusting the sampling frequency of the multi-modal sensor array and the feature weights of the lightweight convolutional network according to the error correction signal; Inject the adjusted sensor sampling parameters into the LSTM-Transformer hybrid model, and synchronize the updated feature weights to the conditional generative adversarial network and the deep reinforcement learning model to form a cross-model parameter collaborative update link.

[0071] In the interactive control method of the dual-screen organic display system of the present invention, the reverse adjustment of the data acquisition strategy realizes system-level optimization through parameter dynamic adaptation and cross-model collaboration mechanisms. In the sensor parameter adjustment stage, the timing deviation compensation coefficient in the error correction signal is input into the multi-modal sensor array control module, and the sampling frequency of the capacitive touch sensor and the sensitivity threshold of the flexible strain sensor are dynamically adjusted based on the amplitude of the compensation coefficient. The feature weight adjustment module analyzes the spatial alignment parameters in the error correction signal, and reconstructs the feature weight matrix of the lightweight convolutional network through the channel attention mechanism to enhance the feature extraction ability of high-frequency touch events and deformation mutations. The adjusted sensor sampling parameters and feature weights are synchronized to each computing node through the distributed parameter server.

[0072] In the parameter injection and collaborative update stage, the adjusted sensor sampling parameters are input into the data preprocessing layer of the LSTM-Transformer hybrid model to optimize the feature alignment benchmark of the multi-head attention mechanism. The updated feature weight matrix is synchronized to the input end of the generator of the conditional generative adversarial network through the weight sharing channel to constrain the feature selection priority in the compensation frame generation process. The deep reinforcement learning model receives the feature weight update signal, reconstructs the feature encoding rule of the state space, and associates the weight coefficient of the deformation response feature with the reward function calculation logic of the task allocation strategy. The parameter update result is recorded in real time in the collaborative optimization database of the meta-learning framework to trigger the next round of Bayesian optimization iteration.

[0073] The cross-model parameter collaborative update link forms a closed-loop feedback mechanism. The LSTM-Transformer hybrid model drives the deep reinforcement learning model to update the communication quality evaluation index in the task allocation strategy based on the delay probability distribution map generated by the new sampling parameters. The conditional generative adversarial network uses the optimized feature weights to improve the spatio-temporal consistency of the compensation frame, and the generated synchronization error data is fed back to the feature weight adjustment module to form a complete control loop from parameter adjustment, model optimization to effect verification. The distributed parameter server monitors the consistency status of each model parameter version, and triggers the gradient synchronization mechanism of the federated learning framework when a version conflict is detected to reconstruct the global consistent parameter configuration.

[0074] Specifically, in the interactive control method of the dual-screen organic display system of the present invention, the exception recovery mechanism includes: Re-initialize the network weights of the LSTM-Transformer hybrid model based on the model gradient aggregation result of the federated learning framework; Trigger the policy reset of the deep reinforcement learning model and the reconstruction of the adversarial loss function of the conditional generative adversarial network according to the real-time system state.

[0075] In the interactive control method of the dual-screen organic display system of the present invention, the anomaly recovery mechanism realizes fast fault tolerance of the system through distributed model reconstruction and dynamic policy adjustment. In the stage of re-initializing the model weights, the federated learning framework aggregates the gradient information of the dual-screen local nodes, and uses the dynamic weighted average strategy to calculate the global gradient mean value, where the weight coefficient is dynamically allocated according to the node data magnitude and historical prediction accuracy. The aggregated gradient data is input into the parameter update module of the LSTM-Transformer hybrid model, and the baseline network weights in the pre-training stage are loaded as the initialization benchmark, and the network parameters are reconstructed in combination with the update direction of the global gradient mean value. The hidden state of the LSTM unit and the position encoding matrix of the Transformer encoder are synchronously reset during the model initialization process to eliminate the influence of historical error accumulation on the prediction accuracy.

[0076] The policy reset of the deep reinforcement learning model is triggered based on the real-time system state. When the synchronization error exceeds the preset threshold or the communication interruption duration reaches the critical value, the policy management module clears the historical task allocation data in the experience replay pool and loads the global consistency model parameters generated by the federated learning framework as the initial policy network weights. The policy reset process retains the weight configuration of the multi-objective reward function and re-initializes the exploration rate parameter to balance the stability and innovation of the task allocation policy. The reset policy network generates a new task allocation plan based on the current rendering queue state and the delay probability distribution map.

[0077] The adversarial loss function reconstruction module of the conditional generative adversarial network monitors the change trend of the synchronization error of the pixel compensation frame. When the error fluctuation amplitude exceeds the dynamic threshold, it triggers the loss function weight adjustment process. The reconstruction loss weight of the generator and the adversarial loss weight of the discriminator are dynamically allocated based on the real-time compensation effect data. The spatio-temporal consistency index of the compensation frame is statistically analyzed through a sliding window, and the proportion of the spatial alignment loss and the temporal continuity loss in the loss function is adaptively adjusted. The reconstructed loss function updates the network parameters of the generator and the discriminator through the online fine-tuning strategy, and synchronously updates the feature alignment target in the knowledge distillation process.

[0078] The anomaly recovery mechanism forms a closed-loop linkage with other modules of the system. The model weight update signal of the federated learning framework triggers the policy reset timer of the deep reinforcement learning model to prevent task allocation conflicts caused by inconsistent model parameter versions. The loss function reconstruction result of the conditional generative adversarial network is fed back to the meta-learning framework to optimize the parameter search space of the Bayesian algorithm. After re-initialization, the LSTM-Transformer hybrid model loads the latest sensor sampling parameters, generates a delay probability distribution map based on the corrected feature weights, and drives a new round of task allocation and pixel compensation processes to achieve fast recovery and system self-healing under abnormal conditions.

[0079] Specifically, in the interactive control method of the dual-screen organic display system of the present invention, the data flow of the closed-loop control link includes: the error correction signal triggers the update of the data acquisition strategy of the multi-modal sensor array, and the updated sensor data is input into the LSTM-Transformer hybrid model; The optimized control parameters synchronously adjust the task allocation strategy of the deep reinforcement learning model and the pixel compensation parameters of the conditional generative adversarial network; The compensated frame data stream and the system state are fed back to the model-agnostic meta-learning framework to form a closed-loop link for cross-level parameter optimization and the anomaly recovery mechanism.

[0080] In the interactive control method of the dual-screen organic display system of the present invention, the data flow of the closed-loop control link realizes the self-adaptive adjustment of the system through multi-level feedback and collaborative optimization. In the data acquisition strategy update stage, the timing deviation compensation coefficient in the error correction signal is input into the multi-modal sensor array control module to dynamically adjust the sampling frequency of the capacitive touch sensor and the sensitivity threshold of the flexible strain sensor. The updated sensor data is processed by cross-screen timing alignment to generate a standardized feature vector set, which is input into the LSTM-Transformer hybrid model to update the delay probability distribution map and eliminate the prediction deviation caused by environmental interference or hardware drift. The sensor parameter adjustment results are synchronously recorded in the node configuration database of the federated learning framework to provide baseline parameters for anomaly recovery.

[0081] The optimized control parameters are synchronously injected into each functional module through a distributed parameter server. The deep reinforcement learning model receives the adjusted exploration rate parameter, resets the data sampling weight of the experience replay pool, and optimizes the generation efficiency of the task allocation strategy. The conditional generative adversarial network loads the updated adversarial loss weight, reconstructs the training objective functions of the generator and the discriminator, and improves the spatio-temporal consistency of the pixel compensation frame. A version consistency verification mechanism is adopted during the parameter synchronization process to prevent multi-model parameter version conflicts caused by network latency.

[0082] The compensated frame data stream and real-time system status are transmitted to the model-independent meta-learning framework via the data bus. The synchronization error index in the frame data stream and the rendering queue depth data construct a high-dimensional parameter space to drive the Bayesian optimization algorithm to update the parameter search strategy. The abnormal recovery mechanism monitors the system status data through the federated learning framework, and triggers the weight reinitialization process of the LSTM-Transformer hybrid model when the synchronization error is detected to be continuously exceeded. The optimized parameters and the recovered model weights are back-injected into the sensor array control module to form a closed-loop control link from data acquisition, model prediction to parameter optimization.

[0083] The system status feedback data includes real-time power consumption indicators and node load rates, which are input into the meta-learning framework to construct multi-objective constraints. The framework generates parameter adjustment suggestions through historical optimization trajectory analysis to dynamically balance synchronization accuracy and energy efficiency. The pixel error distribution data of the compensation frame further drives the online fine-tuning of the conditional generative adversarial network and updates the teacher network feature alignment target during the knowledge distillation process. The adaptability of the closed-loop link is reflected in the linkage update of sensor parameters, model weights and optimization strategies, which enables rapid response to dynamic environmental changes and long-term stability maintenance during dual-screen interaction.

[0084] The technical features of the present invention are explained as follows: Multimodal sensor array: refers to the combination of sensors deployed at the edge nodes of the dual screens, including capacitive touch sensors, flexible strain sensors and ambient light sensors. Capacitive touch sensors detect touch coordinates through matrix scanning; flexible strain sensors measure screen deformation curvature through piezoelectric materials; ambient light sensors collect ambient light intensity data based on photodiode arrays. Multimodal data fusion provides composite inputs of touch, deformation and illumination for subsequent processing.

[0085] Cross-screen timing alignment processing: The independent clock sources of the two screens are synchronized at the nanosecond level through an improved clock synchronization protocol, and the multi-source sensor data is marked with a unified timestamp to eliminate the timing deviation caused by the physical separation of the two screens. The sliding window mechanism buffers and aligns the asynchronous data streams, and removes out-of-bounds data by combining the geometric mapping rules of the touch point and the screen area, generating a time-space aligned intermediate data set.

[0086] LSTM-Transformer hybrid model: A deep learning model that combines the long short-term memory network (LSTM) with the Transformer architecture. The LSTM layer captures the temporal dependency of the touch trajectory, and the Transformer encoder parses the spatial correlation between the touch event and the screen deformation through a multi-head attention mechanism, outputting the spatiotemporal coupling feature vector of the cross-screen communication delay, which is used to generate the delay probability distribution map.

[0087] Deep Reinforcement Learning Model: A reinforcement learning architecture based on a multi-objective reward function. The inputs include a delay probability distribution map, the rendering queue status, and interaction instructions. The policy network parameters are optimized through the temporal difference learning algorithm to generate a dynamic task allocation strategy, balancing the conflicting objectives of synchronization accuracy, response delay, and system power consumption.

[0088] Improved Ant Colony Optimization Algorithm: A dynamic evaporation factor is introduced into the traditional ant colony algorithm to adaptively adjust the pheromone retention ratio according to the congestion degree of the rendering queue. A node connection graph is constructed based on the screen space topological relationship, and the shortest path allocation scheme is selected by combining delay risk and node load data to optimize the physical layer execution efficiency of rendering tasks.

[0089] Conditional Generative Adversarial Network (CGAN): The generator takes the delay probability distribution map as the conditional input and generates pixel compensation frames through the U-Net architecture; the discriminator evaluates the compensation effect based on the dual-screen display consistency constraint. The adversarial training uses the Wasserstein distance as the loss function, and the training stability is enhanced through the gradient penalty strategy.

[0090] Knowledge Distillation Technology: Transfer the knowledge of the teacher network (complete U-Net structure) to the student network (lightweight encoding and decoding module). The student network uses depthwise separable convolutions to reduce the number of parameters and maintains the consistency of the intermediate layer feature distribution through the feature mapping alignment strategy to adapt to the computing power limitations of edge nodes.

[0091] Model-Agnostic Meta-Learning Framework: Maps the system buffer queue depth, synchronization error, and compensation effect to a high-dimensional parameter space, and dynamically adjusts the learning rate of the prediction model, the adversarial loss weight, and the reinforcement learning exploration rate through the Bayesian optimization algorithm. Achieves collaborative optimization of cross-model parameters and improves the system's adaptive ability.

[0092] Dual-Channel Kalman Filter: The main channel constructs an observation equation based on real-time sensor data, and the secondary channel constructs a state equation by fusing model prediction results. The joint estimation of spatio-temporal deviation is iteratively updated through the covariance matrix, and an error correction signal is generated and fed back to the data acquisition and prediction module to form a closed-loop control.

[0093] Federated Learning Framework: Aggregates the gradient information of the dual-screen local models during communication interruption and uses a dynamic weighting strategy to generate globally consistent network weights. Protects the security of gradient data through differential privacy technology to achieve distributed collaborative update and anomaly recovery of model parameters.

[0094] Closed-Loop Control Link: A full-process closed-loop from data collection by sensors, delay prediction, task allocation, pixel compensation to parameter optimization. The error correction signal dynamically adjusts the sensor sampling frequency and model parameters, and the compensated frame data and system state feedback drive the continuous optimization of the meta-learning framework to maintain the spatio-temporal synchronization and system robustness of dual-screen interaction.

[0095] The interactive control method of the dual-screen organic display system provided by the present invention systematically solves the timing deviation problem in cross-screen interaction through a technical chain of multi-modal data fusion, dynamic prediction compensation, and closed-loop parameter optimization. First, the multi-modal sensor array collects touch, deformation, and ambient light data in real time, and realizes cross-screen timing alignment through an improved clock synchronization protocol and a sliding window mechanism, generating a weighted set of timing feature vectors. The LSTM-Transformer hybrid model fuses the timing features and historical delay data, and uses the multi-head attention mechanism to analyze the spatio-temporal correlation between the touch trajectory and the screen deformation, and outputs a cross-screen communication delay probability distribution map. This prediction result provides a quantitative basis for downstream task allocation and pixel compensation, reducing the impact of timing deviation on rendering synchronization from the source.

[0096] In the dynamic task allocation and compensation stage, the deep reinforcement learning model combines the delay probability distribution map and the rendering queue state to generate a task allocation strategy optimized for multiple objectives. The improved ant colony optimization algorithm iteratively calculates the shortest path allocation scheme based on the screen topology relationship, reducing task conflicts caused by communication delays. The conditional generative adversarial network (CGAN) takes the delay probability distribution map as the conditional input, optimizes the adversarial training process through the Wasserstein distance, generates spatio-temporally consistent compensation frames, and corrects the pixel-level misalignment of the original frame sequence. The knowledge distillation technology compresses the scale of the generator network to adapt to the computing power limit of the edge node, ensuring the balance of compensation efficiency and accuracy.

[0097] The closed-loop control mechanism fuses the sensor data and the model prediction results through a dual-channel Kalman filter, generating an error correction signal to reversely adjust the sensor sampling frequency and feature weights. The federated learning framework aggregates the local gradients of the dual screens during communication interruptions to reconstruct the global consistency parameters of the LSTM-Transformer model. The model-agnostic meta-learning framework dynamically collaboratively optimizes the learning rate of the prediction model, the adversarial loss weight, and the exploration rate parameter of reinforcement learning. The compensated frame data and the system state feedback drive cross-hierarchical parameter tuning. The data forms a closed loop from collection, prediction, allocation, compensation to feedback, realizing the dynamic synchronization consistency of the dual-screen interaction instructions and the rendering content.

Claims

1. An interactive control method for a dual-screen organic display system, characterized in that: include: The touch coordinates, deformation curvature and ambient light intensity data are collected in real time by a multimodal sensor array deployed at the edge nodes of the dual screens, and the touch coordinates, deformation curvature and ambient light intensity data are aligned across screens to generate a weighted time series feature vector set; Input the time series feature vector set and the historical data of cross-screen communication delay into the LSTM-Transformer hybrid model, parse the spatiotemporal correlation through the multi-head attention mechanism, and output the probability distribution map of cross-screen communication delay in the future multi-frame period; According to the cross-screen communication delay probability distribution map, real-time rendering queue status and cross-screen interaction instructions, a deep reinforcement learning model is used to generate a dynamic task allocation strategy, and the shortest path allocation scheme for multi-channel rendering tasks is iteratively calculated based on an improved ant colony optimization algorithm; Inputting the cross-screen communication delay probability distribution map and the original frame sequence with timing deviation into a conditional generative adversarial network to generate a cross-screen pixel compensation frame, wherein the conditional generative adversarial network optimizes adversarial training parameters through Wasserstein distance; Input the system buffer queue depth, synchronization error and the compensation effect of the cross-screen pixel compensation frame into the model-independent meta-learning framework, and dynamically adjust the learning rate of the LSTM-Transformer hybrid model, the adversarial loss weight of the conditional generative adversarial network and the exploration rate parameter of the deep reinforcement learning model; The real-time data of the multimodal sensor array and the prediction results of the LSTM-Transformer hybrid model are fused through a dual-channel Kalman filter to generate an error correction signal, which is fed back to the input end of the cross-screen timing alignment processing to adjust the sensor sampling frequency, and trigger the federated learning framework to perform global aggregation updates on the weights of the LSTM-Transformer hybrid model, forming a closed-loop control link of data acquisition, prediction compensation and parameter optimization.

2. The interactive control method of the dual-screen organic display system according to claim 1, characterized in that: The real-time acquisition of cross-screen timing alignment and feature extraction includes: An improved clock synchronization protocol is used to mark multi-source sensor data with a unified timestamp to generate a sensor data stream with synchronized timestamps. Inputting the timestamp-synchronized sensor data stream into a sliding window mechanism for buffer alignment, removing abnormal data according to a preset mapping rule between touch points and screen areas, and generating a spatiotemporally aligned intermediate data set; The time-space aligned intermediate data set is input into a lightweight convolutional network to extract touch trajectory continuity features and deformation dynamic response features, and output a weighted temporal feature vector set.

3. The interactive control method of the dual-screen organic display system according to claim 1, characterized in that: The constructing of a hybrid prediction model to generate a delay probability distribution comprises: Input the weighted time series feature vector set and the historical data of cross-screen communication delay into the LSTM-Transformer hybrid model, parse the spatiotemporal correlation through the multi-head attention mechanism of the LSTM-Transformer hybrid model, and output a probability distribution map of cross-screen communication delay in future multi-frame periods; Based on the sliding window data stream, an online incremental learning strategy is adopted to dynamically update the network weights of the LSTM-Transformer hybrid model to adapt to the dynamic fluctuations of communication delay.

4. The interactive control method of the dual-screen organic display system according to claim 3, characterized in that: The dynamic allocation of multi-channel rendering tasks includes: Taking the cross-screen communication delay probability distribution map, real-time rendering queue status and cross-screen interaction instructions as input, a multi-objective deep reinforcement learning model is constructed to generate a dynamic task allocation strategy; Based on the screen space topological relationship and real-time load data, an improved ant colony optimization algorithm is used to iteratively calculate the shortest path allocation scheme for multi-channel rendering tasks in the dynamic task allocation strategy.

5. The interactive control method of the dual-screen organic display system according to claim 4, characterized in that: The performing of cross-screen pixel compensation comprises: The cross-screen communication delay probability distribution graph and the original frame sequence with timing deviation are input into a conditional generative adversarial network to generate a cross-screen pixel compensation frame; The adversarial training process of the conditional generative adversarial network is optimized using the Wasserstein distance, and the network size of the generator in the conditional generative adversarial network is compressed through the knowledge distillation technology to adapt to the computing power limitations of the edge computing nodes.

6. The interactive control method of the dual-screen organic display system according to claim 5, characterized in that: The dynamic collaborative optimization control parameters include: Mapping the buffer queue depth of the task allocation strategy, the synchronization error of the cross-screen pixel compensation frame, and the system power consumption index into a high-dimensional parameter space, and inputting the model-independent meta-learning framework to generate an optimal control strategy; The learning rate of the LSTM-Transformer hybrid model, the adversarial loss weight of the conditional generative adversarial network, and the exploration rate parameter of the deep reinforcement learning model are dynamically adjusted based on the Bayesian optimization algorithm.

7. The interactive control method of the dual-screen organic display system according to claim 6, characterized in that: Generating an error correction signal comprises: Input the real-time data of the multimodal sensor array and the prediction results of the LSTM-Transformer hybrid model into a dual-channel Kalman filter, generate an error correction signal and feed it back to the delay prediction input end of the LSTM-Transformer hybrid model; When a communication interruption is detected, the federated learning framework is started to aggregate the dual-screen local model gradients and reconstruct the global consistency network weights of the LSTM-Transformer hybrid model.

8. The interactive control method of the dual-screen organic display system according to claim 7, characterized in that: The reverse adjustment data collection strategy includes: Dynamically adjusting the sampling frequency of the multimodal sensor array and the feature weights of the lightweight convolutional network according to the error correction signal; The adjusted sensor sampling parameters are injected into the LSTM-Transformer hybrid model, and the updated feature weights are synchronized to the conditional generative adversarial network and the deep reinforcement learning model to form a cross-model parameter collaborative update link.

9. The interactive control method of the dual-screen organic display system according to claim 8, characterized in that: The abnormal recovery mechanism includes: Reinitialize the network weights of the LSTM-Transformer hybrid model based on the model gradient aggregation results of the federated learning framework; The strategy reset of the deep reinforcement learning model and the adversarial loss function reconstruction of the conditional generative adversarial network are triggered according to the real-time system status.

10. The interactive control method of a dual-screen organic display system according to any one of claims 1 to 9, characterized in that: The data flow of the closed-loop control link includes: the error correction signal triggers the data acquisition strategy update of the multimodal sensor array, and the updated sensor data is input into the LSTM-Transformer hybrid model; The optimized control parameters synchronously adjust the task allocation strategy of the deep reinforcement learning model and the pixel compensation parameters of the conditional generative adversarial network; The compensated frame data stream and the system state are fed back to the model-independent meta-learning framework to form a closed-loop link between cross-level parameter optimization and the abnormality recovery mechanism.

Images