Control method and control system for controlling multi-screen synchronous operation by single screen

Abstract

The invention relates to the technical field of multi-screen interaction control, and particularly discloses a control method and a control system for controlling multi-screen synchronous operation by a single screen, gesture operation of a user on a main touch screen is captured, deep analysis is carried out, and a corresponding operation instruction is generated in combination with optimization of gesture mapping; different control signals are generated and distributed to the multiple slave screens, it is ensured that the signals can be seamlessly transmitted to each screen, the touch screen directly executes the operation control signals, the non-touch screen executes the simulation control signals through simulation gestures, a user only needs to conduct touch operation on the main screen, gesture operation does not need to be executed on the multiple slave screens respectively, and the user experience is improved. The operation complexity of the multi-screen equipment is simplified, the multi-screen equipment can adapt to different types of slave screens, the compatibility of a system to different screens is improved, the control synchronism is improved, it is ensured that the operation response time of multiple slave screens is consistent, the response speed of the equipment is improved, the synchronous operation precision is high, the stability is good, and the experience of a user in multi-screen interaction is more natural and smoother.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-screen interactive control, and in particular to a control method and a control system for controlling synchronous operation of multiple screens with a single screen. Background Art

[0002] With the rapid development of smart devices and robotics, multi-screen devices have gradually become an important part of users' daily operations. In many scenarios, users need to operate multiple displays simultaneously through the main screen, especially in multi-screen display systems, such as smart home control, industrial monitoring, multimedia display, robot shopping guide systems, etc., the application of extended display screens is becoming more and more extensive.

[0003] However, the existing multi-screen interaction methods have the following defects or technical obstacles in practical applications:

[0004] First, hardware differences lead to complex operations. For example, different screen sizes, resolutions, touch support levels and installation locations lead to poor coordination between screens. Some screens support touch operations, while others do not. This requires users to rely on multiple input methods when operating, increasing the difficulty and time cost of operations. For example, users need to perform gesture operations on the touch screen and use a mouse or keyboard for control on the non-touch screen, which greatly reduces the efficiency of multi-screen interaction.

[0005] Second, multi-screen devices lack a unified operation control mechanism. Although there are some multi-screen collaborative technologies, they can only achieve basic content synchronization and lack precise synchronization control of gesture operations and input commands, resulting in delayed or asynchronous feedback and responses between screens, which seriously affects the overall user experience.

[0006] Third, the inconvenience of user operation. Existing multi-screen control systems usually require users to input and operate on each screen separately. Users have to switch between different input devices and control methods, which increases the difficulty of operation and causes inconvenience in operation.

[0007] Therefore, how to achieve more accurate synchronous control of multiple screens through a single touch screen, especially when some screens do not support touch, has become a technical problem that needs to be solved urgently. Summary of the invention

[0008] The purpose of the present invention is to provide a control method, control system and non-temporary storage medium for the synchronous operation of multiple screens with a single screen, in view of the problem that the main screen in the prior art operates multiple slave display screens simultaneously, which is difficult to adapt to the differences in screen resolutions, sizes and touch methods, or the operation synchronization is poor.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is:

[0010] A method for controlling a single screen to synchronously operate multiple screens comprises the following steps:

[0011] S1, capturing operation information of a user performing a touch operation on a main screen, the operation information including: coordinates of a first touch point, pressing force and touch time; and collecting raw data of the operation information;

[0012] S2. Classify the operation information using a classification algorithm to identify the touch operation type corresponding to the operation information;

[0013] S3, converting the operation information into a standard input event; the standard input event includes: event type, coordinates of the second touch point and time information;

[0014] S4, mapping the standard input event to a gesture, and dynamically adjusting the gesture mapping parameters through an adaptive mapping formula; the adaptive mapping formula includes: Formula 1 and Formula 2;

[0015] The formula 1: Among them, x 1 x is the horizontal coordinate of the touch point on the main screen, 2 is the horizontal coordinate of the touch point on the screen; Formula 2: Among them, y 1 The horizontal coordinate of the touch point on the main screen, y 2 is the horizontal coordinate of the touch point on the screen;

[0016] S5. If the slave screen is a touch screen, generating an operation control signal according to the standard input event after gesture mapping; if the slave screen is a non-touch screen, generating an analog control signal according to the standard input event after gesture mapping;

[0017] S6. The touch screen executes the operation control signal; the non-touch screen executes the simulation control signal through a simulated gesture.

[0018] The control method for controlling the synchronous operation of multiple screens by controlling a single screen according to the present invention can more accurately capture the user's gesture operation through steps S1 to S2, and confirm the type of touch operation through classification processing and identification, and more accurately distinguish different operation types such as single-point touch, multi-point touch, sliding and zooming, thereby improving the accuracy of the touch operation response to the slave screen; if the response of the non-touch screen is slow or the operation result is inconsistent with the expectation, the key parameters of the gesture mapping can be dynamically adjusted through step S4, the synchronization and response speed of the gesture mapping and signal generation can be optimized, the gesture mapping algorithm and the signal generation model can be automatically updated, and the synchronization of the operation of the slave screen device can be improved for slave screen devices with different screen resolutions and sizes, thereby ensuring more accurate and smoother multi-screen operations; and, through steps S5 and S6, the touch screen and the non-touch screen are distinguished, and different control signals are generated, which can effectively solve the synchronous operation of the non-touch screen and the touch screen in the multi-screen device, especially when the touch screen and the non-touch screen are mixed, thereby improving the convenient function of controlling multiple screens through a single touch screen.

[0019] Preferably, the method for controlling the synchronous operation of multiple screens by controlling the single screen of the present invention, in S4, gesture mapping the standard input event, also includes using nonlinear transformation to process the gesture mapping of the standard input event.

[0020] As a preferred solution of the present invention, when processing multi-touch and complex gestures, such as rotation, zooming in / out, sliding, etc., traditional linear mapping may cause distortion. Nonlinear transformation technology is used to process complex gesture mapping, especially in a multi-touch environment, to ensure accurate mapping and execution of gestures. The corresponding control signals generated after mapping can be applicable to touch screens and non-touch screens, thereby improving the accuracy of complex gestures or multi-touch control.

[0021] Preferably, in the control method of controlling the synchronous operation of multiple screens by a single screen according to the present invention, the nonlinear transformation comprises Formula 3:

[0022]

[0023] Among them, a and e represent the scaling ratio of the slave screen relative to the main screen in the horizontal and vertical coordinates respectively; b and d represent the rotation angle of the touch point on the main screen and the slave screen respectively; c and f represent the translation amount of the touch operation on the main screen and the slave screen respectively.

[0024] As a preferred solution of the present invention, gesture mapping is performed on quasi-input events corresponding to complex touch operations by adding formula three of nonlinear transformation, the displacement of the touch point is adjusted according to the physical characteristics of different screens, and matrix transformation is introduced during multi-point touch to achieve accurate geometric transformation, further reducing the distortion of the touch operation mapping of the main screen and improving the high precision and stability of operation synchronization; whether it is the response speed from the touch screen or the non-touch screen, it can be calibrated through the dynamic adjustment of gesture mapping, further avoiding delays, freezes or asynchronism problems during the operation, making the user experience in multi-screen interaction more natural and smooth, and improving the accuracy of touch operation on the control of the slave screen, and better synchronization.

[0025] Preferably, in the control method of controlling multiple screens synchronously by a single screen described in the present invention, the dynamics in S4 also include: delay compensation parameters, adjustment operation priority parameters, optimization strategy and progressive adjustment; the delay compensation parameters obtain the delay data of the screen response through a feedback mechanism and adjust the delay compensation value.

[0026] As a preferred solution of the present invention, the response time of each screen is synchronized through dynamic adjustment of the delay compensation parameters. When resources are limited or the response speed varies greatly, the operation priority is adjusted according to the screen type to ensure that the operation of the key screen is completed first, and the accuracy and smoothness of multi-screen operations are ensured through optimization strategies; through progressive adjustment, the gesture mapping parameters and time compensation values ​​are incrementally adjusted to gradually improve the accuracy of synchronous control, which can effectively solve the complexity of user operations and synchronous control problems in multi-screen devices, especially when touch screens and non-touch screens are mixed, thereby realizing the convenient function of controlling multiple screens through a single touch screen.

[0027] Preferably, the control method for controlling the synchronous operation of multiple screens by controlling the single screen of the present invention, in S2, identifying the touch operation type corresponding to the operation information, also includes deeply analyzing the semantics of the user gesture; the method of deeply analyzing the zoom gesture is: calculating the distance change rate between the two touch points to determine the user's zoom intention; adjusting the zoom ratio of the zoom intention on the slave screen according to the pixel density of the slave screen; adjusting the zoom intention adopts formula four:

[0028]

[0029] Among them, PPI screen Indicates the pixel density of the screen, PPI touch Indicates the pixel density of the primary screen.

[0030] As a preferred solution of the present invention, the above method is used to adjust the zoom ratio of the zoom intention on the slave screen according to the pixel density of the slave screen, so that the zoom gesture can be executed more accurately on the slave screen, thereby enhancing the accuracy of the execution and further improving the synchronization of the touch operation of the slave screen following the main screen.

[0031] Preferably, the control method of controlling the synchronous operation of multiple screens by a single screen according to the present invention further includes step S7: real-time monitoring and feedback of the execution of the control signal by each slave screen, and dynamically correcting the gesture mapping function according to the execution, and the dynamic correction adopts formula 5:

[0032]

[0033] Among them, x actual Indicates the horizontal coordinate of the actual touch point, y actual Indicates the ordinate of the actual touch point, x expected Indicates the horizontal coordinate of the desired touch point, y expected Indicates the vertical coordinate of the desired touch point.

[0034] As a preferred solution of the present invention, by real-time monitoring and feedback of the execution of control signals by each slave screen, and by correcting the gesture mapping function through Formula 5, the system can dynamically adjust the gesture mapping to ensure that the user's operation response can be synchronized and accurate in real time regardless of whether it is from a touch screen or a non-touch screen. The feedback mechanism improves the intelligence level of the system, can actively adapt to the user's operating habits and performance changes of the device, and provide a smoother interactive experience.

[0035] Preferably, in the control method of controlling the synchronous operation of multiple screens by a single screen according to the present invention, the classification algorithm specifically includes: sliding detection based on threshold, click detection based on time window, multi-touch recognition and acceleration detection;

[0036] The threshold-based sliding detection determines whether the touch operation is a sliding gesture by calculating the displacement distance and time difference between consecutive touch points;

[0037] The time window-based click detection determines whether the touch operation is a short press or a long press by evaluating the duration of the touch point;

[0038] The multi-touch recognition determines whether the touch operation is a zoom gesture or a rotation gesture by identifying the coordinate relationship and spacing between multiple touch points;

[0039] The acceleration detection measures the speed change of the touch point to determine whether the touch operation is a drag gesture.

[0040] As a preferred solution of the present invention, through the above-mentioned relatively matched classification algorithm, the touch operations of various complex gestures can be judged more accurately, further reducing the probability of misjudgment of handheld touch operations, and thus being able to provide a more flexible and efficient multi-screen operation control solution, further achieving the technical effects of more precise control and operation synchronization, and improving the operating experience and system performance of multi-screen devices.

[0041] Preferably, the control method of controlling the synchronous operation of multiple screens by a single screen according to the present invention converts the operation information into standard input events, specifically including coordinate standardization and event formatting.

[0042] As a preferred solution of the present invention, by more finely converting the operation information into standard input events, the accuracy of operation information processing for touch operations can be further improved, the lag, delay or erroneous operation of touch operations from the slave screen to the main screen can be reduced, and the high precision and stability of the slave screen's response to touch operations can be further improved, making the user experience in multi-screen interaction more natural and smooth.

[0043] To achieve the purpose of the present invention, the present invention also provides a control system for a single screen to control multiple screens synchronously, comprising: a gesture capture module, a gesture parsing module, a gesture mapping module, a signal generation module, a multi-screen control module, a synchronous control module and a feedback mechanism module;

[0044] The gesture capture module is used to capture the operation information of the user performing touch operations on the main screen; the gesture parsing module is used to classify and identify the operation information, and can convert the operation information into standard input events; the gesture mapping module is used to output the standard input events as control instructions; the signal generation module is used to generate control signals from the control instructions; the multi-screen control module is used to distribute multiple control signals to multiple slave screens; the synchronous control module is used to monitor the operation status of the slave screens, and synchronously control the response of the slave screens to the touch operations of the main screen; the feedback mechanism module is used to collect and feedback the execution status of the slave screens on the control signals.

[0045] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0046] 1. Through the control method, the user only needs to operate on one touch screen, without having to perform gesture operations on each screen separately, which greatly simplifies the operation complexity of multi-screen devices;

[0047] 2. Through the optimization of gesture mapping, it can adapt to different types of slave screens, including screens with different resolutions, sizes, and touch support levels, thereby improving the system's compatibility with different screens. In addition, by classifying and identifying touch operations and more accurately generating control signals that match the slave screen, regardless of whether the slave screen supports touch, the system can improve control synchronization through simulated operations or direct control;

[0048] 3. By analyzing the touch operation of the main screen and optimizing the gesture mapping, the operation response time of all screens can be ensured to be consistent, avoiding the time consumption caused by users operating separately on multiple screens, which not only improves the overall operation efficiency, but also improves the response speed of the device;

[0049] 4. The control system no longer requires an independent touch module for non-touch screens, and multi-screen operation is achieved through the control of the main touch screen, which reduces the demand for hardware touch functions and helps reduce the manufacturing cost of the device, especially in multi-screen devices, reducing the configuration complexity of hardware sensors and control modules;

[0050] 5. The control system ensures high precision and stability of operation synchronization. The system can calibrate the response speed of both the touch screen and the non-touch screen, avoiding delays, freezes or asynchrony problems during operation, making the user experience in multi-screen interaction more natural and smooth. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of a control method for a single screen to control multiple screens synchronously according to the present invention;

[0052] Figure 2 It is a schematic diagram of module connection of a control system for a single screen to control multiple screens synchronously according to the present invention;

[0053] Figure 3 It is a connection diagram of a control system for a single screen to control multiple screens for synchronous operation according to the present invention. DETAILED DESCRIPTION

[0054] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0055] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] Embodiment 1:

[0057] refer to Figure 1 and Figure 2As shown, this embodiment discloses a control method for a single screen to control multiple screens synchronously, comprising the following steps:

[0058] S1, capturing operation information of a user performing a touch operation on the main screen, the operation information including: coordinates of a first touch point, pressing force and touch time; collecting raw data of the operation information;

[0059] S2. Classify the operation information through a classification algorithm to identify the touch operation type corresponding to the operation information;

[0060] S3, converting the operation information into a standard input event; the standard input event includes: event type, coordinates of the second touch point and time information;

[0061] S4, mapping the standard input event to a gesture, and dynamically adjusting the gesture mapping parameters through an adaptive mapping formula; the adaptive mapping formula includes formula 1 and formula 2;

[0062] Formula 1: Among them, x 1 x is the horizontal coordinate of the touch point on the main screen, 2 is the horizontal coordinate of the touch point on the screen; Formula 2: Among them, y 1 The horizontal coordinate of the touch point on the main screen, y 2 is the horizontal coordinate of the touch point on the screen;

[0063] S5. If the slave screen is a touch screen, an operation control signal is generated according to the standard input event after gesture mapping; if the slave screen is a non-touch screen, an analog control signal is generated according to the standard input event after gesture mapping;

[0064] S6. The touch screen executes the operation control signal; the non-touch screen executes the simulation control signal through the simulated gesture.

[0065] It should be noted that usually the gesture operations performed by users on the main screen mainly include single-point touch, multi-point touch, sliding or zooming operations. The operation information of the user's touch operation can be captured by the sensor, such as the direction, speed, touch point position, spacing change and other parameters of the user's touch operation gesture; the captured operation information can be used by an ordinary processor to collect the original data of the operation information.

[0066] The operation information is classified and processed through a classification algorithm, and the touch operation type corresponding to the operation information is identified. Specifically, the classification algorithm of this embodiment includes: threshold-based sliding detection, time window-based click detection, multi-touch recognition and acceleration detection; threshold-based sliding detection, by calculating the displacement distance and time difference between continuous touch points, and then judging whether the touch operation is / is a sliding gesture; time window-based click detection, by evaluating the duration of the touch point, and then judging whether the touch operation is a short press or a long press; multi-touch recognition, by identifying the coordinate relationship and spacing between multiple touch points, and then judging whether the touch operation is a zoom gesture or a rotation gesture; acceleration detection, by measuring the speed change of the touch point, and then judging whether the touch operation is / is a drag gesture.

[0067] Specifically, the algorithm classification described in this embodiment is exemplified as follows:

[0068]

[0069] The key to the present invention is to deeply analyze the semantics of user gestures. For example, for a zoom gesture, the user's zoom intention is determined by calculating the distance change rate between the two touch points, and the zoom ratio is adjusted according to the pixel density of the screen: Among them, PPI screen It is from the pixel density of the screen, PPI touch is the pixel density of the main screen; for sliding gestures, the sliding distance, speed and direction are determined by the speed and direction of the multi-touch spacing change, and are appropriately adjusted according to the characteristics of each screen; for example, if the resolution of the main touch screen is 1920x1080 and the resolution of one of the slave screens is 2560x1440, the system will convert the position of the touch operation proportionally; for example, when the user's coordinate position on the main screen is (960, 540), the corresponding position mapped to the slave screen will be (1280, 720).

[0070] Specifically, the parameters of the dynamic adjustment of gesture mapping of S4 of the present invention also include: delay compensation parameters, which obtain the delay data of the screen response through the feedback mechanism, adjust the delay compensation value, and keep the response time of each screen synchronized, which can be achieved by setting the parameter: Adjusted_Time = Local_Time + Predicted_Delay. More specifically, dynamic adjustment also includes adjusting the operation priority parameters, optimization strategy and progressive adjustment; adjusting the operation priority, which can distinguish between touch screens and non-touch screens according to the screen type when resources are limited or the response speed is greatly different, and adjusting the operation priority to ensure that the operation of the key screen is completed first; the optimization strategy can be based on dynamic optimization of feedback, and the system analyzes the collected feedback data, for example, the response time and execution accuracy of the non-touch screen in real time, automatically updates the gesture mapping algorithm and the signal generation model, and ensures the accuracy and smoothness of multi-screen operations; progressive adjustment, which accumulates data after multiple operations through the feedback mechanism, and gradually improves the accuracy of synchronous control by incrementally adjusting the gesture mapping parameters and time compensation values.

[0071] Through the above implementation steps, the present invention can effectively solve the problems of complexity of user operations and synchronous control in multi-screen devices, especially when a touch screen and a non-touch screen are used in combination, thereby realizing the convenient function of controlling multiple screens through a single touch screen.

[0072] Specifically, in S4 of this embodiment, the standard input event is gesture mapped, and the gesture mapping of the standard input event is processed by nonlinear transformation; more specifically, the nonlinear transformation includes Formula 3:

[0073]

[0074] Among them, a and e represent the scaling ratio of the slave screen relative to the main screen in the horizontal and vertical coordinates respectively; b and d represent the rotation angle of the touch point on the main screen and the slave screen respectively; c and f represent the translation amount of the touch operation on the main screen and the slave screen respectively.

[0075] It should be noted that the event type is understood as, for example, click, slide, zoom; the touch point coordinates are understood as the normalized position, and the time information is understood as, for example, click duration, slide speed, etc.; specifically, in S3 of the present invention, the operation information is converted into a standard input event, specifically including coordinate standardization and event formatting; the conversion process is exemplified as follows:

[0076] Coordinate normalization:

[0077] normalized_x=touch_point_x / screen_width

[0078] normalized_y=touch_point_y / screen_height

[0079] Event formatting:

[0080]

[0081] In order to further improve the accuracy and fluency of multi-screen operations, the present invention synchronously optimizes the parsed and mapped control instructions, and ensures that all screens perform corresponding operations at the same or appropriate speed, direction and action by analyzing the current state of each slave screen and the speed and acceleration of the user's gesture. An adjustment mechanism is set to prevent operation asynchrony due to network delays or hardware differences; the specific real-time adjustment mechanism calculates the adjustment value through the following formula: Adjusted_Time=Local_Time+Predicted_Delay; thereby ensuring operation synchronization.

[0082] It should be noted that if the slave screen is a non-touch screen, a simulated control signal is generated according to the standard input event after gesture mapping; the non-touch screen executes the simulated control signal through the simulated gesture. For example, the non-touch screen receives the control signal from the processor through the control unit, and parses and transmits the simulated control signal to the simulator. The system simulates the touch operation on the non-touch screen according to the content of the control signal. In this way, the non-touch screen can respond to the operation of the main touch screen in real time, realize operation synchronization, and realize simulated touch operation, as shown in the following example:

[0083] Simulate click:

[0084] if control_signal['event']=='CLICK':

[0085] click_position=control_signal['position']

[0086] non_touch_screen.simulate_click(click_position)

[0087] Simulate sliding:

[0088]

[0089] Specifically, in order to ensure the accuracy and consistency of gesture operations, the present invention also provides a feedback mechanism, which obtains feedback information of each screen by real-time monitoring of the operation execution status of each slave screen, such as operation response speed, synchronization status, etc., and dynamically adjusts the gesture synchronization algorithm according to the feedback to ensure that the operations between the screens maintain high-precision synchronization. The method for achieving correction or dynamic adjustment can be through machine learning methods, such as reinforcement learning to achieve optimized mapping functions, so that the mapping effect continues to improve in dynamic scenarios; it can also be dynamically corrected according to the difference between the actual touch point coordinates and the expected coordinates. For example, step S7: real-time monitoring and feedback of the execution status of each slave screen on the control signal, according to the execution status, dynamically correcting the gesture mapping function, and the dynamic correction adopts formula 5:

[0090] Among them, x actual Indicates the horizontal coordinate of the actual touch point, y actual Indicates the ordinate of the actual touch point, x expected Indicates the horizontal coordinate of the desired touch point, y expected Indicates the vertical coordinate of the desired touch point.

[0091] Embodiment 2:

[0092] refer to Figure 2 As shown, this embodiment discloses a control system for a single screen to control multiple screens synchronously, including: a gesture capture module, a gesture parsing module, a gesture mapping module, a signal generation module, a multi-screen control module, a synchronous control module and a feedback mechanism module;

[0093] The gesture capture module is used to capture the operation information of the user's touch operation on the main screen; the gesture parsing module is used to classify and identify the operation information, and can convert the operation information into standard input events; the gesture mapping module is used to output the standard input events as control instructions; the signal generation module is used to generate control signals from control instructions; the multi-screen control module is used to distribute multiple control signals to multiple slave screens; the synchronization control module is used to monitor the operation status of the slave screen, and synchronously control the response of the slave screen to the touch operation of the main screen; the feedback mechanism module is used to collect and feedback the execution of the control signal by the slave screen.

[0094] The gesture capture module and gesture analysis module of the present invention pass the standardized input events to the subsequent gesture mapping module and signal generation module. The system generates corresponding control instructions and generates corresponding control signals based on the results of gesture mapping. The generated control signals are distributed to multiple slave screens through the multi-screen control module to ensure that the signals can be seamlessly transmitted to each screen, whether it is a touch screen or a non-touch screen. The module can distinguish between touch screens and non-touch screens and provide different operation signals for them, so that the main screen and multiple slave screens can synchronously execute user gesture operations at the same time.

[0095] Specifically, refer to Figure 3 As shown, the multi-screen system of this embodiment includes a main touch screen and multiple slave screens, such as a touch screen computer with model number ModelA-1111 as the main screen, and some slave screens are non-touch screens, such as slave screens with model number ModelB-2222. These screens are connected through the same controller, and the controller can use an embedded processor, for example, a processor with model number RK3588, and all screens are connected for video output via an HDMI interface, and the main touch screen is connected to the processor via a USB interface to capture the user's touch input.

[0096] The present invention is particularly suitable for multi-screen operation scenarios such as smart devices, robots, and multimedia display systems. It solves the problem of inconsistent operations and lack of synchronization when screen configurations differ greatly. Both industrial equipment control and multi-screen display systems in consumer electronic products can benefit from this solution to improve the efficiency and convenience of multi-screen collaborative operations.

[0097] In summary, the present invention provides a flexible and efficient multi-screen operation control solution. Through technical means such as gesture mapping, signal generation, and multi-screen synchronous control, it enables users to accurately control and synchronize operations on multiple screens on the main touch screen, thereby improving the operating experience and system performance of multi-screen devices.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A control method for a single screen to control multiple screens synchronously, characterized in that: The following steps are involved: S1, capturing operation information of a user performing a touch operation on a main screen, the operation information including: coordinates of a first touch point, pressing force and touch time; and collecting raw data of the operation information; S2. Classify the operation information using a classification algorithm to identify the touch operation type corresponding to the operation information; S3, converting the operation information into a standard input event; the standard input event includes: event type, coordinates of the second touch point and time information; S4, mapping the standard input event to a gesture, and dynamically adjusting the gesture mapping parameters through an adaptive mapping formula; the adaptive mapping formula includes formula 1 and formula 2; The formula 1: Wherein, x1 is the horizontal coordinate of the touch point on the main screen, and x2 is the horizontal coordinate of the touch point on the slave screen; Formula 2: Among them, y1 is the horizontal coordinate of the touch point on the main screen, and y2 is the horizontal coordinate of the touch point on the slave screen; S5. If the slave screen is a touch screen, generating an operation control signal according to the standard input event after gesture mapping; if the slave screen is a non-touch screen, generating an analog control signal according to the standard input event after gesture mapping; S6. The touch screen executes the operation control signal; the non-touch screen executes the simulation control signal through a simulated gesture.

2. The control method of controlling multiple screens synchronously by a single screen according to claim 1, characterized in that: S4 performs gesture mapping on the standard input event, and also includes using non-linear transformation to process the gesture mapping of the standard input event.

3. The control method of controlling multiple screens synchronously by a single screen according to claim 2, characterized in that: The nonlinear transformation includes formula 3: Among them, a and e represent the scaling ratio of the slave screen relative to the main screen in the horizontal and vertical coordinates respectively; b and d represent the rotation angle of the touch point on the main screen and the slave screen respectively; c and f represent the translation amount of the touch operation on the main screen and the slave screen respectively.

4. The control method of controlling multiple screens synchronously by a single screen according to claim 1, characterized in that: The dynamics in S4 also include: delay compensation parameters, adjustment operation priority parameters, optimization strategy and progressive adjustment; the delay compensation parameters obtain the delay data of the screen response through a feedback mechanism and adjust the delay compensation value.

5. The control method of controlling multiple screens synchronously by a single screen according to claim 1, characterized in that: S2 identifies the touch operation type corresponding to the operation information, and also includes deep analysis of the semantics of the user gesture; the deep analysis of the zoom gesture is as follows: calculating the distance change rate between the two touch points to determine the user's zoom intention; adjusting the zoom ratio of the zoom intention on the slave screen according to the pixel density of the slave screen; adjusting the zoom intention using Formula 4: Among them, PPI screen Indicates the pixel density of the screen, PPI touch Indicates the pixel density of the primary screen.

6. The control method of controlling multiple screens synchronously by a single screen according to claim 1, characterized in that: The method further includes step S7: real-time monitoring and feedback of the execution of the control signal by each slave screen, and dynamically correcting the gesture mapping function according to the execution, wherein the dynamic correction adopts formula 5: Among them, x actual Indicates the horizontal coordinate of the actual touch point, y actual Indicates the ordinate of the actual touch point, x expected Indicates the horizontal coordinate of the desired touch point, y expected Indicates the vertical coordinate of the desired touch point.

7. The control method of controlling multiple screens synchronously by a single screen according to claim 1, characterized in that: The classification algorithm specifically includes: threshold-based sliding detection, time window-based click detection, multi-touch recognition and acceleration detection; The threshold-based sliding detection determines whether the touch operation is a sliding gesture by calculating the displacement distance and time difference between consecutive touch points; The time window-based click detection determines whether the touch operation is a short press or a long press by evaluating the duration of the touch point; The multi-touch recognition determines whether the touch operation is a zoom gesture or a rotation gesture by identifying the coordinate relationship and spacing between multiple touch points; The acceleration detection measures the speed change of the touch point to determine whether the touch operation is a drag gesture.

8. The method for controlling the synchronous operation of multiple screens by controlling the single screen according to claim 1, characterized in that: The operation information is converted into a standard input event, specifically including coordinate standardization and event formatting.

9. A control system for a single screen to control multiple screens synchronously, characterized in that: include: Gesture capture module, gesture parsing module, gesture mapping module, signal generation module, multi-screen control module, synchronization control module and feedback mechanism module; The gesture capture module is used to capture the operation information of the user's touch operation on the main screen; the gesture analysis module is used to classify and identify the operation information, and can convert the operation information into a standard input event; The gesture mapping module is used to output the standard input event as a control instruction; the signal generation module is used to generate a control signal from the control instruction; the multi-screen control module is used to distribute a plurality of the control signals to a plurality of slave screens; the synchronization control module is used to monitor the operation status of the slave screens and synchronously control the response of the slave screens to the touch operation of the master screen; The feedback mechanism module is used to collect and feed back the execution status of the control signal from the screen.

Images