A multi-point gesture recognition and generation method based on state machine driving

Abstract

The application belongs to the technical field of display control application, and particularly relates to a multi-point gesture recognition and generation method based on state machine driving, which aims to provide a recognition and generation scheme supporting single-point and multi-point touch gestures. The core of the method is to define a state machine containing an initial state, a pressing state, a single-click state, a double-click state and a sliding state, set transition conditions between the states, take the coordinates and state data of the touch points as inputs, drive state transition, and thus recognize single-point gestures. On this basis, by combining the single-point gesture states of multiple touch points, multi-point gestures such as double-point same-direction sliding and double-point reverse zooming are further recognized. The method adopts state machine design, so that the state boundaries and transition conditions are clear, and the method has the advantages of design visualization, state extraction and recognition facilitation.

Description

Technical Field

[0001] This application belongs to the field of display control application technology, and specifically relates to a method for multi-point gesture recognition and generation based on state machine driving. Background Technology

[0002] After adopting touch screen control in airborne display control systems, human-machine interface control methods such as "single-point touch gestures" and "multi-point touch gestures" will be introduced. The human-machine interface control system needs to complete touch gesture recognition and generation based on the coordinates and status of the touch screen touch points. Summary of the Invention

[0003] To address the aforementioned issues, this application provides a state machine-driven method for multi-point gesture recognition and generation, comprising the following steps:

[0004] Define a state machine for gesture recognition, which includes multiple predefined states and transition conditions between states;

[0005] Receive input data from the contact point, wherein the input data includes at least the contact point coordinates and the contact point state;

[0006] Based on the input data and the transition conditions, the state machine is driven to transition between predefined states.

[0007] Based on the state of the state machine, a corresponding gesture state identifier is generated to complete gesture recognition.

[0008] Preferably, the state machine includes the following states: initial state IDLE, pressed state HOLD, single click state CLICK, double click state DOUBLE CLICK, and sliding state SLIDE.

[0009] Preferably, the states are defined as follows:

[0010] Initial state IDLE: The contact state is RELEASE;

[0011] HOLD state: The contact state is PRESS pressed and the contact coordinates are valid;

[0012] Click state CLICK: The touch state changes from PRESS to RELEASE, and the touch coordinates are valid;

[0013] Double-click state (DOUBLE CLICK): Within a predetermined period, the contact state changes from PRESS to RELEASE twice consecutively, and the contact coordinates remain valid.

[0014] Slide state: The contact state is PRESS pressed, the contact coordinates are valid, and the contact coordinate values ​​change continuously.

[0015] Preferably, the state transition conditions include at least:

[0016] When in the initial state IDLE, if the contact state changes from RELEASE to PRESS, it will transition to the HOLD state.

[0017] When in the HOLD state, if the touch state changes from PRESS to RELEASE within the DclickWaitCycle of the double-click state determination cycle, then it will transition to the CLICK state.

[0018] When in the click state CLICK, if the count ClickCount of the click state CLICK reaches 2 within the double-click state determination cycle DclickWaitCycle, then it will transition to the double-click state DOUBLE CLICK.

[0019] When in the HOLD state, if the contact state remains pressed and the contact coordinates change continuously, then it will transition to the SLIDE state.

[0020] When in the SLIDE state, HOLD state, or CLICK state, if the touch state changes to RELEASE, then the device transitions back to the initial IDLE state.

[0021] When in the DOUBLE CLICK state, it unconditionally transitions back to the initial IDLE state.

[0022] Preferably, in the HOLD state, the number of cycles during which the touch point is in the PRESS state is counted to obtain a count value WaitCount; when WaitCount is greater than or equal to the double-click state determination cycle DclickWaitCycle, the generated gesture state identifier is PRESSHOLD state; when WaitCount is less than the double-click state determination cycle DclickWaitCycle, the generated gesture state identifier is PRESSED state.

[0023] Preferably, when performing multi-point gesture recognition, it includes:

[0024] For each of the multiple touch points, the state machine is used to independently identify the single-point gesture state;

[0025] The gesture states identified at each point are combined, and corresponding multi-point gesture state identifiers are generated according to predetermined combination rules.

[0026] Preferably, the multi-point gesture states include two-point swiping in the same direction (TWO POINT SLIDE), two-point swiping in opposite directions (ZOOM OUT SLIDE), and two-point swiping in opposite directions (ZOOM IN SLIDE).

[0027] Among them, when both two-point gestures are in the SLIDE state and the coordinates of each touch point change in the same direction, it is recognized as a two-point unidirectional SLIDE.

[0028] When both double-point gestures are in the SLIDE state, the coordinates of each touch point change in opposite directions, and the distance between the two points gradually decreases during the swiping process, it is recognized as a double-point reverse swiping ZOOM OUT SLIDE.

[0029] When both double-point gestures are in the SLIDE state, the coordinates of each touch point change in opposite directions, and the distance between the two points gradually increases during the sliding process, it is recognized as a double-point reverse-separated sliding ZOOM IN SLIDE.

[0030] Preferably, in the CLICK state, the number of times the state is entered is counted, and the count value is recorded using ClickCount.

[0031] Preferably, in the initial state IDLE, the following processing is performed: the contact state PRESS count WaitCount is set to 0, the contact state is set to RELEASE, and the contact coordinates are set to the contact coordinates before entering this state.

[0032] Preferably, in the condition for transitioning from the HOLD state to the SLIDE state, the continuous change of the contact coordinates is determined by a SLIDE flag, PressMove, being set to TRUE when the contact coordinates in the current cycle change compared to the previous cycle.

[0033] Preferably, the two-point sliding in the same direction, the two-point sliding outwards in opposite directions, and the two-point sliding in opposite directions in opposite directions, all transition back to the initial state IDLE when the state of any contact point changes to RELEASE.

[0034] Preferably, the method is applied to touchscreen control scenarios in airborne display control systems.

[0035] The advantages of this invention are: it employs a state machine design method, defining predetermined "single-point gesture states" and using single-point coordinates and touch state data as input, designing predetermined transition conditions between states to achieve dynamic transition and gesture recognition of "single-point gesture states." Furthermore, based on these "single-point gesture states," a "multi-point gesture state" transition state machine is designed according to the requirements of multi-point gesture recognition, completing the recognition and generation of "multi-point gestures." The advantages of this method are: by defining the "target state" through a "state machine," state boundaries and state transition conditions can be effectively determined, providing the advantage of "explicit design" for the dynamic recognition and transition of "multi-point gesture states," which facilitates the extraction and recognition of "multi-point gesture" states. Attached Figure Description

[0036] Fig. 1 This is the state machine design diagram of the "single-point gesture recognition" of the present invention.

[0037] Fig. 2 This is the state machine design diagram of the "multi-point gesture recognition" of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are only some, not all, of the embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings. Figs. 1-2 As shown, a multi-point gesture recognition and generation method based on state machine driving is characterized by recognizing multi-point gesture states and generating gesture state identifiers through a state machine driving mechanism. The state machine consists of five single-point control states: "Initial State (IDLE)," "Press State (HOLD)," "Single State (CLICK)," "Double-Click State (DOUBLE CLICK)," and "Slide State (SLIDE)." By defining transition conditions between each associated state, and based on single-point control input data (touch point coordinates, touch point state), the recognition and generation of single-point gesture states are completed.

[0039] 2. The single-point state is defined as follows:

[0040] 1) "Initial State (IDLE)": The contact state is "RELEASE", meaning there is no contact input data;

[0041] 2) "Press and Hold" state: The contact state is "PRESS", the contact coordinates are valid, that is, there is contact input data;

[0042] 3) "Click Status (CLICK)": The touch status changes from "PRESS" to "RELEASE", the touch coordinates are valid, that is, there is touch input data;

[0043] 4) "Double Click State (DOUBLE CLICK)": The contact state changes from "PRESS" to "RELEASE", and then from "PRESS" to "RELEASE" again. The contact coordinates are valid, meaning there is contact input data.

[0044] 5) "Sliding State (SLIDE)": The contact state is "PRESS", the contact coordinates are valid, and the contact coordinate values ​​change continuously, that is, there is contact input data.

[0045] 3. The processing definition for single-point states is as follows:

[0046] 1) "Initial State (IDLE)": The contact state "PRESS" count (WaitCount) is set to 0, the contact state is set to "RELEASE", and the contact coordinates are set to the contact coordinates before the state is entered;

[0047] 2) "Holded State": Upon entering this state, the cycle counting process for the "PRESS" state of the touch point begins, and the count value is recorded using WaitCount. The values ​​of the touch point coordinates in each cycle of this state are compared. If the value of the current cycle changes from the value of the previous cycle, the "PressMove" flag is set to "TRUE". If the cycle count (WaitCount) of the "PRESS" state of this state is greater than or equal to the "DclickWaitCycle", the gesture state flag (OperateType) is set to "PRESS HOLD". If the cycle count (WaitCount) of the "PRESS" state of this state is less than the "DclickWaitCycle", the gesture state flag (OperateType) is set to "PRESSED".

[0048] 3) "Click State (CLICK)": After entering the state, start counting the "Click State (CLICK)" state and record the count value using ClickCount; set the gesture state identifier (OperateType) to "Click State (CLICK)";

[0049] 4) "Double-click state (DOUBLE CLICK)": After entering the state, set the gesture state identifier (OperateType) to "Double-click state (DOUBLE CLICK)"; set the "Click-click state (CLICK)" count (ClickCount) to 0;

[0050] 5) "Slide State (SLIDE)": After entering the state, set the gesture state identifier (OperateType) to "Slide State (SLIDE)";

[0051] 4. The single-point state transition conditions are defined as follows:

[0052] 1) The "initial state (IDLE)" is transitioned to the "pressed state (HOLD)"[2]: the contact state is changed from "RELEASE" to "PRESS";

[0053] 2) The "press state (HOLD)" is moved to the "click state (CLICK)"[3]: During the "double-click state determination cycle (DclickWaitCycle)", the touch state is changed from "PRESS" to "RELEASE";

[0054] 3) The "Click state (CLICK)" is migrated to the "Double-click state (DOUBLE CLICK)"[4]: During the "Double-click state determination cycle (DclickWaitCycle)", the "Click state (CLICK)" state count (ClickCount) is >= 2;

[0055] 4) "Hold" state transitions to "Slide" state [5]: The contact state remains "PRESS", and the contact coordinates change continuously (PressMove).

[0056] 5) The “sliding state (SLIDE)” transitions to the “initial state (IDLE)”[6]: the contact state is “RELEASE”;

[0057] 6) The "Hold" state is moved to the "IDLE" state [7]: the contact state is "RELEASE";

[0058] 7) The “Click state (CLICK)” is migrated to the “Initial state (IDLE)”[8]: the “Click state (CLICK)” state count (ClickCount) is less than 2;

[0059] 8) "Double-click state (DOUBLE CLICK)" migrates to "Initial state (IDLE)" [9]: unconditional direct migration.

[0060] 5. Single-point gesture recognition: Based on the state and transition conditions of the single-point gesture state machine defined above, the single-point gesture can be recognized according to the touch coordinates and touch state of the single point. The state machine design definition is attached to the instruction manual. Fig. 1 .

[0061] 6. Multi-point gesture recognition: Using the single-point gesture recognition function, gestures at each point in a multi-point system can be recognized to obtain the gesture state recognition for each point. Combining the gesture states of each point yields a multi-point gesture recognition scheme. The state machine design definition is attached to the instruction manual. Fig. 2 .

[0062] 1) "Initial state (IDLE)" is migrated to "two-point sliding (TWO POINT SLIDE)" [2]: both two-point gestures are in "sliding state (SLIDE)" and the coordinate change direction (horizontal / vertical direction) of each touch point is consistent;

[0063] 2) "Initial state (IDLE)" transitions to "two-point reverse (opposite) sliding (ZOOM OUT SLIDE)" [3]: Both two-point gestures are in "sliding state (SLIDE)", and the coordinate changes of each touch point are opposite (horizontal / vertical direction). During the "sliding" process, the distance between the touch points gradually decreases.

[0064] 3) “Initial state (IDLE)” transitions to “two-point reverse (separate) sliding (ZOOM IN SLIDE)”[4]: Both two-point gestures are in “sliding state (SLIDE)”, and the coordinates of each touch point change in opposite directions (horizontal / vertical). During the “sliding” process, the distance between the touch points gradually increases.

[0065] 4) "TWO POINT SLIDE" is moved to "IDLE" [5]: the contact state is "RELEASE".

[0066] 5) "Double-point reverse (opposite) sliding (ZOOM OUT SLIDE)" is moved to "Initial state (IDLE)" [6]: the contact state is "RELEASE".

[0067] 6) "ZOOM IN SLIDE" transitions to "IDLE" [7]: the contact state is "RELEASE".

[0068] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-point gesture recognition and generation method based on state machine driving, characterized in that, Includes the following steps: Define a state machine for gesture recognition, which includes multiple predefined states and transition conditions between states; Receive input data from the contact point, wherein the input data includes at least the contact point coordinates and the contact point state; Based on the input data and the transition conditions, the state machine is driven to transition between predefined states. Based on the state of the state machine, a corresponding gesture state identifier is generated to complete gesture recognition.

2. The method for multi-point gesture recognition and generation based on state machine driving according to claim 1, characterized in that, The state machine includes the following states: initial state IDLE, pressed state HOLD, single click state CLICK, double click state DOUBLE CLICK, and sliding state SLIDE.

3. The method for multi-point gesture recognition and generation based on state machine driving according to claim 2, characterized in that, The definitions of each state are as follows: Initial state IDLE: The contact state is RELEASE; HOLD state: The contact state is PRESS pressed and the contact coordinates are valid; Click state CLICK: The touch state changes from PRESS to RELEASE, and the touch coordinates are valid; Double-click state (DOUBLE CLICK): Within a predetermined period, the contact state changes from PRESS to RELEASE twice consecutively, and the contact coordinates remain valid. Slide state: The contact state is PRESS pressed, the contact coordinates are valid, and the contact coordinate values ​​change continuously.

4. The method for multi-point gesture recognition and generation based on state machine driving according to claim 2, characterized in that, The state transition conditions include at least the following: When in the initial state IDLE, if the contact state changes from RELEASE to PRESS, it will transition to the HOLD state. When in the HOLD state, if the touch state changes from PRESS to RELEASE within the DclickWaitCycle of the double-click state determination cycle, then it will transition to the CLICK state. When in the click state CLICK, if the count ClickCount of the click state CLICK reaches 2 within the double-click state determination cycle DclickWaitCycle, then it will transition to the double-click state DOUBLE CLICK. When in the HOLD state, if the contact state remains pressed and the contact coordinates change continuously, then it will transition to the SLIDE state. When in the SLIDE state, HOLD state, or CLICK state, if the touch state changes to RELEASE, then the device transitions back to the initial IDLE state. When in the DOUBLE CLICK state, it unconditionally transitions back to the initial IDLE state.

5. The multi-point gesture recognition and generation method based on state machine driving according to claim 2 or 4, characterized in that, In the HOLD state, the number of cycles during which the touch point is in the PRESS state is counted to obtain the count value WaitCount. When WaitCount is greater than or equal to the double-click state determination cycle DclickWaitCycle, the generated gesture state is identified as PRESSHOLD. When WaitCount is less than the double-click state determination cycle DclickWaitCycle, the generated gesture state is identified as PRESSED.

6. The method for multi-point gesture recognition and generation based on state machine driving according to claim 1, characterized in that, When performing multi-point gesture recognition, it includes: For each of the multiple touch points, the state machine is used to independently identify the single-point gesture state; The gesture states identified at each point are combined, and corresponding multi-point gesture state identifiers are generated according to predetermined combination rules.

7. The method for multi-point gesture recognition and generation based on state machine driving according to claim 6, characterized in that, The multi-point gesture states include two-point swiping in the same direction (TWO POINT SLIDE), two-point swiping in opposite directions (ZOOM OUT SLIDE), and two-point swiping in opposite directions (ZOOM IN SLIDE). Among them, when both two-point gestures are in the SLIDE state and the coordinates of each touch point change in the same direction, it is recognized as a two-point unidirectional SLIDE. When both double-point gestures are in the SLIDE state, the coordinates of each touch point change in opposite directions, and the distance between the two points gradually decreases during the swiping process, it is recognized as a double-point reverse swiping ZOOM OUT SLIDE. When both double-point gestures are in the SLIDE state, the coordinates of each touch point change in opposite directions, and the distance between the two points gradually increases during the sliding process, it is recognized as a double-point reverse-separated sliding ZOOM IN SLIDE.

8. The method for multi-point gesture recognition and generation based on state machine driving according to claim 4, characterized in that, In the CLICK state, the number of times this state is entered is counted, and the count value is recorded using ClickCount.

9. The method for multi-point gesture recognition and generation based on state machine driving according to claim 2, characterized in that, In the initial IDLE state, the following processes are performed: the contact state PRESS count WaitCount is set to 0, the contact state is set to RELEASE, and the contact coordinates are set to the contact coordinates before entering this state.

10. The method for multi-point gesture recognition and generation based on state machine driving according to claim 4, characterized in that, The continuous change of the contact coordinates in the condition of transitioning from the HOLD state to the SLIDE state is determined by a sliding flag PressMove being true TRUE, which is set when the contact coordinates in the current cycle change compared to the previous cycle.

11. The method for multi-point gesture recognition and generation based on state machine driving according to claim 7, characterized in that, The two-point sliding (TWO POINT SLIDE), two-point sliding (ZOOM OUT SLIDE), and two-point sliding (ZOOM IN SLIDE) states all transition back to the initial state IDLE when the state of any contact point changes to RELEASE.

12. The method for multi-point gesture recognition and generation based on state machine driving according to claim 1, characterized in that, The method is applied to touchscreen control scenarios in airborne display control systems.

Images