Gesture recognition method and system based on region division
By dividing the writing area and gesture recognition area on the electronic whiteboard touch screen, and using confidence score to distinguish the operational purpose, the problem of low error touch and recognition accuracy is solved, and the accuracy and user experience of gesture recognition are improved.
Patent Information
- Application Number
- CN202411028702.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-15
AI Technical Summary
The existing gesture recognition method has a high probability of accidentally touching and low recognition accuracy on the electronic whiteboard touch screen, resulting in poor user operation experience, especially in writing or painting, which makes it difficult to distinguish between writing and gestures. operate.
Using the gesture recognition method based on area division, the writing area and gesture recognition area are divided into the screen, touch events are obtained and judged, and the operation purpose is determined using the preset rule base and confidence score. The area position and size are dynamically adjusted to reduce the Miscontact and improve identification accuracy.
It significantly reduces the probability of false triggering during gesture recognition, improves recognition accuracy and user operation experience, and ensures the system's correct response to different types of touch events.
Smart Images

Figure CN118963637A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of screen recognition technology, and in particular to a gesture recognition method and system based on area division. Background Art
[0002] With the popularity of smart devices, touch screens have become the mainstream human-computer interaction method. Electronic whiteboard touch screens control and operate devices by detecting user touch operations. However, with the continuous expansion of the application scenarios of electronic whiteboard touch screens, single touch operations can no longer meet the needs of users, and gesture recognition technology has emerged. Gesture recognition technology can not only recognize simple touch operations of users, but also realize more functions and operations through complex gestures, including zooming, switching screens, and browsing up and down. Most of the existing gesture recognition methods are based on the entire touch screen for recognition, and there are the following problems: High probability of false touch: Users are prone to falsely trigger gesture operations during writing or drawing, resulting in unexpected function execution. Low recognition accuracy: Since the gesture recognition area overlaps with the writing area, it is difficult for the system to distinguish between the user's writing operation and gesture operation, affecting the recognition accuracy. Frequent false touches and inefficient gesture recognition seriously affect the user's operating experience. Summary of the invention
[0003] To solve the above problems, the present invention provides a gesture recognition method and system based on area division, which divides the screen into a writing area and a gesture recognition area, and determines whether the touch events generated in the two areas are the same, and then selects the operation purpose with a higher probability according to the confidence level to achieve more accurate distinction between gesture recognition and writing, thereby improving the user's operating experience.
[0004] To achieve the above object, the technical solution adopted by the present invention is:
[0005] A gesture recognition method based on region division, comprising:
[0006] Divide the writing area and gesture recognition area according to the screen resolution;
[0007] Get touch events in the writing area and gesture recognition area;
[0008] Jointly judge the touch events and match the execution functions.
[0009] Furthermore, dividing the writing area and the gesture recognition area according to the screen resolution includes:
[0010] Get screen parameters, including size data and current resolution settings;
[0011] Initialize according to screen parameters and divide the writing area and gesture recognition area;
[0012] Adjust the position and size of the writing area and gesture recognition area based on user usage data.
[0013] Furthermore, the step of acquiring touch events in the writing area and the gesture recognition area includes:
[0014] Monitor the touch event through the listener to obtain the coordinates of the touch event and the type of the touch event;
[0015] According to the occurrence coordinates and the area where the touch event occurs, the touch events in the writing area and the touch events in the gesture recognition area are classified.
[0016] Furthermore, the types of touch events include normal touch and pressure touch. If the touch pressure is greater than a preset pressure threshold, the touch event is marked as pressure touch; if the touch pressure is less than or equal to the preset pressure threshold, the touch event is marked as normal touch.
[0017] Furthermore, the joint determination of the touch event includes:
[0018] Based on a preset rule library and according to the type of touch event and the area where it occurs, determine the operation purposes of the touch events in the writing area and the gesture recognition area respectively and calculate their confidence scores; the operation purposes include writing and gesture control, and the preset rule library includes a general rule library and a pressure rule library;
[0019] Based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area, continuity determination is performed to determine whether the touch events in the writing area and the gesture recognition area belong to the same touch event and match and execute screen functions;
[0020] If they belong to the same touch event, a single operation purpose is selected according to the confidence score and the corresponding screen function is executed;
[0021] If they do not belong to the same touch event, the operation purpose is selected according to the confidence score and the corresponding screen function is executed.
[0022] Further, the executing the corresponding screen function includes:
[0023] If the touch event is a normal touch and the operation purpose is writing, the default writing is performed;
[0024] If the touch event is a pressure touch and the operation purpose is writing, bold writing is performed;
[0025] If the touch event is a normal touch and the operation purpose is gesture control, the normal rule library is matched to respond to the operation;
[0026] If the touch event is a pressure touch and the operation purpose is pressure control, the pressure rule library is matched to respond to the operation.
[0027] Furthermore, the determining the operation purposes of the touch events in the writing area and the gesture recognition area and calculating the confidence scores thereof respectively according to the type of the touch event and the area where the touch event occurs includes:
[0028] Calculating a touch event vector based on an initial touch point, a moving trajectory, and an end touch point of the touch event;
[0029] The cosine similarity between the touch event vector and the touch rules in the preset rule library is calculated, and the operation purpose is matched according to the cosine similarity and the confidence score is calculated.
[0030] Furthermore, the continuity determination based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area includes:
[0031] Determine the sequence of touch events in the writing area and the gesture recognition area based on the start timestamps and end timestamps of the touch events in the writing area and the gesture recognition area respectively;
[0032] Calculate the time difference by the end timestamp of the first touch event and the start timestamp of the second touch event;
[0033] Calculate the distance difference between the end coordinates of the first touch event and the start coordinates of the second touch event;
[0034] The time difference and distance difference are weighted to determine continuity.
[0035] Furthermore, the formula for determining the continuity based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area is as follows:
[0036]
[0037] Among them, C is the continuity score; T e and T s are the end timestamp of the first touch event and the start timestamp of the second touch event respectively; (X e , Y e ) is the end coordinate of the first touch event; (X s , Y s ) is the starting coordinate of the back touch event; ΔT max and ΔD max They are respectively a preset time threshold and a preset distance threshold; w1 and w2 are weighting coefficients.
[0038] Further, the determining whether the touch events in the writing area and the gesture recognition area respectively belong to the same touch event includes:
[0039] If the continuity score C is greater than or equal to the preset continuity threshold, it is determined to belong to the same touch event;
[0040] If the continuity score C is less than or equal to the preset continuity threshold, it is determined that they do not belong to the same touch event.
[0041] Furthermore, adjusting the position and size of the writing area and the gesture recognition area according to the user usage data includes:
[0042] Extracting touch events whose continuity determination results are the same touch event and whose operation purpose is gesture control, and obtaining the touch event vectors thereof in the non-gesture recognition area, and marking them as false touch event vectors;
[0043] Perform cluster analysis on the false touch event vectors to determine the high-frequency occurrence area of the touch event vectors;
[0044] Adjust the position and size of the gesture recognition area based on the high-frequency occurrence area.
[0045] A gesture recognition system based on area division, applied to any of the above-mentioned gesture recognition methods based on area division, comprising an area division module, an area monitoring module and a determination execution module;
[0046] The area division module is used to divide the writing area and the gesture recognition area according to the screen resolution;
[0047] The area monitoring module is used to obtain touch events in the writing area and the gesture recognition area;
[0048] The determination and execution module is used to jointly determine the touch events and match the execution functions.
[0049] The beneficial effects of the present invention are as follows: the present invention divides the writing area and the gesture recognition area according to the screen resolution and the screen parameters. Through the initial division and the dynamic adjustment of the user usage data, the writing area and the gesture recognition area are clearly separated, and the probability of false triggering during the gesture recognition process is reduced. The touch events of the writing area and the gesture recognition area are obtained through the listener, including the occurrence coordinates and types of the touch events. By classifying the touch events into the writing area and the gesture recognition area, the system can process the touch events of different areas separately, and further reduce the false triggering rate. Based on the preset rule library and the type of the touch event and the area where it occurs, the operation purpose of the touch event is determined and its confidence score is calculated. By performing continuity determination on the touch events of the writing area and the gesture recognition area, it is determined whether they belong to the same touch event, and the corresponding screen function is matched and executed. This process significantly improves the accuracy of gesture recognition and reduces false touch responses. According to the determination result, the corresponding screen function is executed. This process ensures the correct response of the system to different types of touch events and improves the user operation experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1It is a flow chart of the steps of a gesture recognition method based on area division in the present invention.
[0051] Figure 2 It is a flow chart of the steps of determining continuity based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area in the present invention.
[0052] Figure 3 It is a flowchart of the steps of adjusting the position and size of the writing area and the gesture recognition area according to the user usage data in the present invention. DETAILED DESCRIPTION
[0053] See also Figure 1-3 As shown, the present invention relates to a gesture recognition method based on area division, comprising:
[0054] Divide the writing area and gesture recognition area according to the screen resolution;
[0055] Get touch events in the writing area and gesture recognition area;
[0056] Jointly judge the touch events and match the execution functions.
[0057] In some embodiments, the writing area and the gesture recognition area are first divided according to the screen resolution. Specifically, the screen parameters are first obtained, including data such as the screen size and the current resolution, and the gesture recognition area is matched and initialized to a reasonable size according to the screen parameters. This process can be achieved by geometric scaling through the preset area ratio. Based on these parameters, the system initializes the division of the writing area and the gesture recognition area. During the division process, the user's common handwriting habits and screen usage habits can be considered to divide the screen into an area suitable for writing and an area for gesture recognition. This division method can not only improve the accuracy of gesture recognition, but also reduce the probability of the user accidentally triggering gesture recognition during the writing process. With the accumulation of user usage data, the system can dynamically adjust the position and size of the writing area and the gesture recognition area to better meet the user's personalized usage needs. After the division is completed, the system obtains the touch events of the writing area and the gesture recognition area through the listener. The listener monitors the occurrence of touch events in real time, including information such as the occurrence coordinates of the touch events, the touch pressure and the touch time. The system classifies the touch events into writing area touch events or gesture recognition area touch events according to the occurrence coordinates and the area to which the touch events belong. The purpose of this step is to clarify the initial area of the touch event and ensure that the system can distinguish between writing operations and gesture operations when processing touch events. Next, the touch events are jointly determined and matched for execution. Based on the preset rule library, the system combines the type of touch event and the area where it occurs to determine the operation purpose of the touch event and calculate its confidence score. Specifically, the system first analyzes the initial touch point, movement trajectory and end touch point of the touch event to calculate the vector of the touch event. Then, by calculating the cosine similarity between the touch event vector and the touch rules in the preset rule library, the operation purpose is matched and the confidence score is calculated. The system also considers the continuity of the touch event, and determines whether it belongs to the same touch event through the timestamps and coordinates of the touch events in the writing area and the gesture recognition area. If the touch event belongs to the same touch event, the system selects a single operation purpose according to the confidence score and executes the corresponding screen function; if it does not belong to the same touch event, the operation purpose is selected according to the confidence score and the corresponding screen function is executed.
[0058] Furthermore, dividing the writing area and the gesture recognition area according to the screen resolution includes:
[0059] Get screen parameters, including size data and current resolution settings;
[0060] Initialize according to screen parameters and divide the writing area and gesture recognition area;
[0061] Adjust the position and size of the writing area and gesture recognition area based on user usage data.
[0062] In some embodiments, first, screen parameters are obtained, including size data and current resolution settings. When the system is started, the physical size (such as width and height) of the device screen and the current resolution setting are obtained by calling the API of the operating system or directly accessing the hardware interface. Different screen sizes and resolutions will affect the user's operating experience and the accuracy of gesture recognition. Secondly, the writing area and gesture recognition area are divided according to the screen parameters. In this process, the system first determines the position and size of the initial writing area and gesture recognition area according to the physical size and resolution of the screen. Specifically, the system can adopt one of the following methods or a combination of multiple methods: Proportional division method: divide the screen into a writing area and a gesture recognition area according to a certain ratio. For example, the lower half of the screen can be set as the writing area, and the upper half can be set as the gesture recognition area. Boundary division method: set the gesture recognition area at the edge of the screen, and the central part can be set as the writing area. This method can effectively prevent users from accidentally triggering gesture operations when writing. User preference setting method: allow users to customize the position and size of the writing area and gesture recognition area, and divide them according to the user's settings. After completing the initial area division, the system adjusts the position and size of the writing area and gesture recognition area according to the user's usage data. Analyze users' operating habits and preferences by continuously monitoring and recording users' touch operation data.
[0063] Furthermore, the step of acquiring touch events in the writing area and the gesture recognition area includes:
[0064] Monitor the touch event through the listener to obtain the coordinates of the touch event and the type of the touch event;
[0065] According to the occurrence coordinates and the area where the touch event occurs, the touch events in the writing area and the touch events in the gesture recognition area are classified.
[0066] First, the touch event is monitored through the listener to obtain the coordinates of the touch event and the type of the touch event. When the system is initialized, multiple listeners are set, and the code is inserted to implement the monitoring, which is used to monitor the touch events in the writing area and the gesture recognition area respectively. The listener can be implemented by calling the API provided by the operating system or directly accessing the hardware interface to monitor all touch operations on the touch screen in real time. The listener not only records the coordinates of the touch event (that is, the position of the touch point on the screen), but also records the type of touch event. The types of touch events usually include ordinary touch and pressure touch. Ordinary touch events refer to touch operations performed by users with normal pressure, while pressure touch events refer to touch operations performed by users with a force exceeding the preset pressure threshold. The information of these two types of touch events is very important for subsequent gesture recognition and processing. In actual applications, when the user operates on the touch screen, the listener will capture the touch event in real time. For example, when the user writes in the writing area, the listener will record the coordinates of each touch point and mark it as an ordinary touch event. If the user performs a sliding gesture operation in the gesture recognition area, the listener will also record the coordinates of each touch point, but will mark it as a normal touch event or a pressure touch event depending on the operation force. Secondly, according to the coordinates of the touch event and the area to which it belongs, the touch event of the writing area and the touch event of the gesture recognition area are classified. After receiving the touch event data transmitted by the listener, the system will determine the area to which it belongs based on the coordinates of the touch point, that is, the writing area or the gesture recognition area. The system pre-sets the boundaries of the writing area and the gesture recognition area, and these boundary information is stored in the system's memory for quick search and comparison. Coordinate comparison: The system compares the coordinates of the touch point with the preset area boundaries. If the coordinates of the touch point fall within the boundary of the writing area, the touch event is classified as a touch event of the writing area; if the coordinates of the touch point fall within the boundary of the gesture recognition area, the touch event is classified as a touch event of the gesture recognition area. Type marking: While classifying the touch event, the system marks the touch event according to the type of touch event recorded by the listener. The marking of normal touch events and pressure touch events helps in subsequent gesture recognition and processing.
[0067] Furthermore, the types of touch events include normal touch and pressure touch. If the touch pressure is greater than a preset pressure threshold, the touch event is marked as pressure touch; if the touch pressure is less than or equal to the preset pressure threshold, the touch event is marked as normal touch.
[0068] It should be noted that when the system is initialized, a preset pressure threshold is set to distinguish between normal touch events and pressure touch events. The setting of the pressure threshold can be based on the hardware capabilities of the device and the user's usage habits, and a suitable value can be determined through experiments and debugging. The threshold is stored in the system's configuration file or memory to facilitate the subsequent type determination of touch events. When the user operates on the touch screen, the system obtains the data of the touch event in real time through the listener, including information such as the coordinates of the touch point, touch pressure, and touch time. Specifically, the listener not only records the position of the touch point on the screen, but also detects the pressure applied by the user to the touch point. The detection of touch pressure depends on the hardware sensor of the touch screen, which can accurately measure the user's operating force. After obtaining the touch event data, the system marks the touch event as normal touch or pressure touch based on the comparison result of the touch pressure value and the preset pressure threshold. The specific determination method is as follows: Normal touch: If the touch pressure is less than or equal to the preset pressure threshold, the touch event is marked as normal touch. Normal touch events usually indicate operations such as clicking, sliding, and writing performed by the user with normal force. Pressure touch: If the touch pressure is greater than the preset pressure threshold, the touch event is marked as a pressure touch. A pressure touch event indicates that the user has applied a greater operating force, which is usually used to trigger specific functions or achieve more sophisticated operational control. In some specific embodiments, when the user writes on the electronic whiteboard, the system detects through the listener that the touch pressure value is 30 units (assuming the preset pressure threshold is 50 units), then the system marks the touch event as a normal touch event. Normal touch events trigger the default writing function, and the system converts the recorded touch point coordinates into a writing trajectory and displays it on the screen in real time.
[0069] Furthermore, the joint determination of the touch event includes:
[0070] Based on a preset rule library and according to the type of touch event and the area where it occurs, determine the operation purposes of the touch events in the writing area and the gesture recognition area respectively and calculate their confidence scores; the operation purposes include writing and gesture control, and the preset rule library includes a general rule library and a pressure rule library;
[0071] Based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area, continuity determination is performed to determine whether the touch events in the writing area and the gesture recognition area belong to the same touch event and match and execute screen functions;
[0072] If they belong to the same touch event, a single operation purpose is selected according to the confidence score and the corresponding screen function is executed;
[0073] If they do not belong to the same touch event, the operation purpose is selected according to the confidence score and the corresponding screen function is executed.
[0074] In some embodiments, first, based on the preset rule library and according to the type of touch event and the area where it occurs, the operation purpose of the touch event in the writing area and the gesture recognition area is determined and its confidence score is calculated. The preset rule library is divided into a general rule library and a pressure rule library, which are used to process general touch events and pressure touch events respectively. The general rule library and the pressure rule library store the operation modes and recognition parameters corresponding to various touch events. Specifically, when a touch event occurs, the system first selects the corresponding rule library according to its type (general touch or pressure touch). Then, the system determines the operation purpose of the touch event according to the occurrence area of the touch event (writing area or gesture recognition area) and the matching rules in the rule library. The operation purpose generally includes two categories: writing and gesture control. For example, when the user performs a general touch operation in the writing area, the system extracts relevant rules from the general rule library, and determines that the operation purpose of the touch event is writing according to the starting point, end point and trajectory shape of the touch event. At the same time, the system calculates the confidence score of the touch event to quantify the reliability of the determination. The confidence score is calculated based on the similarity between the touch event and the predefined pattern in the rule library, and is matched using algorithms such as cosine similarity. Based on the continuity judgment result, the corresponding screen function is further matched and executed. If the touch events in the writing area and the gesture recognition area belong to the same touch event, the system will select a single operation purpose and execute the corresponding screen function based on the confidence score, and exclude the redundant touch responses in the other area. For example, if the confidence score indicates that the touch event is more likely to be a writing operation, the system will execute the writing function; if the confidence score indicates that the touch event is more likely to be gesture control, the system will execute the corresponding gesture control function. If the touch events in the writing area and the gesture recognition area do not belong to the same touch event, the system will process these touch events separately and select the corresponding operation purpose and screen function according to their respective confidence scores. For example, when a user performs a pressure touch operation in the gesture recognition area and a normal touch operation in the writing area at the same time, the system will process these two events separately and execute gesture control and writing functions. Through the above method, this embodiment realizes accurate determination and processing of touch events, ensures that the system can correctly distinguish between writing and gesture operations, reduces false triggers, improves the accuracy of gesture recognition, and optimizes the user's operating experience. For example, when a user writes and performs gesture operations on an electronic whiteboard, the system can accurately distinguish these operations and respond to the user's instructions in real time, providing a smooth interactive experience.
[0075] Further, the executing the corresponding screen function includes:
[0076] If the touch event is a normal touch and the operation purpose is writing, the default writing is performed;
[0077] If the touch event is a pressure touch and the operation purpose is writing, bold writing is performed;
[0078] If the touch event is a normal touch and the operation purpose is gesture control, the normal rule library is matched to respond to the operation;
[0079] If the touch event is a pressure touch and the operation purpose is pressure control, the pressure rule library is matched to respond to the operation.
[0080] It should be noted that after the system determines the operation purpose and type of the touch event, it enters the stage of executing the corresponding screen function. According to the type of touch event and the determination result, the system selects different response operations. First, if the touch event is a normal touch and the operation purpose is writing, the system executes the default writing function. The default writing function is applicable to writing operations performed by users with normal pressure, including text input and graphic drawing. The system generates writing content according to the trajectory of the touch point and displays it on the screen in real time. For example, when the user writes normally on the electronic whiteboard, the system captures the moving trajectory of the touch point, converts it into writing content, and displays the user's writing results on the screen. Secondly, if the touch event is a pressure touch and the operation purpose is writing, the system executes the bold writing function. The bold writing function is applicable to writing operations performed by users with greater pressure, usually used to emphasize or highlight certain content. The system adjusts the thickness of the writing handwriting according to the touch pressure value and displays the bold writing content on the screen. For example, when the user writes heavily on the electronic whiteboard, the system detects a greater touch pressure, converts it into bold writing handwriting, and displays the user's bold writing results on the screen. Next, if the touch event is a normal touch and the purpose of the operation is gesture control, the system matches the normal rule library to respond to the operation. Normal gesture control includes basic gesture operations such as sliding, zooming, and rotating. The system recognizes and responds according to the preset normal rule library. For example, when the user performs a normal sliding gesture in the gesture recognition area, the system recognizes the sliding direction and distance, and performs the corresponding screen switching or page scrolling function. Finally, if the touch event is a pressure touch and the purpose of the operation is gesture control, the system matches the pressure rule library to respond to the operation. Pressure gesture control is suitable for special gesture operations performed by users applying greater pressure, and is usually used to trigger advanced functions or shortcut operations. The system recognizes and responds according to the preset pressure rule library. For example, when the user performs a pressure sliding gesture in the gesture recognition area, the system recognizes the sliding direction, distance, and pressure value, and performs corresponding shortcut operations, such as opening an application, launching a shortcut menu, etc.
[0081] Furthermore, the determining the operation purposes of the touch events in the writing area and the gesture recognition area and calculating the confidence scores thereof respectively according to the type of the touch event and the area where the touch event occurs includes:
[0082] Calculating a touch event vector based on an initial touch point, a moving trajectory, and an end touch point of the touch event;
[0083] The cosine similarity between the touch event vector and the touch rules in the preset rule library is calculated, and the operation purpose is matched according to the cosine similarity and the confidence score is calculated.
[0084] In some embodiments, first, the system collects detailed data of the touch event in real time through a listener, including the initial touch point, the moving trajectory, and the end touch point. The initial touch point is the position at the beginning of the touch event, the moving trajectory is the path during the touch event, and the end touch point is the position at the end of the touch event. These data are collected in real time by the sensor of the touch screen and the listener of the system, and recorded in the event data cache of the system. In the specific implementation process, when the user operates on the touch screen, the system records the initial point, multiple intermediate points, and the end point of each touch event, and records the touch pressure and the touch timestamp. These data constitute complete touch event information. Next, based on these data, the system calculates the touch event vector. The touch event vector is a multidimensional vector that describes the spatial behavior of the touch event, usually including parameters such as the position information of the touch point, the touch pressure, and the touch speed. Then the calculated touch event vector is compared with the touch rule vector in the preset rule library. The preset rule library stores standard vectors of various touch events. These standard vectors are obtained through training and summarization of a large amount of actual touch data, and can accurately describe common writing and gesture operation modes. By calculating the cosine similarity between the touch event vector and each rule vector, the system can determine which rule the touch event is most similar to. The higher the cosine similarity, the higher the degree of match between the touch event and the rule. Based on the calculated cosine similarity, the system calculates a confidence score for each possible operation purpose. The confidence score indicates the reliability of the system's judgment on the operation purpose of the touch event. The value range is usually between 0 and 1, and the higher the value, the higher the confidence. For example, in a specific application scenario, the user performs a touch operation in the writing area, and the system records the initial touch point (100, 200), the movement trajectory (from 100, 200 to 150, 250), and the end touch point (150, 250). The system converts these data into a touch event vector, and then calculates the cosine similarity between the vector and the standard writing vector and gesture vector in the rule library. Assume that the calculation results are: the cosine similarity with the standard writing vector is 0.95, and the cosine similarity with the gesture vector is 0.40. Based on these similarity values, the system determines that the operation purpose of the touch event is writing, and sets the confidence score to 0.95. Through the above method, this embodiment realizes the accurate determination of the operation purpose of the touch event, and calculates the confidence score through cosine similarity, ensuring that the system can accurately identify the user's intention and perform the corresponding operation. This method not only improves the recognition accuracy of the system, but also enhances the user experience, enabling users to achieve more sophisticated and diverse operations through touch.
[0085] Furthermore, the continuity determination based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area includes:
[0086] Determine the sequence of touch events in the writing area and the gesture recognition area based on the start timestamps and end timestamps of the touch events in the writing area and the gesture recognition area respectively;
[0087] Calculate the time difference by the end timestamp of the first touch event and the start timestamp of the second touch event;
[0088] Calculate the distance difference between the end coordinates of the first touch event and the start coordinates of the second touch event;
[0089] The time difference and distance difference are weighted to determine continuity.
[0090] Furthermore, the formula for determining the continuity based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area is as follows:
[0091]
[0092] Among them, C is the continuity score; T e and T s are the end timestamp of the first touch event and the start timestamp of the second touch event respectively; (X e , Y e ) is the end coordinate of the first touch event; (X s , Y s ) is the starting coordinate of the back touch event; ΔT max and ΔD max They are respectively a preset time threshold and a preset distance threshold; w1 and w2 are weighting coefficients.
[0093] In some specific embodiments, for example, in a specific application scenario, the user completes a touch operation in the writing area, and the end timestamp is T e =10:00:01, the ending coordinate is (X e , Y e )=(150,200). Then the user starts a new touch operation in the gesture recognition area, and its starting timestamp is T s =10:00:02, the starting coordinate is (X s , Y s )=(155,205). The system first calculates the time difference: ΔT is 1 second, ΔD=7.07; then by preset ΔT max and ΔD max The continuity score C is calculated using w1 as the denominator and w2 as weighting coefficients.
[0094] Further, the determining whether the touch events in the writing area and the gesture recognition area respectively belong to the same touch event includes:
[0095] If the continuity score C is greater than or equal to the preset continuity threshold, it is determined to belong to the same touch event;
[0096] If the continuity score C is less than or equal to the preset continuity threshold, it is determined that they do not belong to the same touch event.
[0097] Specifically, if the continuity score C is greater than or equal to the preset continuity threshold, it is determined that the touch events in the writing area and the gesture recognition area belong to the same touch event. This means that these touch events are a continuous operation of the user and should be processed as a whole. For example, on an electronic whiteboard, the user may accidentally swipe into the gesture operation area after completing the writing operation. The system recognizes the continuity of these operations and does not perform gesture processing but only processes them as writing operations. If the continuity score C is less than the preset continuity threshold, the system determines that the touch events in the writing area and the gesture recognition area do not belong to the same touch event. This means that these touch events are independent and should be processed separately. For example, the user performs a gesture operation again after a period of time after the writing operation. The system recognizes that the interval between the two operations is long, so it treats them as independent events.
[0098] Furthermore, adjusting the position and size of the writing area and the gesture recognition area according to the user usage data includes:
[0099] Extracting touch events whose continuity determination results are the same touch event and whose operation purpose is gesture control, and obtaining the touch event vectors thereof in the non-gesture recognition area, and marking them as false touch event vectors;
[0100] Perform cluster analysis on the false touch event vectors to determine the high-frequency occurrence area of the touch event vectors;
[0101] Adjust the position and size of the gesture recognition area based on the high-frequency occurrence area.
[0102] In some embodiments, first, the system extracts touch events whose continuity determination results are the same touch event and whose operation purpose is gesture control. These touch events indicate that the user may have accidentally touched the writing area or other non-gesture recognition areas when performing gesture operations. By analyzing these events, the system can identify the problem areas in the user's actual operation. In specific operations, the system records the data of all touch events, including the start point, end point, touch pressure, touch time, etc. of the event. When the system determines that some touch events are gesture controlled and have continuity with touch events in non-gesture recognition areas, these touch events will be extracted and marked as false touch events. The system will create vectors for these false touch events and record their detailed features. Next, the system performs cluster analysis on the false touch event vectors to determine the high-frequency occurrence area of the touch event vectors. Cluster analysis is an important method in data mining. By clustering the false touch event vectors according to their spatial features, the system can identify the high-frequency areas of false touches in user operations. For example, the system can use the K-means clustering algorithm to divide the false touch event vectors into several clusters, each cluster representing a high-frequency occurrence area. After the cluster analysis is completed, the system adjusts the position and size of the gesture recognition area according to the high-frequency occurrence area. The specific adjustment method is as follows: Identify the boundary of the high-frequency area: The system determines the boundary of each high-frequency area according to the distribution of the cluster center vector and the false touch events in the cluster. Adjust the gesture recognition area: The system reduces the occurrence of false touch events by expanding the scope of the gesture recognition area to cover the high-frequency false touch area. For example, if the high-frequency false touch area is concentrated in the lower right corner of the screen, the system can expand the gesture recognition area to this area to ensure that the user's gesture operation is accurately recognized. Dynamic update: The system regularly collects data and clusters analysis of false touch events, and dynamically adjusts the position and size of the gesture recognition area to adapt to the user's operating habits and changes. For example, in a specific application scenario, when the user uses an electronic whiteboard for gesture operation, the system detects that the user frequently has false touch events in the upper right corner. Through cluster analysis, the system determines that this area is a high-frequency false touch area. In order to reduce false touch events, the system expands the gesture recognition area to the upper right corner, so that the user's operation in this area can be correctly recognized as a gesture operation, rather than a false touch. Through the above method, this embodiment realizes the dynamic adjustment of the position and size of the writing area and the gesture recognition area according to the user's usage data. By analyzing false touch events, the method improves the accuracy of gesture recognition, reduces the occurrence of false touches, optimizes the user's operating experience, and enables the system to better adapt to the user's actual operating needs.
[0103] The present invention also includes a gesture recognition system based on region division, including a region division module, a region monitoring module and a determination execution module;
[0104] The area division module is used to divide the writing area and the gesture recognition area according to the screen resolution;
[0105] The area monitoring module is used to obtain touch events in the writing area and the gesture recognition area;
[0106] The determination and execution module is used to jointly determine the touch events and match the execution functions.
[0107] Specifically, the area division module works in coordination with the touch controller and the data processing unit. The touch controller is responsible for obtaining the physical parameters of the screen and the current resolution setting. Specifically, the touch controller contains a high-precision ADC (analog-to-digital converter) and a screen parameter detection circuit. These hardware components can monitor and obtain information such as the size, resolution, and brightness of the screen in real time. The data processing unit receives these parameters and divides the writing area and the gesture recognition area according to a preset algorithm. The configuration file pre-stored in the memory provides a standard division scheme for different resolutions and screen sizes, and the dynamic adjustment of the area is achieved through a software algorithm. The hardware part of the area monitoring module includes a highly sensitive touch sensor array and an embedded processor. The touch sensor array is arranged on the surface of the screen and can detect the user's touch operation in real time. The embedded processor is responsible for preliminarily processing the data of the sensor array and transmitting the data of the touch event to the data processing unit. These touch event data include information such as the coordinates of the touch point, the touch pressure, and the touch time. The embedded processor transmits the data to the data processing unit through a high-speed bus to ensure that the system can respond to user operations in real time.
[0108] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.
Claims
1. A gesture recognition method based on region division, characterized in that: include: Divide the writing area and gesture recognition area according to the screen resolution; Get touch events in the writing area and gesture recognition area; Jointly judge the touch events and match the execution functions.
2. The method for gesture recognition based on area division according to claim 1, characterized in that: The division of the writing area and the gesture recognition area according to the screen resolution includes: Get screen parameters, including size data and current resolution settings; Initialize according to screen parameters and divide the writing area and gesture recognition area; Adjust the position and size of the writing area and gesture recognition area based on user usage data.
3. The gesture recognition method based on area division according to claim 2, characterized in that: The method of acquiring touch events in the writing area and the gesture recognition area includes: Monitor the touch event through the listener to obtain the coordinates of the touch event and the type of the touch event; According to the occurrence coordinates and the area where the touch event occurs, the touch events in the writing area and the touch events in the gesture recognition area are classified.
4. The method for gesture recognition based on area division according to claim 3, characterized in that: The types of touch events include normal touch and pressure touch. If the touch pressure is greater than a preset pressure threshold, the touch event is marked as pressure touch; if the touch pressure is less than or equal to the preset pressure threshold, the touch event is marked as normal touch.
5. The method for gesture recognition based on area division according to claim 4, characterized in that: The joint determination of the touch event includes: Based on a preset rule library and according to the type of touch event and the area where it occurs, determine the operation purposes of the touch events in the writing area and the gesture recognition area respectively and calculate their confidence scores; the operation purposes include writing and gesture control, and the preset rule library includes a general rule library and a pressure rule library; Based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area, continuity determination is performed to determine whether the touch events in the writing area and the gesture recognition area belong to the same touch event and match and execute screen functions; If they belong to the same touch event, a single operation purpose is selected according to the confidence score and the corresponding screen function is executed; If they do not belong to the same touch event, the operation purpose is selected according to the confidence score and the corresponding screen function is executed.
6. The method for gesture recognition based on area division according to claim 5, characterized in that: The execution of the corresponding screen function includes: If the touch event is a normal touch and the operation purpose is writing, the default writing is performed; If the touch event is a pressure touch and the operation purpose is writing, bold writing is performed; If the touch event is a normal touch and the operation purpose is gesture control, the normal rule library is matched to respond to the operation; If the touch event is a pressure touch and the operation purpose is pressure control, the pressure rule library is matched to respond to the operation.
7. The method for gesture recognition based on area division according to claim 5, characterized in that: The step of determining the operation purposes of the touch events in the writing area and the gesture recognition area and calculating the confidence scores thereof according to the type of the touch event and the area where the touch event occurs comprises: Calculating a touch event vector based on an initial touch point, a moving trajectory, and an end touch point of the touch event; The cosine similarity between the touch event vector and the touch rules in the preset rule library is calculated, and the operation purpose is matched according to the cosine similarity and the confidence score is calculated.
8. The method for gesture recognition based on area division according to claim 7, characterized in that: The continuity determination based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area includes: Determine the sequence of touch events in the writing area and the gesture recognition area based on the start timestamps and end timestamps of the touch events in the writing area and the gesture recognition area respectively; Calculate the time difference by the end timestamp of the first touch event and the start timestamp of the second touch event; Calculate the distance difference between the end coordinates of the first touch event and the start coordinates of the second touch event; The time difference and distance difference are weighted to determine continuity.
9. The method for gesture recognition based on area division according to claim 8, characterized in that: The formula for determining continuity based on the touch event occurrence timestamps and coordinates of the writing area and the gesture recognition area is as follows: Among them, C is the continuity score; T e and T s are the end timestamp of the first touch event and the start timestamp of the second touch event respectively; (X e , Y e ) is the end coordinate of the first touch event; (X s ,Y s ) is the starting coordinate of the back touch event; ΔT max and ΔD max They are respectively a preset time threshold and a preset distance threshold; w1 and w2 are weighting coefficients.
10. The method for gesture recognition based on area division according to claim 9, characterized in that: The determining whether the touch events in the writing area and the gesture recognition area belong to the same touch event comprises: If the continuity score C is greater than or equal to the preset continuity threshold, it is determined to belong to the same touch event; If the continuity score C is less than or equal to the preset continuity threshold, it is determined that they do not belong to the same touch event.
11. The method for gesture recognition based on area division according to claim 10, characterized in that: The adjusting the position and size of the writing area and the gesture recognition area according to the user usage data includes: Extracting touch events whose continuity determination results are the same touch event and whose operation purpose is gesture control, and obtaining the touch event vectors thereof in the non-gesture recognition area, and marking them as false touch event vectors; Perform cluster analysis on the false touch event vectors to determine the high-frequency occurrence area of the touch event vectors; Adjust the position and size of the gesture recognition area based on the high-frequency occurrence area.
12. A gesture recognition system based on region division, applied to a gesture recognition method based on region division as claimed in any one of claims 1 to 11, characterized in that: It includes a region division module, a region monitoring module and a determination execution module; The area division module is used to divide the writing area and the gesture recognition area according to the screen resolution; The area monitoring module is used to obtain touch events in the writing area and the gesture recognition area; The determination and execution module is used to jointly determine the touch events and match the execution functions.
