Pressure bracelet gesture learning and recognition method and device

Abstract

This invention provides a method for learning and recognizing gestures on a pressure wristband, comprising: training a center vector based on pressure data and constructing a gesture library, including: making a gesture, collecting pressure data from various pressure sensors, generating sample data based on the pressure data; repeating the gesture to generate multiple sample data; generating an initial center vector based on the sample data; using the initial center vector as a standard, filtering the sample data to obtain final sample data and generating the final center vector; generating the final center vector for each gesture one by one to construct the gesture library; recognizing the gestures: collecting pressure values ​​from various pressure sensors, with the pressure values ​​forming a real-time vector; determining whether a dexterity gesture has switched based on confidence level; determining whether a dexterity gesture has switched based on the distance between the current vector and the real-time vector; determining the target gesture for dexterity gesture switching based on multi-point voting; and switching to the target gesture when the first, second, and third conditions are met simultaneously.

Description

Technical Field

[0001] This invention relates to the field of dexterity technology, specifically to a method and device for learning and recognizing hand gestures using a pressure wristband. Background Technology

[0002] The dexterous hand is worn as a prosthesis on the disabled arm to assist it in performing certain movements, such as grasping objects. During these movements, it is crucial to accurately capture the arm's motion information and control the dexterous hand accordingly. To achieve this, the dexterous hand needs to learn the arm's motion patterns, identify the arm's motion information based on these patterns, determine the target movement, and achieve accurate motor control. Summary of the Invention

[0003] The purpose of this invention is to provide a pressure wristband gesture learning and recognition method for recognizing dexterous hand gestures based on collected arm pressure information.

[0004] On the one hand, a method for gesture learning and recognition using a pressure wristband is provided. The pressure wristband has multiple built-in pressure sensors. The wristband is worn on the outer periphery of the arm, and the pressure sensors contact the arm directly or through a medium. The method includes the following steps: Based on pressure data collected by pressure sensors, the center vectors of various gestures are trained, and a gesture library is constructed, including: When a gesture is made, each pressure sensor collects pressure data, and a sample data is generated based on the pressure data collected by each pressure sensor; repeating the same gesture generates multiple sample data. Generate the initial center vector of a gesture based on all sample data of the gesture; Using the initial center vector of the gesture as a standard, the sample data of the gesture is filtered to obtain the final sample data of the gesture; the final center vector of the gesture is regenerated based on the final sample data of the gesture. Generate the final center vector corresponding to each gesture one by one to build a gesture library; Gesture recognition and execution specifically include: During dexterity hand movement, pressure values ​​from various parts of the arm are collected simultaneously by various pressure sensors, and the pressure values ​​form a real-time vector. The first condition is to determine whether the dexterity hand gesture has been switched based on confidence level; The second condition is to determine whether the dexterous hand gesture is switched based on the distance between the current vector and the real-time vector; The third condition is to determine the target gesture for dexterous hand gesture switching based on multi-point voting. When the first, second, and third conditions for dexterous hand gesture switching are met simultaneously, the dexterous hand switches to the target gesture.

[0005] On the other hand, it provides a pressure wristband gesture learning and recognition device, including: A gesture library module is built to train the center vectors of various gestures based on pressure data collected by a pressure sensor, and to construct the gesture library, including: The sample data generation unit is used to generate a sample data based on the pressure data collected by each pressure sensor when a gesture is made; and to generate multiple sample data by repeating the same gesture. Generate the initial center vector unit, which is used to generate the initial center vector of a gesture based on all sample data of a gesture; A gesture library unit is constructed to filter the sample data of the gesture based on the initial center vector of the gesture to obtain the final sample data of the gesture; to regenerate the final center vector of the gesture based on the final sample data of the gesture; and to construct the gesture library by generating the final center vector corresponding to each gesture one by one. The recognition and execution module, used for recognizing and executing gestures of dexterous hands, includes: The real-time vector unit is used to simultaneously collect pressure values ​​from various parts of the arm through various pressure sensors during dexterity hand movements, and the pressure values ​​constitute a real-time vector. The confidence level judgment unit, with the first condition, is used to determine whether the dexterity hand gesture has been switched based on confidence level. The distance judgment unit has a second condition: it determines whether the dexterity hand gesture has been switched based on the distance between the current vector and the real-time vector. The third condition, based on multi-point voting judgment, is to determine the dexterous hand's target gesture switching based on multi-point voting judgment. An execution unit is used to switch the dexterous hand to the target gesture when the first, second, and third conditions for dexterous hand gesture switching are met simultaneously.

[0006] The advantages of this invention are as follows: (1) By judging whether the dexterous hand gesture has switched based on confidence, the model can accurately distinguish the gesture. The larger the difference between the best candidate value and the second best candidate value, the more confident the neural network model is in distinguishing the two. The smaller the difference, the more the neural network model hesitates between the two gesture categories. When the difference between the best candidate value and the second best candidate value is not large enough, the model does not rush to switch the output, but maintains the previous output result. That is, the dexterous hand keeps the current gesture still, thereby avoiding frequent shaking.

[0007] (2) Determine whether the dexterous hand gesture has switched based on the distance between the current vector and the real-time vector. By comparing the difference in the distance between the gestures in two consecutive frames and their respective best matching centers, the absolute distance judgment is transformed into a relative distance improvement judgment, thus cleverly introducing the inertia of state preservation, which greatly enhances the stability and robustness of the system in noisy environments and effectively prevents false switching and jitter.

[0008] (3) Based on multi-point voting, the system determines the target gesture for dexterous hand gesture switching. The system does not rely on the single sensor's vote, but makes the final decision through multi-sensor voting. The incorrect recognition of a few sensors can be covered by the votes of other correctly recognized sensors, thus ensuring the stability of the entire system when facing local sensor failure. Attached Figure Description

[0009] Figure 1 This is a flowchart of the gesture learning and recognition method for pressure wristbands.

[0010] Figure 2 This is a flowchart of a gesture library built by training the center vectors of various gestures based on pressure data collected by pressure sensors.

[0011] Figure 3 This is a flowchart for gesture recognition. Detailed Implementation

[0012] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] In the description of this invention, it should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" used to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0014] A sleeve is connected to the tail of the dexterous hand. The sleeve is similar in shape to the arm. A pressure wristband is arranged on the inner side of the sleeve. The pressure wristband has multiple pressure sensors built in. After the sleeve is worn on the arm, the pressure wristband is worn on the outer periphery of the arm. The pressure sensors contact the arm directly or through a medium. For example, the pressure sensors are arranged in the interlayer of the sleeve. The pressure sensors collect the pressure changes caused by the movement of the arm through the inner wall of the sleeve.

[0015] The pressure wristband gesture learning and recognition method includes the following steps: Step S1: Based on the pressure data collected by the pressure sensor, train the center vectors of each gesture and construct a gesture library, including: Step S11: When a gesture is made, each pressure sensor collects pressure data, and a sample data is generated based on the pressure data collected by each pressure sensor; the same gesture is made repeatedly to generate multiple sample data.

[0016] Step S12: Generate the initial center vector of a gesture based on all sample data of the gesture.

[0017] Step S13: Using the initial center vector of the gesture as the standard, filter all sample data of the gesture to obtain the final sample data of the gesture; regenerate the final center vector of the gesture based on the final sample data of the gesture.

[0018] Generate the final center vector corresponding to each gesture one by one to build a gesture library.

[0019] Step S2, recognizing the gesture, specifically includes: Step S21: During dexterity hand movement, pressure values ​​of multiple parts of the arm are collected simultaneously through various pressure sensors. Step S22, the first condition, determine whether the dexterity hand gesture has been switched based on confidence level; Step S23, the second condition, determines whether the dexterous hand gesture is switched based on the distance between the current vector and the real-time vector; Step S24, the third condition, determines the switching of dexterous hand gestures based on multi-point voting; Step S3: When the first, second, and third conditions for dexterous hand gesture switching are met simultaneously, the dexterous hand switches to the target gesture.

[0020] In this embodiment, step S11, when a gesture is made, each pressure sensor collects pressure data, and a sample data is generated based on the pressure data collected by each pressure sensor; repeating the same gesture is used to generate multiple sample data, specifically: Make a gesture and hold it for the required sampling time. The pressure sensor samples at a preset frequency. Calculate the average value of all data collected by the pressure sensor within the specified sampling time period, which is taken as the pressure value collected by the pressure sensor during this sampling process.

[0021] Based on this, the pressure values ​​collected by each pressure sensor in the pressure wristband are calculated, and each pressure value constitutes the sample data of this sampling.

[0022] For example, a hand gesture is made and held for 2 seconds at a preset frequency of 100 Hz. The pressure sensor collects pressure information at a frequency of 100 Hz, collecting a total of 200 data points. The average of these 200 data points is calculated, and the resulting average value is the pressure value at the corresponding position on the arm collected by the pressure sensor in this acquisition.

[0023] The pressure wristband in this embodiment is equipped with 16 pressure sensors, and the pressure values ​​collected by each pressure sensor are obtained through the above method.

[0024] Each pressure value constitutes the sample data for this sampling. For example, the sample data for this collection is generated using the pressure values ​​of each pressure sensor collected in a single sampling. The sample data [Pi1,Pi2,...,Pi16] is obtained by collecting pressure values ​​from 16 pressure sensors, where i indicates the sampling number, and the sample data is a 16-dimensional vector.

[0025] For each repetition of a gesture, the collected data generates a 16-dimensional vector. Similarly, repeating the gesture multiple times generates multiple 16-dimensional vectors. For example, repeating the sampling 10 times generates 10 16-dimensional vectors. [P11,P12,...,P116],...,[P101,P102,...,P1016].

[0026] In this embodiment, step S12, generating the initial center vector of a gesture based on all sample data of a gesture, specifically involves: For all sample data collected for a gesture, the average value of all sample data collected by a sensor is calculated to obtain a parameter of a center vector; for each pressure sensor, the average value of all sample data collected by a single pressure sensor is calculated to obtain each parameter of the center vector, and each parameter is used to construct the initial center vector of the gesture.

[0027] A single sample data obtained from a single collection is represented as: [Pi1,Pi2,...,PiM].

[0028] The average value of all sample data collected by a sensor is (ΣPiX) / N.

[0029] In this case, a single gesture collects N sample data points, where i represents the pressure value collected by the pressure sensor in the i-th instance, X represents the X-th pressure sensor, and M is the number of pressure sensors.

[0030] The initial center vector of the gesture is C = [(ΣPi1) / N, (ΣPi2) / N, ..., (ΣPiM) / N].

[0031] In this embodiment, step S13, using the initial center vector of the gesture as a standard, filters the sample data of the gesture to obtain the final sample data of the gesture, specifically as follows: A preset distance threshold is defined, which specifies the maximum allowable deviation between the qualified sample data and the initial center vector.

[0032] For a gesture, calculate the Euclidean distance from each sample data of the gesture to the initial center vector. This distance represents the degree of difference between the sample data and the standard gesture.

[0033] When the Euclidean distance of a sample data point to the initial center vector is greater than a preset distance threshold, it indicates a significant deviation from the target data and is considered an outlier; therefore, the corresponding sample data is removed. When the Euclidean distance of a sample data point to the initial center vector is not greater than the preset distance threshold, it indicates that the sample data is acceptable and is retained. Through this method, outlier sample data is removed, and all retained sample data constitute the final sample dataset for the gesture.

[0034] A sample data of a gesture is represented as: [Pi1,Pi2,...,PiM].

[0035] The center vector of the gesture is C = [(ΣPi1) / N, (ΣPi2) / N, ..., (ΣPiM) / N].

[0036] The Euclidean distance from the sample data to the initial center vector is: L=sqrt((Pi1-(ΣPi1) / N)^2 +...+(PiM-(ΣPiM) / N)^2).

[0037] Using the method in step S12, the final center vector of the gesture is generated using the final sample data of the gesture obtained after filtering.

[0038] Generate the final center vector corresponding to each gesture one by one to build a gesture library.

[0039] In this embodiment, step S22, determining whether a dexterous hand gesture has switched based on confidence level, specifically involves: The principle and steps of the model for judging gesture similarity through neural networks are the same as those in step S24, which involves training a classification model for each sensor using a neural network. Therefore, they will not be elaborated here.

[0040] The model for judging gesture similarity based on neural networks compares the real-time vector with the center vector of each gesture to determine the center vector that is closest to the real-time vector as the first center vector and the center vector that is second closest to the real-time vector as the second center vector.

[0041] Regarding the process of calculating the best candidate value and the second best candidate value, this embodiment uses the cosine similarity function to calculate the similarity between two vectors. The similarity between the real-time vector and each vector in the gesture library is calculated respectively. The vector with the highest similarity to the real-time vector is the closest, and its corresponding center vector is the first center vector. The vector with the second highest similarity to the real-time vector is the second closest, and its corresponding center vector is the second center vector.

[0042] The similarity between the real-time vector and the first center vector is represented as the best candidate value.

[0043] The similarity between the real-time vector and the second center vector is represented as the second-best candidate value.

[0044] Preset first threshold.

[0045] The confidence level of gesture switching is obtained by calculating the difference between the best and second-best candidate values.

[0046] If the confidence level of the gesture switching is less than the first threshold, it means that the model that judges the gesture similarity through the neural network cannot clearly distinguish the current gesture and hesitates between the gestures corresponding to the first center vector or the second center vector. In this case, the gesture of the dexterous hand remains unchanged.

[0047] If the confidence level of gesture switching is greater than the first threshold, it means that the model that judges gesture similarity through the neural network can clearly identify the gesture indicated by the best candidate value, thus satisfying the first condition for gesture switching.

[0048] The larger the difference between the best candidate value and the second-best candidate value, the more confident the neural network model is in distinguishing between the two. The smaller the difference, the more the neural network model hesitates between the two gesture categories. When the difference between the best candidate value and the second-best candidate value is not large enough, it does not rush to switch the output, but maintains the previous output result. That is, the dexterous hand keeps the current gesture still, thereby avoiding frequent shaking.

[0049] Taking a two-dimensional vector constructed from sample data from two pressure sensors as an example: Real-time vector, which is a vector generated from the pressure data collected by each pressure sensor during the judgment process.

[0050] At the current moment, the real-time vector is: R = [0.6, 0.5].

[0051] Center vector: The center vector corresponding to the clenched fist gesture is [1,0], and the center vector corresponding to the open hand gesture is [0,1].

[0052] The first threshold is set to 0.3. The cosine similarity function is used to calculate the similarity between two vectors. The larger the similarity value, the closer the dexterous hand's current gesture is to the standard gesture corresponding to the center vector. The maximum similarity value is 1.

[0053]

[0054]

[0055] Clenched fist gesture: SA=0.768, open hand gesture: SB=0.640.

[0056] The best candidate value is 0.768, the second best candidate value is 0.640, and the confidence level is the difference between the best and second best candidate values, which is 0.128. The confidence level of 0.128 is less than the first threshold of 0.3, which means that the neural network model cannot clearly determine which gesture in the gesture library corresponds to the real-time vector, and the dexterous hand keeps the gesture unchanged.

[0057] For example, at the next moment, the real-time vector is R=[0.95,0.1], and the center vector is: the center vector corresponding to the clenched fist gesture is =[1,0], and the center vector corresponding to the open hand gesture is =[0,1].

[0058] Clenched fist gesture: SA=0.996, open hand gesture: SB=0.095.

[0059] A confidence score of 0.901 > 0.3 indicates that the neural network model can definitively determine that the gesture in the real-time vector corresponds to a fist gesture in the gesture library. This satisfies the first condition for gesture switching, provided the neural network model can accurately recognize the gestures of a dexterous hand.

[0060] In this embodiment, step S23, determining whether the dexterity hand gesture has been switched based on the distance between the current vector and the real-time vector, specifically involves: At the previous moment before the current gesture judgment of the dexterous hand, the vector representing the dexterous hand gesture is the current vector; at the same time, the center vector corresponding to the dexterous hand gesture at the previous moment is denoted as the current center vector; the distance between the current vector and the current center vector is calculated as the current distance, and the distance is calculated using Euclidean distance.

[0061] When judging the gesture of a dexterous hand, the vector representing the dexterous hand gesture is the real-time vector. The center vector that is closest to the real-time vector in the gesture library is determined by the neural network, which is the real-time center vector. The distance between the real-time vector and the real-time center vector is calculated as the real-time distance, which is calculated using Euclidean distance.

[0062] Preset a second threshold; If the difference between the current distance and the real-time distance is less than the second threshold, the dexterous hand maintains the current gesture.

[0063] If the difference between the current distance and the real-time distance is greater than the second threshold, then the second condition for dexterous hand gesture switching is met, and the gesture is prepared to be switched to the gesture corresponding to the real-time center vector.

[0064] For example, suppose the gesture library has two gestures: the center vector of the clenched fist gesture is [1.0, 0.0], and the center vector of the open palm gesture is [0.0, 1.0]. The second threshold is 0.3.

[0065] Scenario 1: The gesture was previously identified as a clenched fist, and it still closely resembles a clenched fist gesture.

[0066] The previous moment, corresponding to the current state: The corresponding current vector is [1.1, 0.2], and the corresponding current center vector is [1.0, 0.0].

[0067] Current distance = sqrt((1.1-1.0)²+(0.2-0.0)²)≈0.224.

[0068] The moment of determining dexterity hand gestures corresponds to the real-time state: The real-time vector of the real-time state is [1.15, 0.25], and the real-time center vector matched by the real-time state is [1.0, 0.0].

[0069] Real-time distance = sqrt((1.15-1.0)²+(0.25-0.0)²)≈0.291.

[0070] Difference = Current distance − Real-time distance = 0.224 − 0.291 = −0.067, −0.067 < 0.3, maintain the current fist gesture.

[0071] Scenario 2: The previous recognition was a clenched fist gesture, but now it is closer to an open palm gesture.

[0072] The previous moment, corresponding to the current state: Current vector = [1.3, 0.6], current fist-clenching gesture center vector = [1.0, 0.0] Current distance = sqrt((1.3-1)²+(0.6-0)²)≈0.671.

[0073] The moment of determining dexterity hand gestures corresponds to the real-time state: Real-time vector = [0.05, 0.95], real-time center vector [0.0, 1.0] Real-time distance = sqrt((0.05-0)²+(1.0-0.95)²)≈0.071.

[0074] The difference = current distance - real-time distance = 0.671 - 0.071 = 0.6, 0.6 ≥ 0.3, which meets the gesture switching condition and allows switching to the open palm gesture.

[0075] By comparing the difference in distance between two consecutive frames of gestures and their respective best matching centers, the absolute distance judgment is transformed into a relative distance improvement judgment, thus cleverly introducing the inertia of state preservation, which greatly enhances the stability and robustness of the system in noisy environments and effectively prevents accidental switching and jitter.

[0076] In this embodiment, step S24, the determination of dexterous hand gesture switching based on multi-point voting, specifically includes: Pressure sensor data acquisition and feature extraction: For each pressure sensor, a corresponding set of feature vectors is established for each gesture. For each pressure sensor, a classification model is trained, including: for a pressure sensor, using gesture and feature vector set as input parameters, a classification model is trained for the pressure sensor through a neural network; and corresponding classification models are trained for each pressure sensor. The current gesture is identified by a classification model using multiple pressure sensors, and the identification results are statistically analyzed to determine the target gesture.

[0077] In this embodiment, the pressure sensor data acquisition and feature extraction, for each pressure sensor, establishes a corresponding feature vector set for each gesture, specifically including: The first step is to acquire the pressure time series data of a pressure sensor during a single gesture, extract pressure feature values ​​based on the pressure time series data, and construct a feature vector of the pressure sensor for the gesture. The second step is to collect multiple sample data of a single gesture for a pressure sensor. Each sample data corresponds to a feature vector, and the generated feature vectors form a set of feature vectors corresponding to the gesture. The third step is to establish a set of corresponding feature vectors for each gesture using a pressure sensor. For each pressure sensor, the first, second, and third steps are executed sequentially, and a corresponding feature vector set is established for each gesture.

[0078] For example, in this embodiment, 16 pressure sensors are built into the dexterous hand. During use, these 16 pressure sensors surround the arm and collect pressure information during gestures. The 16 pressure sensors are: pressure sensor P1, pressure sensor P2, ..., pressure sensor P15, and pressure sensor P16. The gestures trained and recognized are: clenched fist, open palm, and thumbs-up.

[0079] The first step is to extract the feature vector for a single sensor and a single gesture: Take pressure sensor P1, corresponding to a clenched fist gesture, as an example.

[0080] A dexterous hand makes a fist gesture, and for one second, pressure sensor P1 collects pressure data at a frequency of 100 Hz, obtaining a pressure time series data [0.1, 0.5, 1.2, 2.3, 3.0, 3.1, 2.9, ...] Newtons.

[0081] Extracted pressure characteristic values: peak pressure = 3.1 N, average pressure = 2.1 N, pressure rise slope = 3.0 N / s, time to reach peak value = 0.4 s, pressure variance = 0.5.

[0082] The feature vector is formed by arranging these 5 feature values: VP1_clenched fist_1=[3.1,2.1,3.0,0.4,0.5].

[0083] The second step is to collect multiple samples of a single gesture for a single sensor: For example, repeating the fist-clenching gesture 15 times corresponds to obtaining 15 feature vectors: VP1_clenched fist_1=[3.1,2.1,3.0,0.4,0.5],VP1_clenched fist_2=[2.9,2.0,2.8,0.42,0.4],... VP1_clenched fist_15=[3.2,2.2,3.1,0.38,0.6].

[0084] Fifteen vectors constitute the feature vector set of the pressure sensor P1 for the fist-clenching gesture, namely: [VP1_clenching_1,VP1_clenching_2,...,VP1_clenching_15].

[0085] The third step is to establish a corresponding feature vector set for each gesture using a single pressure sensor: For a fist-clenching gesture: the feature vector set of pressure sensor P1 for a fist-clenching gesture is: [VP1_clenching_1,VP1_clenching_2,...,VP1_clenching_15].

[0086] For an open palm gesture: the feature vector set of the pressure sensor P1 for the open palm gesture: [VP1_open_1,VP1_open_2,...,VP1_open_15].

[0087] Thumbs up: The feature vector set of the thumbs up gesture by pressure sensor P1: [VP1_thumb_1,VP1_thumb_2,...,VP1_thumb_15].

[0088] The fourth step is to repeat steps one, two, and three for each pressure sensor to establish a feature vector set for each pressure sensor.

[0089] In this embodiment, the process of identifying the current gesture using a classification model based on multiple pressure sensors, and statistically analyzing the identification results to determine the target gesture, specifically includes: For a pressure sensor, pressure time-series data is acquired in real time, and pressure feature values ​​are extracted based on the pressure time-series data. The pressure feature values ​​constitute the real-time feature vector of the pressure sensor. The real-time feature vector of the pressure sensor is input into the classification model of the pressure sensor to determine the current gesture of the dexterous hand and obtain a judgment result.

[0090] The current hand gesture is determined based on the real-time feature vector of each pressure sensor, resulting in multiple judgment results. The judgment results are statistically analyzed, and the gesture corresponding to the judgment result with the most statistical results is the target gesture.

[0091] For example, a dexterous hand makes a fist gesture, reads the pressure time-series data of each pressure sensor, and generates a real-time feature vector for each pressure sensor.

[0092] Taking pressure sensor P1 as an example, the real-time feature vector of P1 is [3.1, 2.1, 3.0, 0.4, 0.5].

[0093] The real-time feature vector of pressure sensor P1 is input into the classification model of pressure sensor P1 for recognition. The recognition result is a "clenched fist" gesture.

[0094] Similarly, pressure time-series data from pressure sensors P2 to P16 are read, and a corresponding real-time feature vector is generated for each pressure sensor. The real-time feature vectors of each pressure sensor are then input into the classification model for that pressure sensor for identification, resulting in a separate identification result for each pressure sensor.

[0095] For example, The hand gesture identified by the real-time feature vector of pressure sensor P2 is: clenched fist; The recognition result of the real-time feature vector of pressure sensor P3 is: the gesture is: palm outstretched; Up to pressure sensor P16; The gesture for recognizing the real-time feature vector of pressure sensor P16 is: raising a thumb.

[0096] According to statistics, the recognition result of the gesture was a clenched fist 11 times, the recognition result of the gesture was an open palm 4 times, and the recognition result of the gesture was a raised thumb 1 time. The overall judgment of the gesture recognition result is a clenched fist.

[0097] However, the system does not rely on a single sensor's "one-vote decision," but rather makes the final decision through a vote among multiple sensors. Misidentifications by a few sensors can be overridden by the "votes" of other correctly identified sensors, thus ensuring the stability of the entire system in the face of partial sensor failure.

[0098] Subtle changes in hand movements and inconsistencies in sensor-skin contact pressure can be smoothly addressed by a multi-sensor voting mechanism. This uncertainty can be addressed by ensuring reliable results as long as most sensors maintain consistent judgments in the feature space.

[0099] Another embodiment provides a pressure wristband gesture learning and recognition device, comprising: A gesture library module is built to train the center vectors of various gestures based on pressure data collected by a pressure sensor, and to construct the gesture library, including: The sample data generation unit is used to generate a sample data based on the pressure data collected by each pressure sensor when a gesture is made; and to generate multiple sample data by repeating the same gesture. Generate the initial center vector unit, which is used to generate the initial center vector of a gesture based on all sample data of a gesture; A gesture library unit is constructed to filter the sample data of the gesture based on the initial center vector of the gesture to obtain the final sample data of the gesture; to regenerate the final center vector of the gesture based on the final sample data of the gesture; and to construct the gesture library by generating the final center vector corresponding to each gesture one by one. The recognition and execution module, used for recognizing and executing gestures of dexterous hands, includes: The real-time vector unit is used to simultaneously collect pressure values ​​from various parts of the arm through various pressure sensors during dexterity hand movements, and the pressure values ​​constitute a real-time vector. The confidence level judgment unit, with the first condition, is used to determine whether the dexterity hand gesture has been switched based on confidence level. The distance judgment unit has a second condition: it determines whether the dexterity hand gesture has been switched based on the distance between the current vector and the real-time vector. The third condition, based on multi-point voting judgment, is to determine the dexterous hand's target gesture switching based on multi-point voting judgment. An execution unit is used to switch the dexterous hand to the target gesture when the first, second, and third conditions for dexterous hand gesture switching are met simultaneously.

[0100] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A method for learning and recognizing gestures on a pressure wristband, wherein the pressure wristband has multiple built-in pressure sensors, and the pressure wristband is worn on the outer periphery of the arm, wherein the pressure sensors directly or through a medium contact the arm, characterized in that it includes building a gesture library and recognizing gestures; Based on pressure data collected by pressure sensors, the center vectors of various gestures are trained, and a gesture library is constructed, including: When a gesture is made, each pressure sensor collects pressure data, and a sample data is generated based on the pressure data collected by each pressure sensor; repeating the same gesture generates multiple sample data. Generate the initial center vector of a gesture based on all sample data of the gesture; Using the initial center vector of the gesture as a standard, the sample data of the gesture is filtered to obtain the final sample data of the gesture; the final center vector of the gesture is regenerated based on the final sample data of the gesture. Generate the final center vector corresponding to each gesture one by one to build a gesture library; Gesture recognition specifically includes: During dexterity hand movement, pressure values ​​from various parts of the arm are collected simultaneously by various pressure sensors, and the pressure values ​​form a real-time vector. The first condition is to determine whether the dexterity hand gesture has been switched based on confidence level; The second condition is to determine whether the dexterous hand gesture is switched based on the distance between the current vector and the real-time vector; The third condition is to determine the target gesture for dexterous hand gesture switching based on multi-point voting. When the first, second, and third conditions for dexterous hand gesture switching are met simultaneously, the dexterous hand switches to the target gesture.

2. The pressure wristband gesture learning and recognition method according to claim 1, characterized in that... When a gesture is made, each pressure sensor collects pressure data, and a sample data is generated based on the pressure data collected by each pressure sensor, specifically as follows: Make a gesture and hold it for the required sampling time. The pressure sensor samples at a preset frequency. Calculate the average value of all data collected by a pressure sensor within the sampling time, which is taken as the pressure value collected by the pressure sensor during this sampling process. Based on this, calculate the pressure values ​​collected by each pressure sensor in the pressure bracelet. Each pressure value constitutes the sample data for this sampling.

3. The pressure wristband gesture learning and recognition method according to claim 1, characterized in that... The initial center vector of a gesture is generated based on all sample data of that gesture, specifically as follows: For all sample data collected for a gesture, the average value of all sample data collected by a sensor is calculated to obtain a parameter of a center vector; for a single pressure sensor, the average value of all sample data collected by the single pressure sensor is calculated to obtain each parameter of the center vector, and each parameter is used to construct the initial center vector of the gesture.

4. The pressure wristband gesture learning and recognition method according to claim 1, characterized in that... Using the center vector of the gesture as a standard, the sample data of the gesture is filtered to obtain the final sample data of the gesture, specifically as follows: Preset distance threshold; For a gesture, the Euclidean distance from the sample data to the initial center vector is calculated. When the Euclidean distance from the sample data to the initial center vector is greater than a preset distance threshold, the corresponding sample data is removed. Based on this, the sample data is filtered to obtain the final sample data of the gesture.

5. The pressure wristband gesture learning and recognition method according to claim 1, characterized in that... The method of determining whether a dexterous hand gesture has switched based on confidence level is as follows: The real-time vector is compared with the center vectors of each gesture to determine the center vector that is closest to the real-time vector as the first center vector and the center vector that is second closest to the real-time vector as the second center vector. The distance between the real-time vector and the first center vector is represented as the best candidate value; the distance between the real-time vector and the second center vector is represented as the second best candidate value; a first threshold is preset; the difference between the second best candidate value and the best candidate value is calculated to obtain the confidence level of gesture switching; If the confidence level of the gesture switching is less than the first threshold, the gesture of the dexterous hand remains unchanged. If the confidence level of gesture switching is greater than the first threshold, then the first condition for dexterous hand gesture switching is met.

6. The pressure wristband gesture learning and recognition method according to claim 1, characterized in that... The method of determining whether a dexterous hand gesture has switched based on the distance between the current vector and the real-time vector specifically involves: At the previous moment before the current gesture judgment of the dexterous hand, the vector representing the dexterous hand gesture is the current vector; at the same time, the center vector corresponding to the dexterous hand gesture at the previous moment is denoted as the current center vector; the distance between the current vector and the current center vector is calculated as the current distance; When judging the gesture of a dexterous hand, the vector representing the dexterous hand gesture is the real-time vector. The center vector that is closest to the real-time vector in the gesture library is determined by the neural network, which is the real-time center vector. The distance between the real-time vector and the real-time center vector is calculated as the real-time distance. Preset a second threshold; If the difference between the current distance and the real-time distance is less than the second threshold, the dexterous hand maintains the current gesture; If the difference between the current distance and the real-time distance is greater than the second threshold, then the second condition for dexterous hand gesture switching is met, and the gesture is prepared to be switched to the gesture corresponding to the real-time center vector.

7. The pressure wristband gesture learning and recognition method according to claim 1, characterized in that... The method of determining dexterous hand gesture switching based on multi-point voting is as follows: Pressure sensor data acquisition and feature extraction: For each pressure sensor, a corresponding set of feature vectors is established for each gesture. For each pressure sensor, a classification model is trained, including: for a pressure sensor, using gesture and feature vector set as input parameters, a classification model is trained for the pressure sensor through a neural network; and corresponding classification models are trained for each pressure sensor. The current gesture is identified by a classification model using multiple pressure sensors, and the identification results are statistically analyzed to determine the target gesture.

8. The pressure wristband gesture learning and recognition method according to claim 7, characterized in that... The pressure sensor data acquisition and feature extraction, for each pressure sensor, establishes a corresponding feature vector set for each gesture, including: The first step is to acquire the pressure time series data of a pressure sensor during a single gesture, extract pressure feature values ​​based on the pressure time series data, and construct a feature vector of the pressure sensor for the gesture. The second step is to collect multiple sample data of a single gesture for a pressure sensor. Each sample data corresponds to a feature vector, and the generated feature vectors form a set of feature vectors corresponding to the gesture. The third step is to establish a set of corresponding feature vectors for each gesture using a pressure sensor. For each pressure sensor, the first, second, and third steps are executed sequentially to establish a corresponding feature vector set for each gesture.

9. The pressure wristband gesture learning and recognition method according to claim 7, characterized in that... The process involves using a classification model based on multiple pressure sensors to identify the current gesture, and then statistically analyzing the identification results to determine the target gesture. Specifically, this includes: For a pressure sensor, pressure time-series data is acquired in real time, and pressure feature values ​​are extracted based on the pressure time-series data. The pressure feature values ​​form the real-time feature vector of the pressure sensor. The real-time feature vector of the pressure sensor is input into the classification model of the pressure sensor to determine the current gesture of the dexterous hand and obtain a judgment result. The current hand gesture is determined based on the real-time feature vector of each pressure sensor, resulting in multiple judgment results. The judgment results are statistically analyzed, and the gesture corresponding to the judgment result with the most counts is the target gesture.

10. A pressure wristband gesture learning and recognition device, characterized in that... ,include: A gesture library module is built to train the center vectors of various gestures based on pressure data collected by a pressure sensor, and to construct the gesture library, including: The sample data generation unit is used to generate a sample data based on the pressure data collected by each pressure sensor when a gesture is made; and to generate multiple sample data by repeating the same gesture. Generate the initial center vector unit, which is used to generate the initial center vector of a gesture based on all sample data of a gesture; A gesture library unit is constructed to filter the sample data of the gesture based on the initial center vector of the gesture to obtain the final sample data of the gesture; to regenerate the final center vector of the gesture based on the final sample data of the gesture; and to construct the gesture library by generating the final center vector corresponding to each gesture one by one. The recognition and execution module, used for recognizing and executing gestures of dexterous hands, includes: The real-time vector unit is used to simultaneously collect pressure values ​​from various parts of the arm through various pressure sensors during dexterity hand movements, and the pressure values ​​constitute a real-time vector. The confidence level judgment unit, with the first condition, is used to determine whether the dexterity hand gesture has been switched based on confidence level. The distance judgment unit has a second condition: it determines whether the dexterity hand gesture has been switched based on the distance between the current vector and the real-time vector. The third condition, based on multi-point voting judgment, is to determine the dexterous hand's target gesture switching based on multi-point voting judgment. An execution unit is used to switch the dexterous hand to the target gesture when the first, second, and third conditions for dexterous hand gesture switching are met simultaneously.

Images