A method for fine-grained gesture recognition

Abstract

This invention relates to a fine-grained gesture recognition method, comprising: Step S1: acquiring motion signals; Step S2: obtaining a first sample through sliding window slicing and enhancement processing; Step S3: obtaining a second sample by performing variable mode decomposition on the first sample to retain high-frequency vibration information; Step S4: extracting a first feature and a second feature respectively; Step S5: extracting style statistical features and injecting the extracted style statistical features into the first feature and the second feature respectively; Step S6: after concatenation, obtaining a classification result and a first probability sequence after weighted contrastive learning; Step S7: obtaining a second probability sequence; Step S8: obtaining a total loss based on the first sub-loss, the second sub-loss, and the third sub-loss; Step S9: adjusting the model parameters of the recognition model based on the total loss, and repeating S1 to S8 to complete the training of the recognition model; Step S10: acquiring the motion signal of the wrist when the thumb taps any joint of other fingers, and obtaining the recognition result based on the trained recognition model. Compared with existing technologies, this invention solves the problems of poor model robustness and weak generalization ability in existing sensor-based fine-grained gesture recognition due to the scarcity of training samples and significant domain shifts between users.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and human-computer interaction technology, and in particular to a method for fine-grained gesture recognition. Background Technology

[0002] With the development of human-computer interaction technology, thumb-to-finger gestures have gained widespread attention in wearable devices, virtual reality, and assisted input applications due to their naturalness, intuitiveness, and ability to be performed with one hand. Existing thumb gesture recognition methods utilize various sensing techniques, such as electrical impedance tomography (EIT), infrared, acoustic, pressure sensors, and cameras. However, these methods often rely on external devices or specific environments, leading to inconvenience in wearing them and poor robustness. In recent years, gesture recognition based on signals from the inertial measurement unit (IMU) in commercial smartwatches has gradually emerged. It requires no external devices, is easy to deploy, and has good practicality. However, most methods have a limited number of recognized gestures or require additional data collection and user adaptation, limiting their widespread adoption in real-world scenarios.

[0003] In practical applications of gesture recognition, differences in gesture execution methods, wrist postures, and physiological structures among individual users can lead to significant variations in signal distribution, even for the same gesture, resulting in a significant decrease in model recognition accuracy on new users. Domain generalization, a research hotspot in transfer learning in recent years, aims to train a model with generalization capabilities on source domain data, enabling it to achieve good performance in the target domain without accessing target domain data. Existing domain generalization methods have been widely studied in computer vision, such as data augmentation, domain-invariant feature learning, and meta-learning, but their application in IMU-based gesture recognition is still in its early stages, especially in generalizing to finer movements, where there is still considerable room for improvement.

[0004] Thumb-to-finger gesture recognition also exhibits significant fine-grained classification characteristics, meaning that the differences between different gesture categories are minimal and their signal features are highly similar, leading to blurred classification boundaries and easy confusion. To address this issue, some recent studies have attempted to introduce hierarchical structures, incorporating semantic relationships between categories into the model to improve the ability to distinguish difficult-to-classify samples. However, these methods are mostly focused on image tasks and have not been fully studied in the sensor field, especially in IMU-based gesture recognition. The thumb-to-finger gesture naturally possesses a semantic structure with "finger tapping action" as the upper-level category and "fingert tip / finger root / finger middle tapping action" as the lower-level subcategories, providing prior support for establishing a hierarchical classification model.

[0005] Existing thumb gesture recognition schemes still have shortcomings in terms of generalization, fine classification ability, and structural modeling. There is an urgent need for a unified recognition framework that takes into account robustness, generalization ability, and hierarchical perception ability. Summary of the Invention

[0006] The purpose of this invention is to provide a fine-grained gesture recognition method to overcome the shortcomings of the existing technology.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A method for fine-grained gesture recognition includes:

[0009] Step S1: Acquire the motion signal of the wrist when the thumb taps any joint of another finger, wherein the motion signal is acquired by an IMU sensor;

[0010] Step S2: Based on the motion signal, obtain the first sample through sliding window slicing and enhancement processing;

[0011] Step S3: Obtain the second sample by performing variable mode decomposition on the first sample to retain high-frequency vibration information;

[0012] Step S4: Extract the first feature and the second feature based on the first sample and the second sample, respectively;

[0013] Step S5: Extract style statistical features based on the first feature, and inject the extracted style statistical features into the first feature and the second feature respectively;

[0014] Step S6: After concatenating the first feature and the second feature after injecting style statistical features, the classification result and the first probability sequence are obtained after weighted contrastive learning, wherein the first probability sequence includes the probability corresponding to each segment of each finger;

[0015] Step S7: Obtain a second probability sequence based on the first feature after the injection style statistical features, wherein the second probability sequence includes the probability corresponding to each finger;

[0016] Step S8: Obtain the first sub-loss based on the classification results and sample annotation information, obtain the second sub-loss based on the first probability sequence and the second probability sequence, obtain the third sub-loss based on the weighted contrastive learning results, and obtain the total loss based on the first sub-loss, the second sub-loss and the third sub-loss;

[0017] Step S9: Adjust the model parameters of the recognition model based on the total loss, and repeat steps S1 to S8 to complete the training of the recognition model;

[0018] Step S10: Obtain the motion signal of the wrist when the thumb taps any joint of the other fingers, and obtain the recognition result based on the trained recognition model.

[0019] The enhancement process includes signal axis rotation, alignment, time warping, signal scaling, amplitude perturbation, Gaussian noise injection, and random sampling.

[0020] Step S5 includes:

[0021] Step S5-1: Extract the first statistical feature based on the first feature, wherein the first statistical feature includes the first mean and the first variance, and the mathematical expression is:

[0022]

[0023] in: The first mean, The first variance, The height of the feature, The width of the feature. The first feature extracted;

[0024] Step S5-2: Batch shuffling the extracted first features to obtain shuffled first features. ;

[0025] Step S5-3: Extract style statistical features based on the shuffled first feature, wherein the style statistical features include the second mean. Second variance ;

[0026] Step S5-4: Use random convex weights to fuse the first statistical feature and style statistical feature to obtain the first style scaling factor. and first style translation factor :

[0027]

[0028] in: The mixing coefficient;

[0029] Step S5-5: Extract the second statistical feature based on the second feature, and fuse the second statistical feature and style statistical feature using random convex weights to obtain the second style scaling factor. Second style translation factor :

[0030]

[0031] in: As the second feature, The second statistical mean is the second statistical characteristic. This refers to the second statistical variance in the second statistical characteristic.

[0032] Step S5-6: Adjust the first style scaling factor and first style translation factor The first feature is obtained by injecting the first feature and the style statistical features, and the second style scaling factor is applied. Second style translation factor The second feature obtained after injecting the style statistical features is:

[0033]

[0034]

[0035] in: This is the first feature after injecting style statistical features. This is the second feature after injecting style statistics features.

[0036] The third sub-loss is:

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] in: For the loss of the third child, For the set of positive sample pairs, This is the weighted aggregate value of positive sample pairs. This is the weighted aggregate value of negative sample pairs. z i For the embedding features of anchor point samples, I For the index set of the original representation and the enhanced representation, For the set of negative sample pairs, The weighting factor for negative samples. α The weighting factor for positive samples. For temperature scaling parameters, The embedding features of the positive samples corresponding to the anchor points. Embedding features of the negative samples corresponding to the anchor points.

[0043] In the weighted comparison learning stage, for positive sample pairs, the lower the similarity, the greater the weight is assigned to enhance the model's ability to narrow down difficult samples, while the higher the similarity, the smaller the weight is assigned to prevent redundant shrinkage from causing feature collapse; for negative sample pairs, the higher the similarity, the greater the weight is assigned to help discover difficult negative samples.

[0044] The first sub-loss is:

[0045]

[0046] in: For the loss of the first child, For the training dataset, For the first in the dataset One sample, For the first Hierarchical labels corresponding to each sample This represents the index of the observed class label in the hierarchy. for The The value of each label node, This is the normalized joint probability.

[0047] The first sub-loss obtained based on the classification results and sample annotation information includes:

[0048] Step S8-1: Construct a hierarchical tree ,in, Represents the set of all node pairs. This is a matrix describing the inclusion relationships between node pairs. This is a matrix describing the mutual exclusion relationships between node pairs;

[0049] Step S8-2: Define the valid tag space To capture structurally consistent tag combinations, all Construct a matrix ,in It is the sum of the number of fingers and the number of knuckles;

[0050] Step S8-3: Calculate the probabilistic classification loss using the values ​​of the output layer according to the structure of the hierarchical tree, where each activation function node of the output layer corresponds to an active label in the hierarchical tree;

[0051] Step S8-4: Let To represent a binary label vector for hierarchical assignment, given a single input sample ,distribute The nonnormalized joint probability is calculated as follows:

[0052]

[0053]

[0054] in: For non-normalized joint probabilities, Let be the potential function. For the first i The output of the activation function node of the output layer of each label node, For the first i The value of each label node;

[0055] Step S8-5: The joint probability is normalized using the partition function, and all legal assignments in the activity tree state space are applied. Summation, where the normalized joint probability is... for:

[0056]

[0057] in: It is the partition function;

[0058] Step S8-6: Obtain the first sub-loss based on the normalized joint probability.

[0059] The IMU sensor includes an accelerometer and a gyroscope.

[0060] A fine-grained gesture recognition device includes a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement the method described above.

[0061] A storage medium having a program stored thereon, which, when executed, implements the method described above.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The proposed style injection module effectively models and utilizes the performance differences between users, enhancing the model's robustness to style changes, which is different from the static feature extraction method of traditional methods that does not model user differences.

[0064] 2. Weighted supervised contrastive learning can simultaneously enhance class discriminative power and intra-class distribution structure, making the model more expressive in fine-grained gesture recognition, and especially alleviating the feature overfitting problem under sparse data.

[0065] 3. The introduction of hierarchical structure enables the model not only to learn the differences between categories, but also to guide the model learning process with the help of the semantic structure between categories, which is especially suitable for action or gesture recognition tasks with hierarchical structure.

[0066] 4. Compared with existing technologies that rely on a large amount of labeled data and target domain optimization, this invention can achieve superior cross-user generalization capabilities under conditions of low resources and no target domain access, which is closer to actual deployment needs. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the technical route of the present invention;

[0068] Figure 2 Schematic diagram of the style injection module;

[0069] Figure 3Construct a hierarchical structural diagram for finger structure and gesture tree;

[0070] Figure 4 A diagram illustrating the function of hierarchical structures. Figure 1 ;

[0071] Figure 5 A diagram illustrating the function of hierarchical structures. Figure 2 ;

[0072] Figure 6 A visual comparison diagram of standard supervised contrastive learning and weighted supervised contrastive learning;

[0073] Figure 7 The graph shows the improvement of the final model over the original model for each gesture.

[0074] Figure 8 A diagram showing the key layout for finger keyboard input;

[0075] Figure 9 This is a schematic diagram of the main steps of the method of the present invention. Detailed Implementation

[0076] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0077] This application constructs a human motion recognition system based on low-resource wearable sensor data, focusing on solving fine-grained thumb-to-finger gesture recognition tasks while also possessing adaptability to general human motion recognition tasks. The theoretical basis lies in the fact that IMU sensors (including a three-axis accelerometer and a three-axis gyroscope) can capture high-frequency transient motions and wrist execution style information generated during finger tapping or human movement, making them suitable for both fine-grained motion recognition and macroscopic behavior analysis. By introducing a style injection module and a feature enhancement mechanism based on weighted supervised contrastive learning, combined with semantic hierarchical modeling of gesture or motion categories, this invention can extract robust and discriminative motion features across users and in low-resource scenarios, achieving stable and generalized human motion recognition.

[0078] A method for fine-grained gesture recognition, such as Figure 1 and Figure 9 As shown, it includes:

[0079] Step S1: Acquire the motion signal of the wrist when the thumb taps any joint of the other fingers. The motion signal is collected by an IMU sensor. In this embodiment, the IMU sensor includes an accelerometer and a gyroscope.

[0080] Step S2: Based on the motion signal, obtain the first sample through sliding window slicing and enhancement processing;

[0081] The system first performs sliding window slicing on the collected raw IMU signals and uses various signal augmentation strategies to build data diversity. Within each sampling window, data augmentation methods, including signal axis rotation, alignment, time warp, signal scaling, amplitude perturbation, Gaussian noise injection, and random sampling, are applied to simulate different user execution styles and sensor noise, thereby improving model robustness. The augmented samples and the original samples together form the sample pool for model input.

[0082] Step S3: Obtain the second sample by performing variable mode decomposition on the first sample to retain high-frequency vibration information;

[0083] Specifically, the system employs a dual-branch structure for feature extraction. One path directly processes the raw IMU data, while the other path introduces Variational Mode Decomposition (VMD) into the triaxial acceleration signal to obtain high-frequency motion components as a "content" representation of fine-grained motion features. The VMD signal preserves the sudden high-frequency vibration information in the tapping gesture, enabling the capture of subtle differences in the touch positions of different fingers.

[0084] Step S4: Extract the first feature and the second feature based on the first sample and the second sample, respectively;

[0085] Step S5: Extract style statistical features based on the first feature, and inject the extracted style statistical features into the first feature and the second feature respectively. In order to further improve the cross-domain generalization ability, the system introduces a style injection module in both of the above paths. Specifically, the system extracts the channel mean and variance from the early features of the original IMU branch as user style statistics and injects them into the original path and the VMD path respectively.

[0086] The process is as follows Figure 2 As shown, it specifically includes:

[0087] Step S5-1: Extract the first statistical feature based on the first feature, wherein the first statistical feature includes the first mean and the first variance, and the mathematical expression is:

[0088]

[0089] in: The first mean, The first variance, The height of the feature, The width of the feature. The first feature extracted;

[0090] Step S5-2: Batch shuffling the extracted first features to obtain shuffled first features. ;

[0091] Step S5-3: Extract style statistical features based on the shuffled first feature, wherein the style statistical features include the second mean. Second variance ;

[0092] Step S5-4: Use random convex weights to fuse the first statistical feature and style statistical feature to obtain the first style scaling factor. and first style translation factor :

[0093]

[0094] in: The mixing coefficient;

[0095] Step S5-5: Extract the second statistical feature based on the second feature, and fuse the second statistical feature and style statistical feature using random convex weights to obtain the second style scaling factor. Second style translation factor :

[0096]

[0097] in: As the second feature, The second statistical mean is the second statistical characteristic. This refers to the second statistical variance in the second statistical characteristic.

[0098] Step S5-6: Adjust the first style scaling factor and first style translation factor The first feature is obtained by injecting the first feature and the style statistical features, and the second style scaling factor is applied. Second style translation factor The second feature obtained after injecting the style statistical features is:

[0099]

[0100]

[0101] in: This is the first feature after injecting style statistical features. This is the second feature after injecting style statistics features.

[0102] Overall, this mechanism is used in the original path for regularization training to prevent the network from overfitting to individual execution styles; in the VMD path, it serves as contextual information to alleviate the sensitivity of high-frequency content to execution differences, thereby improving the overall style invariance modeling capability.

[0103] Step S6: After concatenating the first feature and the second feature after injecting style statistical features, the classification result and the first probability sequence are obtained after weighted contrastive learning. The first probability sequence includes the probability of each segment corresponding to each finger.

[0104] In the feature comparison learning stage, the system designs a weighted supervised comparison learning module. For the constructed sample pairs, the system introduces a similarity-based weighting strategy for positive and negative samples: for positive sample pairs, those with lower similarity are assigned larger weights to enhance the model's ability to narrow down difficult samples; those with higher similarity are assigned smaller weights to prevent feature collapse caused by redundant contraction; for negative sample pairs, the higher the similarity, the larger the weight, thereby effectively identifying difficult negative samples and improving inter-class discriminativeness. This module, together with the cross-entropy classification objective, trains the backbone network, enabling the model to obtain a more structured and generalizable representation space.

[0105] Specifically, for positive samples, introduce The weighting factor for each positive sample, and The weighted aggregation value for positive sample pairs is given by the following formula:

[0106]

[0107] The anchor index is denoted as The set of all other indexes besides the anchor point is denoted as Anchor point The set of positive sample pairs is defined as ,in and These represent the class labels for the anchor point and the candidate positive sample, respectively. This indicates the cardinality of the positive sample set at the current anchor point. The embedding features representing the anchor point Represents the embedding features of positive samples. It is a temperature scaling parameter.

[0108] Similarly, for negative samples, we introduce The weighting factor for each negative sample, and The weighted aggregate value for negative sample pairs is given by the following formula:

[0109]

[0110] The anchor index is denoted as The set of all other indexes besides the anchor point is denoted as Anchor point The set of negative sample pairs is defined as ,in and These represent the class labels for the anchor point and the candidate negative sample, respectively. This represents the cardinality of the negative sample set at the current anchor point. Represents the embedding features of negative samples. It is a temperature scaling parameter.

[0111] Combination and The third sub-loss of weighted contrastive learning This can be expressed by the following formula:

[0112]

[0113]

[0114]

[0115]

[0116]

[0117] in: For the loss of the third child, For the set of positive sample pairs, This is the weighted aggregate value of positive sample pairs. This is the weighted aggregate value of negative sample pairs. z i For the embedding features of anchor point samples, I For the index set of the original representation and the enhanced representation, For the set of negative sample pairs, The weighting factor for negative samples. α The weighting factor for positive samples. For temperature scaling parameters, The embedding features of the positive samples corresponding to the anchor points. Embedding features of the negative samples corresponding to the anchor points.

[0118] like Figure 5 As shown, compared with standard contrastive learning, our weighted contrastive learning strategy selectively enhances the samples with the most information, making the boundaries of the embeddings of different classes of feature spaces clearer.

[0119] Step S7: Obtain the second probability sequence based on the first feature after the injection style statistical features, wherein the second probability sequence includes the probability corresponding to each finger;

[0120] like Figure 3 As shown, the system introduces a hierarchical modeling structure at the task level to guide the model in learning key discrimination regions using inter-class semantic structures. This structure organizes all gesture categories into a two-layer hierarchy: the top layer represents finger categories, and the bottom layer represents specific striking locations (such as fingertips, middle joints, and bases). During the training phase, the system constructs a tree structure mapping matrix and designs a hierarchical consistency loss function to ensure that in network prediction, the activation of the bottom-level categories implicitly includes the discrimination information of their parent categories, thereby reducing inter-category ambiguity and enhancing the ability to distinguish structurally similar categories.

[0121] Step S8: Obtain the first sub-loss based on the classification results and sample annotation information, obtain the second sub-loss based on the first probability sequence and the second probability sequence, obtain the third sub-loss based on the weighted contrastive learning results, and obtain the total loss based on the first sub-loss, the second sub-loss and the third sub-loss;

[0122] The first loss is:

[0123]

[0124] in: For the loss of the first child, For the training dataset, For the first in the dataset One sample, For the first Hierarchical labels corresponding to each sample This represents the index of the observed class label in the hierarchy. for The The value of each label node, This is the normalized joint probability.

[0125] The first sub-loss, derived from the classification results and sample annotation information, includes:

[0126] Step S8-1: Construct a hierarchical tree ,in, Represents the set of all node pairs. This is a matrix describing the inclusion relationships between node pairs. To describe the mutual exclusion matrix of node pairs, each class is assigned a binary label, and only label configurations that satisfy hierarchical constraints are considered valid;

[0127] Step S8-2: Define the valid tag space To capture structurally consistent tag combinations, all Construct a matrix ,in It is the sum of the number of fingers and the number of knuckles;

[0128] Step S8-3: Calculate the probabilistic classification loss using the values ​​of the output layer according to the structure of the hierarchical tree, where each activation function node of the output layer corresponds to an active label in the hierarchical tree;

[0129] Step S8-4: Let To represent a binary label vector for hierarchical assignment, given a single input sample ,distribute The nonnormalized joint probability is calculated as follows:

[0130]

[0131]

[0132] in: For non-normalized joint probabilities, Let be the potential function. For the first i The output of the activation function node of the output layer of each label node, For the first i The value of each label node;

[0133] Step S8-5: The joint probability is normalized using the partition function, and all legal assignments in the activity tree state space are applied. Summation, where the normalized joint probability is... for:

[0134]

[0135] in: It is the partition function;

[0136] If the legal hierarchical label of a certain observed sample is , its first The number of label nodes is 1, that is... According to the joint probability Calculate the first sample The edge probability of each label node In essence, it involves calculating all... The sum of the legal hierarchical labels is used. For example, if a leaf node is the index finger's upper knuckle, its legal label would be 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0, meaning the parent node must be the index finger. Since there's only one legal label here, the summation result is the probability of that label itself. Because when the leaf node is the index finger's upper knuckle, the marginal probability calculation includes the labels of nodes containing the index finger joint, excluding other leaf nodes or parent nodes. Therefore, the method of calculating marginal probabilities reflects the hierarchical relationship.

[0137] Step S8-6: Obtain the first sub-loss based on the normalized joint probability. The first sub-loss r is in the form of negative log-likelihood. Minimizing the loss is equivalent to maximizing the marginal likelihood, which is equivalent to maximizing the marginal probability of the observed ground truth label. The optimization process will force the model to predict results that conform to the hierarchical structure, that is, to approach the legal labels such as 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0.

[0138] Step S9: Adjust the model parameters of the recognition model based on the total loss, and repeat steps S1 to S8 to complete the training of the recognition model;

[0139] Step S10: Obtain the motion signal of the wrist when the thumb taps any joint of the other fingers, and obtain the recognition result based on the trained recognition model.

[0140] When using this system for fine-grained gesture recognition between the thumb and other fingers, initial model training must first be completed in a predefined scenario. The system collects raw IMU signals from multiple users performing standard gestures in different environments. Through the aforementioned data augmentation, style injection, weighted supervised contrastive learning, and hierarchical modeling modules, it extracts feature representations that are both discriminative and generalizable. During training, the model does not depend on target domain data, thus possessing good cross-user transfer capabilities.

[0141] During the deployment phase, users can complete gesture input or action recognition using IMU sensing devices such as wearable commercial smartwatches without retraining the model. The system automatically collects real-time sensor data from new users and performs classification and prediction using the trained model, achieving low-resource and low-cost human perception and interaction recognition functions. This system is particularly suitable for application scenarios such as mobile interaction, wearable computing, and assisted input, and has good practicality and promotion potential.

[0142] like Figures 4 to 8 As shown, comparative experiments were conducted on the self-built fine-grained dataset FingerTap in this embodiment to systematically verify the effectiveness and generalization ability of the method of the present invention. Under low resource settings, the method of the present invention, SCH-Net, achieved the highest classification accuracy on the FingerTap dataset, with an accuracy of 83.50%, which is 7.62% higher than the second-best method, fully verifying the method's ability to model subtle differences in actions.

[0143] We also compared the accuracy improvement of the final model over ERM across 12 specific gesture categories, as shown in Table 1:

[0144] Table 1

[0145]

[0146] The results show that the system achieves significant improvements in most categories, especially in difficult-to-recognize finger categories such as Index-2 and Ring-1, where the accuracy improvement exceeds 10%. This indicates that the three-module collaborative mechanism proposed in this invention has significant advantages in capturing fine-grained gesture differences and suppressing irrelevant style perturbations, and constructs a more robust and discriminative representation space.

[0147] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for fine-grained gesture recognition, characterized in that, include: Step S1: Acquire the motion signal of the wrist when the thumb taps any joint of another finger, wherein the motion signal is acquired by an IMU sensor; Step S2: Based on the motion signal, obtain the first sample through sliding window slicing and enhancement processing; Step S3: Obtain the second sample by performing variable mode decomposition on the first sample to retain high-frequency vibration information; Step S4: Extract the first feature and the second feature based on the first sample and the second sample, respectively; Step S5: Extract style statistical features based on the first feature, and inject the extracted style statistical features into the first feature and the second feature respectively; Step S6: After concatenating the first feature and the second feature after injecting style statistical features, the classification result and the first probability sequence are obtained after weighted contrastive learning, wherein the first probability sequence includes the probability corresponding to each segment of each finger; Step S7: Obtain a second probability sequence based on the first feature after the injection style statistical features, wherein the second probability sequence includes the probability corresponding to each finger; Step S8: Obtain the first sub-loss based on the classification results and sample annotation information, obtain the second sub-loss based on the first probability sequence and the second probability sequence, obtain the third sub-loss based on the weighted contrastive learning results, and obtain the total loss based on the first sub-loss, the second sub-loss and the third sub-loss; Step S9: Adjust the model parameters of the recognition model based on the total loss, and repeat steps S1 to S8 to complete the training of the recognition model; Step S10: Obtain the motion signal of the wrist when the thumb taps any joint of the other fingers, and obtain the recognition result based on the trained recognition model; Step S5 includes: Step S5-1: Extract the first statistical feature based on the first feature, wherein the first statistical feature includes the first mean and the first variance, and the mathematical expression is: in: The first mean, The first variance, The height of the feature, The width of the feature. The first feature extracted; Step S5-2: Batch shuffling the extracted first features to obtain shuffled first features. ; Step S5-3: Extract style statistical features based on the shuffled first feature, wherein the style statistical features include the second mean. Second variance ; Step S5-4: Use random convex weights to fuse the first statistical feature and style statistical feature to obtain the first style scaling factor. and first style translation factor : in: The mixing coefficient; Step S5-5: Extract the second statistical feature based on the second feature, and fuse the second statistical feature and style statistical feature using random convex weights to obtain the second style scaling factor. Second style translation factor : in: As the second feature, It is the second statistical mean in the second statistical characteristic. This refers to the second statistical variance in the second statistical characteristic. Step S5-6: Adjust the first style scaling factor and first style translation factor The first feature is obtained by injecting the first feature and the style statistical features, and the second style scaling factor is applied. Second style translation factor The second feature obtained after injecting the style statistical features is: in: This is the first feature after injecting style statistical features. This is the second feature after injecting style statistics features.

2. The fine-grained gesture recognition method according to claim 1, characterized in that, The enhancement process includes signal axis rotation, alignment, time warping, signal scaling, amplitude perturbation, Gaussian noise injection, and random sampling.

3. The method for fine-grained gesture recognition according to claim 1, characterized in that, The third sub-loss is: in: For the loss of the third child, For the set of positive sample pairs, This is the weighted aggregate value of positive sample pairs. This is the weighted aggregate value of negative sample pairs. z i For the embedding features of anchor point samples, I For the index set of the original representation and the enhanced representation, For the set of negative sample pairs, The weighting factor for negative samples. α The weighting factor for positive samples. For temperature scaling parameters, The embedding features of the positive samples corresponding to the anchor points. Embedding features of the negative samples corresponding to the anchor points.

4. The fine-grained gesture recognition method according to claim 3, characterized in that, In the weighted comparison learning stage, for positive sample pairs, the lower the similarity, the greater the weight is assigned to enhance the model's ability to bring difficult samples closer together, while the higher the similarity, the smaller the weight is assigned to prevent redundant shrinkage from causing feature collapse. For negative sample pairs, the higher the similarity, the greater the weight, making it difficult to find negative samples.

5. The fine-grained gesture recognition method according to claim 1, characterized in that, The first sub-loss is: in: For the loss of the first child, For the training dataset, For the first in the dataset One sample, For the first Hierarchical labels corresponding to each sample This represents the index of the observed class label in the hierarchy. for The The value of each label node, This is the normalized joint probability.

6. The fine-grained gesture recognition method according to claim 5, characterized in that, The first sub-loss obtained based on the classification results and sample annotation information includes: Step S8-1: Construct a hierarchical tree ,in, Represents the set of all node pairs. This is a matrix describing the inclusion relationships between node pairs. This is a matrix describing the mutual exclusion relationships between node pairs; Step S8-2: Define the valid tag space To capture structurally consistent tag combinations, all Construct a matrix ,in It is the sum of the number of fingers and the number of knuckles; Step S8-3: Calculate the probabilistic classification loss using the values ​​of the output layer according to the structure of the hierarchical tree, where each activation function node of the output layer corresponds to an active label in the hierarchical tree; Step S8-4: Let To represent a binary label vector for hierarchical assignment, given a single input sample ,distribute The nonnormalized joint probability is calculated as follows: in: For non-normalized joint probabilities, Let be the potential function. For the first i The output of the activation function node of the output layer of each label node, For the first i The value of each label node; Step S8-5: The joint probability is normalized using the partition function, and all legal assignments in the activity tree state space are applied. Summation, where the normalized joint probability is... for: in: It is the partition function; Step S8-6: Obtain the first sub-loss based on the normalized joint probability.

7. The fine-grained gesture recognition method according to claim 1, characterized in that, The IMU sensor includes an accelerometer and a gyroscope.

8. A fine-grained gesture recognition device, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-7.

9. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-7.

Images