Gesture recognition method based on forearm muscle group deformation

Abstract

According to the gesture recognition method based on forearm muscle group deformation, high-sensitivity strain sensors are worn at the positions of flexion and extension muscle groups on the inner side and the outer side of an arm, body surface deformation is directly measured, a machine learning model is designed and trained for gesture recognition, deep muscle electric signal interaction is avoided, the reliability of data collection is improved, and the gesture recognition accuracy is improved. And meanwhile, the information amount is reduced, and the post-processing difficulty is reduced. The machine learning model fuses a one-dimensional convolutional neural network and a Transform structure, the one-dimensional convolutional neural network extracts spatial features, the Transform structure processes time sequence data by using a self-attention mechanism, and the processing capacity for complex time dependence is enhanced. The model maps data to an implicit vector space through position coding, time and space features are effectively extracted, and the generalization ability and the processing speed of the model are improved while high recognition precision is kept. The method can be widely applied to the fields of virtual reality, sign language recognition, robot control and rehabilitation and the like, and is particularly expected to meet the requirements of forearm amputation patients on intelligent artificial limb control.

Description

Technical Field

[0001] The invention relates to gesture recognition technology, and in particular to a gesture recognition method based on forearm muscle deformation. Background Art

[0002] Hands are responsible for most tasks in daily life. Wearable human-computer interaction systems with the ability to detect and recognize gestures have attracted widespread attention and can be used in various application scenarios such as virtual reality (VR), sign language recognition, robot control, rehabilitation, etc. Existing gesture recognition methods based on wearable devices can be mainly divided into two types: one is a gesture recognition method based on flexible pressure / strain sensors to detect hand deformation, and the other is a gesture recognition method based on flexible electromyography electrodes to detect arm surface electromyography signals.

[0003] The gesture recognition method based on hand deformation decomposes the gesture into the bending angles of different finger joints and wrists by wearing one or more strain sensors on the back or joints of the five fingers, thereby decoupling and recognizing the gestures with high precision. However, wearing bulky gloves may cause discomfort in the hands and hinder the free movement of the fingers. The restriction of finger movement affects the flexibility of the hands and the convenience of operation. In addition, this gesture recognition method based on direct detection of hand movement cannot be applied to the potential needs of some forearm amputees in intelligent prosthetic control.

[0004] The gesture recognition method based on the surface electromyography signal of the arm is generated with neural activity and muscle excitation. It originates from the bioelectric activity of the alpha motor neurons of the spinal cord under the control of the motor cortex of the brain. It has a certain correlation with the muscle movement state and can reflect the intention of a specific gesture to a certain extent. By attaching flexible electrodes to the arm to collect and analyze the surface electromyography signals obtained from the arm muscles, the gesture recognition is realized. It has the advantages of high safety, simple operation, convenient use, and non-invasiveness. However, the surface electromyography signal is the combined effect of the superficial muscle electrical signal and the electrical activity on the nerve trunk on the surface of the skin. It is not directly related to the deep muscles in the area where the electrodes are attached, which brings challenges to signal acquisition and subsequent analysis. Due to the instability of the mechanical-electrical interface between the electrodes and the skin at the wearing site, as well as skin sweating and muscle fatigue, it is difficult to ensure the reliability and repeatability of the surface electromyography signals obtained.

[0005] In general, the gesture recognition method based on hand deformation has a bulky device design, which hinders the normal movement of fingers. The gesture recognition method based on the surface electromyography signal of the arm has problems such as complicated post-processing and unstable mechanical-electrical interface at the wearing part.

[0006] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0007] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide a gesture recognition method based on forearm muscle deformation.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A gesture recognition method based on forearm muscle deformation includes the following steps:

[0010] S1. Strain data collection: The subjects wore multiple strain sensors on the flexor and extensor muscles on the inside and outside of their arms to measure the body surface deformation of these parts under different gestures;

[0011] S2. Gesture recognition model training: Preprocess the collected strain data, such as using methods such as add noise, time shift, and time warp to enhance the data, and then use the overlapping sliding window method to process the data to capture the detailed features of the overlapping parts and achieve data enhancement; Based on the preprocessed strain data, train a machine learning model that integrates a one-dimensional convolutional neural network and a Transformer structure to classify and identify gestures;

[0012] S3: Real-time gesture recognition: Use the trained model to pre-process the real-time collected signal and perform gesture recognition to achieve real-time gesture recognition.

[0013] Furthermore, step S1 includes: at least two strain sensors respectively measure the deformation of the inner and outer muscle concentrations of the forearm to identify the deformation differences of the wearing parts under different gestures.

[0014] Furthermore, step S1 includes: using a plurality of resistive strain sensors to detect denser muscle groups near the elbow.

[0015] Furthermore, step S1 includes: the subject's fingers and wrists move with different degrees of freedom, each movement lasts for a predetermined time and is repeated multiple times; the resistance response of the strain sensor generated by the strain under different gesture movements is converted into an analog voltage signal through a parallel voltage divider circuit, and the analog voltage signal is transmitted to a computer terminal to construct a data set.

[0016] Furthermore, in step S2, the overlapping sliding window method for processing data specifically includes: using window length and step length as a window segmentation method, wherein the window length is used to determine the number of data points contained in each window, and the step length is used to determine the degree of overlap and the moving step between windows; for example, a 100-point window length and a 10-point step length are used as a window segmentation method. By processing the data in this way, the detailed features of the overlapping parts in the data are captured, and the accuracy of gesture recognition is improved as a data enhancement method.

[0017] Furthermore, in step S2, the machine learning model integrating the one-dimensional convolutional neural network and the Transformer structure specifically includes:

[0018] The spatial features between different channels in strain data are effectively extracted through a one-dimensional convolutional neural network;

[0019] Handle time series classification tasks through the Transformer structure with self-attention mechanism to capture temporal dependencies;

[0020] The strain data is taken as a time series and mapped into the latent vector space through position-encoded word embedding to extract temporal and spatial features, thereby realizing the classification and recognition of gesture actions.

[0021] Furthermore, in step S2, the machine learning model integrating the one-dimensional convolutional neural network and the Transformer structure specifically includes the following structure:

[0022] Preprocessing module: Use methods such as add noise, time shift, and time warp to enhance data, process the noisy data, and use a 100-point window length and a 10-point step length for data enhancement and segmentation;

[0023] One-dimensional convolution layer: contains a one-dimensional convolution operation with a padding size of 3 and a stride of 1, which is used to extract local features;

[0024] Batch Normalization Layer: Normalizes the output of the convolutional layer to stabilize the training process;

[0025] Rectifier Linear Layer (ReLU): applies nonlinear activation function to increase the nonlinear expression ability of the model;

[0026] Padding and convolution: Perform a one-dimensional convolution operation with a padding size of 1 and a stride of 1 to further extract features;

[0027] Transformer encoder layer: contains position encoding and a multi-head self-attention mechanism with an embedding scale of 64, 4 heads and 9 layers, for processing time series data;

[0028] Feedforward neural network: A 64*4 feedforward neural network is connected after the Transformer encoder layer to further process features;

[0029] Pooling layer: performs pooling operations on features to reduce dimensions and extract important information;

[0030] Fully connected layer: maps the extracted features to the final classification output with an output dimension of 64.

[0031] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the gesture recognition method based on forearm muscle group deformation.

[0032] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the gesture recognition method based on forearm muscle group deformation is implemented.

[0033] The present invention has the following beneficial effects:

[0034] The present invention proposes a gesture recognition method based on deformation of forearm muscle groups. The method directly measures the surface deformation of the force-applying muscle groups by wearing highly sensitive and well-fitting strain sensors on the flexor muscle groups on the inner side of the subject's arm and the extensor muscle groups on the outer side, and performs gesture recognition in combination with the machine learning model designed by the present invention. The method of the present invention does not involve interaction with deep muscle electrical signals, so there is no signal reliability problem caused by the instability of the electrical interface, which improves the reliability of data acquisition. At the same time, since the amount of information of strain data is greatly reduced compared with that of electromyographic signals, and does not contain frequency domain information, this effectively reduces the difficulty of signal acquisition and post-processing. In addition, the application scenarios of the present invention are more extensive. It is not only suitable for general gesture recognition, but also expected to meet the potential needs of patients with partial forearm amputation in intelligent prosthetic control. By detecting the contraction and fluctuation state of the forearm flexor and extensor muscle groups, different gestures can be distinguished, thereby broadening the application scope of gesture recognition technology. Furthermore, by designing a machine learning model that integrates a one-dimensional convolutional neural network and a Transformer structure, efficient classification and recognition of forearm muscle deformation data is achieved. The one-dimensional convolutional neural network is responsible for extracting spatial features, while the Transformer structure uses a self-attention mechanism to process time series data. This structural design not only improves the accuracy of the model in classifying gestures, but also enhances the model's ability to handle complex time dependencies. Carefully designed network structure parameters, including padding size, step size, embedding scale, number of heads, and number of layers, enable the model to capture and learn patterns in the data more finely. In addition, the design of the model maps data into a latent vector space through position-encoded word embedding, effectively extracting temporal and spatial features, thereby maintaining high recognition accuracy while also improving the model's generalization ability and processing speed.

[0035] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a flow chart of a gesture recognition method based on forearm muscle deformation according to an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of the wearing position of the strain sensor according to an embodiment of the present invention.

[0038] Figure 3 Typical strain data measured by the sensor array under different gesture actions according to the embodiment of the present invention.

[0039] Figure 4 It is a schematic diagram of the structure of the fusion network model according to an embodiment of the present invention.

[0040] Figure 5 1 is the confusion matrix of gesture recognition results of 16 typical actions in an embodiment of the present invention.

[0041] Figure 6 This is a demonstration of real-time recognition of gestures according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0043] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0044] See also Figure 1 The embodiment of the present invention provides a gesture recognition method based on forearm muscle deformation, comprising the following steps:

[0045] Step S1, strain data collection: the subject wears multiple strain sensors on the flexor and extensor muscles on the inner and outer sides of the arm to measure the body surface deformation of the parts under different gestures.

[0046] See also Figure 2In a preferred embodiment, in step S1, at least two strain sensors are used to measure the deformation of the inner and outer muscle concentrations of the forearm respectively to identify the deformation differences of the wearing parts under different gestures. Multiple resistive strain sensors can be used to detect the denser muscle groups near the elbow. The fingers and wrists of the subject move with different degrees of freedom, each movement lasts for a predetermined time and is repeated multiple times; the resistance response of the strain sensor generated by the strain under different gestures is converted into an analog voltage signal through a parallel voltage divider circuit, and the analog voltage signal is transmitted to a computer terminal to construct a data set.

[0047] Step S2, gesture recognition model training: preprocessing the collected strain data, using methods such as add noise, time shift, time warp to enhance the data, and then using an overlapping sliding window method to process the data to capture the detailed features of the overlapping parts, to achieve data enhancement and improve recognition accuracy; based on the preprocessed strain data, training a machine learning model that integrates a one-dimensional convolutional neural network and a Transformer structure for classification and recognition of gestures;

[0048] In a preferred embodiment, in step S2, the overlapping sliding window method for processing data specifically includes: using window length and step length as a window segmentation method, wherein the window length is used to determine the number of data points contained in each window, and the step length is used to determine the degree of overlap and the moving step between windows; for example, a 100-point window length and a 10-point step length are used as a window segmentation method. By processing the data in this way, the detailed features of the overlapping parts in the data are captured, and the accuracy of gesture recognition is improved as a data enhancement method.

[0049] like Figure 4As shown, in a preferred embodiment, the machine learning model integrating the one-dimensional convolutional neural network and the Transformer structure specifically includes: effectively extracting the spatial features between different channels in the strain data through the one-dimensional convolutional neural network; processing the time series classification task through the Transformer structure with a self-attention mechanism to capture the time dependency; taking the strain data as a time series, mapping it to the latent vector space through the position-encoded word embedding to extract the time and space features, thereby realizing the classification and recognition of gesture actions. The machine learning model integrating the one-dimensional convolutional neural network and the Transformer structure specifically includes the following structure: preprocessing module: process the noisy data, use a 100-point window length and a 10-point step size for data enhancement and segmentation; one-dimensional convolution layer: contains a one-dimensional convolution operation with a padding size of 3 and a step size of 1, which is used to extract local features; batch normalization layer: normalizes the output of the convolution layer to stabilize the training process; linear rectification layer (ReLU): applies a nonlinear activation function to increase the nonlinear expression ability of the model; padding and convolution: performs a one-dimensional convolution operation with a padding size of 1 and a step size of 1 to further extract features; Transformer encoder layer: contains a multi-head self-attention mechanism with a position encoding and an embedding scale of 64, with 4 heads and 9 layers, which is used to process time series data; feedforward neural network: a 64*4 feedforward neural network is connected after the Transformer encoder layer to further process features; pooling layer: performs pooling operations on features to reduce dimensions and extract important information; fully connected layer: maps the extracted features to the final classification output with an output dimension of 64. This preferred embodiment achieves efficient classification and recognition of forearm muscle deformation data by integrating a one-dimensional convolutional neural network and a machine learning model of a Transformer structure, wherein the one-dimensional convolutional neural network is responsible for extracting spatial features, while the Transformer structure uses a self-attention mechanism to process time series data. The design of this structure not only improves the accuracy of the model in classifying gestures, but also enhances the model's ability to handle complex time dependencies. Carefully designed network structure parameters, including padding size, step size, embedding scale, number of heads and layers, enable the model to capture and learn patterns in the data more finely. In addition, the design of the model maps data into a latent vector space through position-encoded word embedding, effectively extracting temporal and spatial features, thereby maintaining high recognition accuracy while also improving the model's generalization ability and processing speed.

[0050] Step S3, real-time gesture recognition: using the trained model to pre-process the real-time collected signal and perform gesture recognition to achieve real-time gesture recognition.

[0051] The specific embodiments and experimental verification of the present invention are further described below.

[0052] A gesture recognition method based on forearm muscle deformation, the main process of which includes: wearing a highly sensitive and well-fitting strain sensor on the flexor and extensor muscles on the inside and outside of the arm, measuring the body surface deformation of the part under different gestures, establishing a data set, and performing gesture recognition in combination with the machine learning method designed by the present invention. The process of the gesture recognition method based on forearm muscle deformation is as follows: Figure 1 shown.

[0053] 1. Strain data collection

[0054] This gesture recognition method uses multiple strain sensors to measure the deformation of the forearm muscles. To ensure the accuracy of gesture recognition, there should be at least two strain sensors, ensuring that at least one sensor on the inner and outer sides of the forearm measures the deformation of the two muscle concentrations respectively. At the same time, the sensors should be sensitive and fit well enough to recognize the deformation differences of the wearing parts under different gestures. The embodiment of the present invention is demonstrated with four resistive strain sensors, and the wearing positions are as follows: Figure 2 As shown, the detection is mainly focused on the denser muscle groups near the elbows, which have more obvious deformations with different gestures, thereby improving the accuracy of gesture recognition.

[0055] Through the parallel voltage divider circuit, the resistance response of the sensor generated by the strain under different gestures is converted into an analog voltage signal, which is transmitted to the computer terminal through the NI data acquisition card, and the signal is stored and displayed by LabVIEW. The stored data is subsequently used for machine learning or deep learning methods to realize gesture recognition. As a demonstration, the data set contains 16 gestures with different degrees of freedom of fingers and wrists, such as Figure 3 As shown in the figure, the subjects were asked to make corresponding gestures from a static state. Each action lasted for 2 seconds and was repeated 10 times. The data was collected by the NI data acquisition card at a sampling rate of 400 Hz. The typical signals measured by the sensor under different gestures are as follows: Figure 3 shown.

[0056] 2. Gesture recognition model training

[0057] Based on the obtained strain data, the machine learning model is trained to solve the classification problem of gesture movements, and the trained model is used to realize gesture recognition. Before passing the data into the network model, a series of preprocessing operations need to be performed on the data. First, the data is enhanced by using methods such as add noise, time shift, and time warp. Since it is a long time series data, the data is processed by an overlapping sliding window method to capture the detailed features of the overlapping part. At the same time, it is used as a data enhancement method to improve the recognition accuracy. The size of the window length affects the recognition accuracy and response delay. As a demonstration, a 100-point window length and a 10-point step length are used as the window segmentation method, and then distributed to the training set, validation set, and test set. The present invention designs a network model that integrates a one-dimensional convolutional neural network and a Transformer structure, such as Figure 4 As shown in the figure (the specific model structure and parameter design are mentioned above and will not be repeated here), the strain data obtained is essentially a time series. The deep learning method can better handle the classification problem of gesture recognition. The Transformer with self-attention mechanism has excellent performance in the time series classification task, and the one-dimensional convolution can effectively extract the spatial features between different channels as the word embedding of the position encoding to map the data into the latent vector space and extract the time and space features.

[0058] 3. Use the trained model for gesture recognition

[0059] The trained model has a recognition accuracy of 97.17% for the validation set. The confusion matrix is ​​as follows: Figure 5 Calling the trained model can realize real-time gesture recognition. The collected real-time signal will be preprocessed in the same way as the signal in the training set, and then the LabVIEW program will be called to display the signal waveform in real time, and the python script will be called to realize gesture recognition, as shown in the figure below. Figure 6 shown.

[0060] In summary, the present invention proposes a gesture recognition method based on the deformation of the forearm muscle groups. By wearing highly sensitive and well-fitting strain sensors on the muscle groups that directly apply force, such as the flexor muscles on the inner side of the arm and the extensor muscles on the outer side of the arm, the surface deformation of the force-applying muscle groups is directly measured. Combined with the machine learning model designed by the present invention, gesture recognition is accurately achieved by capturing the deformation of the forearm muscle groups.

[0061] The present invention uses strain sensing to quantify the surface deformation of the flexor and extensor muscles, and uses a designed machine learning model for gesture recognition. It does not involve interaction with deep muscle electrical signals, so there is no signal reliability problem caused by electrical interface instability. In addition, the amount of strain data is greatly reduced compared to electromyographic signals, so it can effectively reduce the difficulty of signal acquisition and post-processing.

[0062] Compared with the traditional gesture recognition method, the main advantages of the present invention are:

[0063] (1) Data collection reliability is high. Compared with the gesture recognition method based on surface electromyographic signals, the gesture recognition method based on forearm muscle deformation proposed in the present invention does not involve interaction with deep muscle electrical signals, and there is no signal reliability problem caused by the instability of the electrical interface.

[0064] (2) The difficulty of data post-processing is reduced. Compared with the gesture recognition method based on surface electromyography signals, the gesture recognition method based on forearm muscle deformation of the present invention can obtain simpler and more intuitive strain data. Compared with electromyography signals, the amount of information is greatly reduced and does not contain frequency domain information, which can effectively reduce the difficulty of signal acquisition and post-processing.

[0065] (3) Efficient model structure design, integrating one-dimensional convolutional neural network and Transformer structure, improves the accuracy and processing speed of gesture recognition, while enhancing the model's deep understanding and generalization ability of time series data.

[0066] (4) It has a wider range of application scenarios. Compared with gesture recognition methods based on hand deformation, it can distinguish different gestures by detecting the contraction and fluctuation state of the forearm flexor and extensor muscles, which is expected to meet the potential needs of patients with partial forearm amputation in prosthetic control.

[0067] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0068] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0069] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.

[0070] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0071] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0072] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0073] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0074] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0075] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0076] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0077] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0078] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0079] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A gesture recognition method based on forearm muscle deformation, characterized in that: The following steps are involved: S1. Strain data collection: The subjects wore multiple strain sensors on the flexor and extensor muscles on the inside and outside of their arms to measure the body surface deformation of these parts under different gestures; S2. Gesture recognition model training: Preprocess the collected strain data and use an overlapping sliding window method to process the data to capture the detailed features of the overlapping parts and achieve data enhancement. Based on the preprocessed strain data, train a machine learning model that integrates a one-dimensional convolutional neural network and a Transformer structure to classify and recognize gestures. S3. Real-time gesture recognition: Use the trained model to pre-process the real-time collected signal and perform gesture recognition to achieve real-time gesture recognition.

2. The gesture recognition method based on forearm muscle deformation according to claim 1, characterized in that: Step S1 includes: at least two strain sensors respectively measure the deformation of the inner and outer muscle concentrations of the forearm to identify the deformation differences of the wearing parts under different gestures.

3. The gesture recognition method based on forearm muscle deformation according to claim 1 or 2, characterized in that: Step S1 includes: using a plurality of resistive strain sensors to detect a dense muscle group near the elbow.

4. The gesture recognition method based on forearm muscle deformation according to any one of claims 1 to 3, characterized in that: Step S1 includes: the subject's fingers and wrists move with different degrees of freedom, each movement lasts for a predetermined time and is repeated multiple times; the resistance response of the strain sensor generated by the strain under different gestures is converted into an analog voltage signal through a parallel voltage divider circuit, and the analog voltage signal is transmitted to a computer terminal to construct a data set.

5. The gesture recognition method based on forearm muscle deformation according to any one of claims 1 to 4, characterized in that: In step S2, the overlapping sliding window method for processing data specifically includes: using window length and step length as a window segmentation method, wherein the window length is used to determine the number of data points contained in each window, and the step length is used to determine the degree of overlap and the moving step between windows; for example, a 100-point window length and a 10-point step length are used as a window segmentation method. By processing the data in this way, the detailed features of the overlapping parts in the data are captured, and the accuracy of gesture recognition is improved as a data enhancement method.

6. The gesture recognition method based on forearm muscle deformation according to any one of claims 1 to 5, characterized in that: In step S2, the machine learning model integrating the one-dimensional convolutional neural network and the Transformer structure specifically includes: The spatial features between different channels in strain data are effectively extracted through a one-dimensional convolutional neural network; Handle time series classification tasks through the Transformer structure with self-attention mechanism to capture temporal dependencies; The strain data is taken as a time series and mapped into the latent vector space through position-encoded word embedding to extract temporal and spatial features, thereby realizing the classification and recognition of gesture actions.

7. The gesture recognition method based on forearm muscle deformation according to any one of claims 1 to 6, characterized in that: In step S2, the machine learning model integrating the one-dimensional convolutional neural network and the Transformer structure specifically includes the following structure: Preprocessing module: Process the noisy data and use a 100-point window length and a 10-point step length for data enhancement and segmentation; One-dimensional convolution layer: contains a one-dimensional convolution operation with a padding size of 3 and a stride of 1, which is used to extract local features; Batch Normalization Layer: Normalizes the output of the convolutional layer to stabilize the training process; Rectifier Linear Layer (ReLU): applies nonlinear activation function to increase the nonlinear expression ability of the model; Padding and convolution: Perform a one-dimensional convolution operation with a padding size of 1 and a stride of 1 to further extract features; Transformer encoder layer: contains position encoding and a multi-head self-attention mechanism with an embedding scale of 64, 4 heads and 9 layers, for processing time series data; Feedforward neural network: A 64*4 feedforward neural network is connected after the Transformer encoder layer to further process features; Pooling layer: performs pooling operations on features to reduce dimensions and extract important information; Fully connected layer: maps the extracted features to the final classification output with an output dimension of 64.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the gesture recognition method based on forearm muscle deformation according to any one of claims 1 to 7 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the gesture recognition method based on forearm muscle deformation according to any one of claims 1 to 7 is implemented.

Images