Lightweight network model for surface electromyography signal gesture recognition and construction method thereof

Abstract

This invention discloses a lightweight network model and its construction method for gesture recognition based on surface electromyography (EMG) signals, solving the problem of existing network structures being large and complex, making it difficult to effectively capture EMG information. The lightweight network model of this invention is a two-layer lightweight feature fusion structure: the first layer uses two cascaded lightweight multi-stream features, and the second layer uses a lightweight multi-scale feature fusion module to fuse the output. The model construction steps are: acquiring EMG signal data; preprocessing; constructing the network model; training the model; inputting EMG signals and obtaining the gesture type. This invention designs the feature fusion module based on the EMG acquisition location and inter-muscle group collaboration. Lightweight residual attention is added to the multi-stream feature fusion module, effectively extracting features without increasing additional computational overhead. It captures multi-scale features, enhances feature propagation, and improves recognition accuracy. The introduction of depthwise separable convolution into the multi-stream feature fusion module significantly reduces the number of parameters and computational load. It has applications in clinical medicine, robotics, and other fields.

Description

Technical Field

[0001] This invention belongs to the field of biosignal recognition technology, mainly involving surface electromyography (SEMG) signal recognition, specifically a lightweight network model and its construction method for SEMG signal gesture recognition. SEMG signals offer advantages such as being non-invasive, non-traumatic, and easy to operate. This makes SEMG signals of significant practical value in clinical medicine, industrial robot control, human-machine interface design, rehabilitation medicine, and consumer electronics. Background Technology

[0002] Electromyography (EMG) is a weak bioelectrical signal that records muscle activity. It originates when a person performs limb movements; the spine generates a control signal and transmits it to muscle fibers. After depolarization and repolarization, the muscle fibers generate action potentials at the neuromuscular junction. Depending on the acquisition method, EMG signals can be divided into two categories: surface EMG signals and invasive EMG signals. Surface EMG is a comprehensive photoelectric signal representing potential muscle movements on the skin surface. It is a non-stationary signal, and its intensity is sensitively proportional to the degree of muscle activity, allowing it to represent muscle activity to a certain extent. Compared to invasive EMG signals, surface EMG signals have advantages such as being non-invasive, non-traumatic, and easy to operate. This makes surface EMG signals of significant practical value in clinical medicine, industrial robot control, human-machine interface design, rehabilitation medicine, and consumer electronics. The applications of surface EMG signals are extensive, and their analysis has important clinical significance. In today's society, surface EMG signals are widely used in clinical medicine, biomedicine, and many other fields. Electromyography (EMG) enables the diagnosis of various muscle diseases, improving the accuracy of medical diagnoses. Furthermore, by analyzing the characteristics of EMG signals on the human body surface, precise control of prosthetic limb movements can be achieved, providing a more flexible lifestyle for patients who have lost limbs. Surface EMG signals also play a crucial role in motion analysis in rehabilitation medicine, helping medical professionals better understand patients' movement patterns and rehabilitation progress. Through these applications, disabled patients can perform some basic daily activities, improving their quality of life to some extent.

[0003] Currently, research on surface electromyography (SEMG) gesture recognition mainly falls into two categories: one is based on traditional engineering methods to analyze SEMG signals and perform gesture recognition and classification; the other is based on deep learning networks for feature extraction of SEMG signals for gesture recognition. Deep learning methods demonstrate significant advantages over traditional SEMG gesture recognition algorithms in several aspects. Better modeling of temporal information allows the model to more accurately capture the dynamic features in the EEMG signal. Multi-level feature learning improves the model's expressive power, enabling it to better adapt to variations in individuals and actions. Support for transfer learning enhances the model's generalization ability on other related tasks, adapting to personalized SEMG gesture recognition needs. The utilization of large-scale data and the introduction of adaptive and optimization algorithms further improve the performance of deep learning in the field of SEMG gesture recognition. Overall, these advantages make deep learning-based SEMG gesture recognition methods more competitive, especially in handling complex data and improving system performance.

[0004] Training convolutional neural network (CNN) models requires substantial hardware support. As the number of layers in a CNN increases, the model gains the ability to express more complex functions. In applications with a wide variety of gestures, researchers often choose to stack deeper layers in CNNs to improve classification performance and fully capture the features of electromyographic (EMG) signals. However, as CNN structures become more deep, wider, and more complex, their demands on computational power, storage space, and energy consumption gradually increase. When the model complexity exceeds the actual complexity of the task, it may lead to insufficient generalization ability for unknown data and model degradation. Currently, training CNNs not only requires higher hardware specifications but also takes longer.

[0005] With the widespread research into deep learning in the field of electromyography (EMG) signal gesture recognition, many neural network construction methods have been proposed. Nanjing University of Posts and Telecommunications disclosed a convolutional neural network for EMG signal gesture recognition in its patent application, "A Method for EMG Signal Gesture Recognition Based on a Two-Branch Multi-Stream Network" (Application No.: CN202111273722.0, Publication No.: CN113988135A). To improve gesture recognition accuracy, this invention constructs two branch networks from both a global and local perspective. One branch network consists of ten parallel network streams, a feature fusion layer, two fully connected layers, and a Softmax classifier. Each network stream comprises five one-dimensional convolutional layers and one long short-term memory layer. The other branch network contains three two-dimensional convolutional layers, two fully connected layers, and a Softmax classifier. Finally, a voting mechanism is used to combine the outputs of the two branch networks to determine the final recognition result, allowing the two branch networks to complement each other, compensating for their inherent shortcomings, and thus improving the accuracy of gesture recognition. However, its shortcomings lie in the fact that the design of the multi-stream architecture does not consider the positional information of the electrodes, processing all electrodes separately and ignoring the synergistic effect of muscle groups, thus failing to effectively distinguish different types of hand gestures. Furthermore, it does not consider how the complexity of the network structure can hinder the practical application of EMG gesture recognition. Its two branch networks, through multiple layers of convolutional operations, significantly increase the complexity of the network model, failing to meet the requirements of efficient operation on embedded devices and the high real-time performance demands of EMG gesture recognition models for model inference.

[0006] In summary, current methods for electromyography (EMG) gesture recognition suffer from the problem of neglecting the synergistic effects between muscle groups, failing to effectively extract features from surface EMG gesture signals, resulting in low gesture recognition accuracy. Furthermore, existing methods often pursue higher accuracy by increasing the number of CNN layers and convolutional kernels, leading to a proliferation of complex convolutional neural networks. While these networks have achieved some accuracy, their complex structures also incur significant computational and storage costs. As network model structures become more complex, the number of parameters and computational demands increase, leading to a greater reliance on GPUs. This limits the application of these networks on resource-constrained systems and devices, hindering their wider development. Most current CNN models are still cascaded structures. Although increasing the number of convolutional layers can enhance the feature extraction capabilities of cascaded networks, they cannot fully extract features from input data for sequences with long-term dependencies, such as sEMG signals. Recent methods have largely reported low accuracy, while achieving similar accuracy requires high computational costs and suffers from overfitting issues. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings and problems of the prior art and provide a lightweight network model for surface electromyography (EMG) signal gesture recognition that significantly reduces the number of parameters and computational load while improving recognition accuracy. This addresses the issues of low recognition rates and complex feature extraction in existing EMG gesture recognition network models, which fail to consider the practical applications of EMG gesture recognition network models in embedded devices and the high real-time requirements. The lightweight network model for EMG signal gesture recognition proposed in this invention has a simple structure, is easy to implement, and achieves high recognition accuracy with fewer parameters and less computational load.

[0008] This invention discloses a lightweight network model for gesture recognition based on surface electromyography (EMG) signals. It consists of a backbone network and a classification module connected in series. Between the input and output of the backbone network, two multi-stream convolutional networks are connected in series. The key feature is that, between the input and output of the lightweight network model's backbone network, a two-layer lightweight feature fusion network structure is established according to the EMG acquisition location. The first layer's lightweight multi-stream feature fusion module is essentially two interconnected multi-stream feature fusion modules, referred to as the first multi-stream feature fusion module and the second multi-stream feature fusion module. Each multi-stream feature fusion module is connected in series with its own fusion module, forming a unified lightweight multi-stream feature fusion module. The second layer includes a lightweight multi-scale feature fusion module, which has two convolutional structures adapted to the first layer. One convolutional structure performs feature fusion with the first multi-stream feature fusion module, and the other convolutional structure performs feature fusion with the second multi-stream feature fusion module. The output of the fusion module connected to the second multi-stream feature fusion module is the total output of the lightweight network model's backbone network. The multi-stream feature fusion module is composed of a depthwise separable convolutional structure and a lightweight residual attention module connected in series. The lightweight residual attention module is composed of a CA attention mechanism and a SAM attention mechanism connected in series, and a residual connection is used between the input and output of the lightweight residual attention module.

[0009] This invention also provides a method for constructing a lightweight network model for gesture recognition based on surface electromyography signals, characterized by the following steps:

[0010] (1) Acquisition of electromyographic signal data: Electromyographic signals of each subject were acquired using 12 Dear Hughes active dual differential wireless electrodes at a sampling rate of 2000 Hz. During the acquisition process, 8 electrodes were evenly distributed on the forearm, while the other 2 electrodes were located on the flexor digitorum superficialis and extensor digitorum superficialis, respectively, and the other 2 electrodes were distributed on the biceps and triceps. During the acquisition process, each hand gesture of each subject was repeated 6 times, each lasting 5 seconds, followed by a 3-second rest. The acquired electromyographic data was saved as a .mat file as the raw electromyographic signal data.

[0011] (2) Preprocessing of the acquired electromyographic (EMG) signals: The raw EMG signals were obtained by reading the EMG signal file. A digital notch filter and a Butterworth digital filter were used to filter out power frequency interference, obtaining the useful frequency band data of the EMG signals. The EMG signals were segmented using an overlapping sliding window, and the EMG signal data was converted into image data. In the specific experimental process, the repetition order of each gesture was numbered 1-6, and the experimental data generated from repetitions numbered 1, 3, 4, and 6 were used as the training set X. train Y train The experimental data generated repeatedly, numbered 2 and 5, were used as the test set X. test Y test The gesture recognition accuracy for each subject was defined as the ratio between the number of correctly predicted test samples and the total number of test samples. The overall gesture recognition accuracy was defined as the average gesture classification accuracy of all subjects in the experiment.

[0012] (3) Constructing a lightweight network model for gesture recognition based on surface electromyography signals, referred to as the lightweight network model: Between the input and output of the backbone network of the lightweight network model, there is a two-layer lightweight feature fusion network structure. The first layer of the lightweight multi-stream feature fusion module consists of two interconnected multi-stream feature fusion modules, referred to as the first multi-stream feature fusion module and the second multi-stream feature fusion module, respectively. Each multi-stream feature fusion module is connected in series with its own fusion module, forming a lightweight multi-stream feature fusion module as a whole. The second layer has a lightweight multi-scale feature fusion module, which has two convolutional structures adapted to the first layer. One convolutional structure performs feature fusion with the first multi-stream feature fusion module, and the other convolutional structure performs feature fusion with the second multi-stream feature fusion module. The output of the fusion module connected to the second multi-stream feature fusion module, which contains the convolutional structure of the lightweight multi-scale feature fusion module, is the total output of the backbone network of the lightweight network model. The multi-stream feature fusion module is composed of a depthwise separable convolutional structure and a lightweight residual attention module connected in series. The lightweight residual attention module is composed of a CA attention mechanism and a SAM attention mechanism connected in series, and a residual connection is used between the input and output of the lightweight residual attention module.

[0013] (4) Obtaining the lightweight network model with the best recognition performance through training: The lightweight network model is trained using preprocessed electromyography data, and its corresponding parameters are saved; First, the training set X is used... train Y train The lightweight network model is trained separately, and the adaptive learning rate for each parameter is calculated using the Adam optimizer, then passed through X. test Y testDetermine the recognition performance of the lightweight network model, determine the optimal parameters layer by layer, and save the final optimal lightweight network model obtained from training.

[0014] (5) Apply the trained lightweight network model to perform electromyographic gesture recognition: The optimal lightweight network model obtained from the training is used to input the electromyographic gesture signal to be recognized into the lightweight network model. The electromyographic gesture signal data is filtered and denoised to be converted into the electromyographic gesture image corresponding to the electromyographic gesture signal. Then, the electromyographic gesture image is input into the lightweight network model to obtain the result with the highest voting probability in the Softmax classifier as the predicted category.

[0015] This invention solves the problem that current gesture recognition methods cannot effectively capture diverse electromyographic signal information, resulting in low recognition accuracy. Simultaneously, it addresses the issue that current gesture recognition methods, which improve accuracy by stacking multi-layer network structures and increasing the number of convolutional kernels, require a large number of parameters and computational resources, making them unsuitable for practical applications on resource-constrained devices and mobile devices.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] (1) More refined perception: This invention constructs a lightweight network model for surface electromyography (EMG) signal gesture recognition. Considering the synergistic effect between muscle groups based on the EMG acquisition location, a two-layer lightweight feature fusion network structure is set up. This invention utilizes a multi-stream feature fusion module to more effectively focus on the features of different muscle regions, thereby capturing diverse EMG signal information. The lightweight residual attention module RCASM proposed in this invention is added after each multi-stream branch convolution, which adds almost no additional computational overhead, while enabling the neural network to have a more refined perception of different regions of the image, better understand the image content, and effectively extract EMG gesture features.

[0018] (2) Capturing Multi-Scale Features: This invention integrates feature information at different levels and scales through a lightweight multi-scale feature fusion module, enriching the model's feature representation capabilities, enhancing the model's understanding and representation of input data, and improving the model's receptive field size, robustness, and generalization ability. It captures multi-scale features while reducing the number of parameters, thus enhancing feature propagation.

[0019] (3) Reduction in parameter count and computational cost: This invention not only replaces traditional convolution with depthwise separable convolution in the multi-stream feature fusion module, but also incorporates a lightweight multi-scale feature fusion module and a lightweight residual attention module in the network model. The combination of these three modules effectively reduces the number of parameters and computational cost of the model. This invention achieves high accuracy in electromyographic gesture recognition with relatively low parameter count and computational cost. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the implementation process of the present invention;

[0021] Figure 2 This is a block diagram of the lightweight network model of the present invention;

[0022] Figure 3 This is the overall architecture diagram of the lightweight network model of the present invention;

[0023] Figure 4 This is a schematic diagram of the overlapping sliding window sampling method of the present invention;

[0024] Figure 5 This is a schematic diagram of the lightweight residual attention module of the present invention.

[0025] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Detailed Implementation

[0026] Example 1: Currently, most gesture recognition methods based on surface electromyography (SEMG) signals face problems such as complex feature extraction and low gesture recognition accuracy. As the number of layers in convolutional neural network models continues to increase, the demand for device memory and computing resources also grows. To address these issues, this invention conducts research and experiments, proposing a lightweight network model for SEMG signal gesture recognition and its construction method.

[0027] This invention presents a lightweight network model for gesture recognition based on surface electromyography (EMG) signals. It consists of a backbone network and a classification module connected in series. Existing technologies connect multiple multi-stream convolutional networks between the input and output of the backbone network, with these multiple multi-stream convolutional networks forming the backbone network. See also... Figure 2 , Figure 2 This is a block diagram of the lightweight network model of the present invention. The lightweight network model for gesture recognition based on surface electromyography (EMG) signals constructed in this invention, referred to simply as the lightweight network model, includes a two-layer lightweight feature fusion network structure between the input and output of the backbone network, based on the EMG acquisition location. The present invention considers the synergistic effect between muscle groups in designing the lightweight feature fusion module. The first layer of the lightweight multi-stream feature fusion module consists of two interconnected multi-stream feature fusion modules, referred to as the first multi-stream feature fusion module and the second multi-stream feature fusion module, respectively. Each multi-stream feature fusion module is connected in series with its own fusion module, forming the overall lightweight multi-stream feature fusion module. The second layer includes a lightweight multi-scale feature fusion module, which has two convolutional structures adapted to the first layer, see [link to relevant documentation]. Figure 3 , Figure 3This is a diagram of the overall architecture of the lightweight network model of the present invention. One convolutional structure includes two convolutional layers, which perform feature fusion with the first multi-stream feature fusion module. The other convolutional structure includes one convolutional layer, which performs feature fusion with the fusion module of the second multi-stream feature fusion module. The output of the fusion module, which is connected to the second multi-stream feature fusion module and contains the convolutional structure of the lightweight multi-scale feature fusion module, is the total output of the backbone network of the lightweight network model. The multi-stream feature fusion module of the present invention is composed of a depthwise separable convolutional structure and a lightweight residual attention module connected in series. See [link to documentation]. Figure 5 , Figure 5 This is a schematic diagram of the lightweight residual attention module of the present invention. The lightweight residual attention module is composed of CA attention mechanism and SAM attention mechanism connected in series, and residual connection is used between the input and output of the lightweight residual attention module.

[0028] Considering that in practical applications, the surface electromyography (EMG) signal gesture recognition research of this invention requires deploying the network model on mobile terminals or embedded devices, and also places high demands on the real-time performance of the network model inference, this leads to a significant contradiction. The lightweight network model of this invention combines EMG signals generated at different locations, and feature extraction from these signal combinations achieves high discriminative power when recognizing different types of gestures, thus improving the performance of gesture recognition. Meanwhile, existing methods cannot effectively extract features from EMG signal information and neglect the practical applications of surface EMG signal gesture recognition. The lightweight network model of this invention, by introducing a lightweight residual attention module and a lightweight multi-scale feature fusion module, extracts EMG features more effectively while significantly reducing the number of parameters and computational load. This invention introduces depthwise separable convolution into the multi-stream feature fusion module, greatly reducing the number of parameters and computational load. This invention avoids simply increasing the complexity of the network model to improve the accuracy of surface EMG gesture recognition; instead, it achieves high accuracy in gesture recognition through a simplified network model structure.

[0029] To address the issue of low gesture recognition accuracy, this invention analyzes the relationship between electromyography (EMG) signals and gesture movements by analyzing the distribution of muscle groups. The original EMG image is segmented into multiple sub-images based on muscle groups, and each sub-image is input into a multi-stream feature fusion module. This method aims to more comprehensively consider the synergistic effects of muscle groups, improving the performance of gesture recognition. To address the problems of complex model layers and high parameter computation, this invention utilizes a lightweight residual attention module to enable the model to perceive different regions of the image more subtly and better understand image content with almost no increase in computation. This invention constructs a lightweight network model using a two-layer feature fusion module. The lightweight multi-scale feature fusion module enriches the model's feature representation capabilities with fewer parameters, enhancing the model's understanding and representation of input data, and improving the model's receptive field size, robustness, and generalization ability. Traditional convolutions in the multi-stream feature fusion module are replaced with depthwise separable convolutions. While maintaining relatively high accuracy, this significantly reduces the number of model parameters and computational burden, improving the model's practical usability on resource-constrained mobile devices and embedded systems.

[0030] This invention extracts features from different muscle regions using two multi-stream feature fusion modules. For cascaded structures that cannot fully extract features from sEMG input data, it effectively captures diverse electromyographic signal information. Simultaneously, it incorporates a lightweight attention mechanism and lightweight multi-scale feature fusion, enabling the model to better understand and express the semantic information in the input data, thus improving the overall feature extraction efficiency. The lightweight network model proposed in this invention has a simplified structure and high efficiency in extracting surface electromyographic signal features. While effectively reducing the number of model parameters and computational burden, it also achieves higher gesture recognition accuracy compared to other methods.

[0031] Example 2: The overall structure of the lightweight network model for surface electromyography (EMG) signal gesture recognition is the same as in Example 1. The two-layer lightweight feature fusion network structure in the backbone of the lightweight network model for EMG signal gesture recognition of this invention is described in [reference needed]. Figure 2The lightweight network model comprises three main modules: a multi-stream feature fusion module, a lightweight residual attention module, and a lightweight multi-scale feature fusion module. Based on the electrode positions acquired by EMG, this invention divides the input data into three parts within each multi-stream feature fusion module. During the multi-stream convolution stage, three multi-stream convolutional network branches are employed, corresponding to the three data input streams. Each input stream uses an independent CNN network for feature extraction. After each multi-stream convolution, a batch normalized convolutional module and a lightweight residual attention module are added. The lightweight residual attention module, formed by CA and SAM through residual connections, captures orientation and position information, helping the model to more accurately locate and identify targets of interest. This allows the model to have a more detailed perception of different regions of the image and a better understanding of the image content. In the lightweight network model backbone, two multi-stream feature fusion modules and two branches of the lightweight multi-scale feature fusion module perform feature fusion. The lightweight multi-scale feature fusion module uses multiple convolutional layers with a kernel size of 3*3 to capture the same characteristics as convolutional layers with kernel sizes of 5*5 and 7*7, effectively reducing the number of parameters while capturing multi-scale feature information of electromyography signals. Replacing the ordinary convolutions in the multi-stream feature fusion module with depthwise separable convolutions significantly reduces the number of parameters and computational cost. When classifying gestures through the classification module, considering the excessive number of nodes and the large proportion of parameters in the flattening layer, the flattening layer is replaced with a global average pooling layer to make the network model more lightweight and reduce the risk of overfitting. Finally, the vector obtained by passing the output of the fully connected layer through the Dropout layer is input into the Softmax layer to obtain the result with the highest voting probability as the predicted category.

[0032] This invention improves gesture recognition performance by comprehensively considering the synergistic effects of muscle groups through a multi-stream feature fusion module. A lightweight residual attention module enables the model to perceive different regions of an image more subtly with almost no increase in computation, leading to a better understanding of image content. A lightweight multi-scale feature fusion module enriches the model's feature representation capabilities with fewer parameters, enhancing its understanding and representation of input data, and improving its receptive field size, robustness, and generalization ability. Replacing traditional convolutions in the multi-stream feature fusion module with depthwise separable convolutions significantly reduces the number of parameters and computational burden while maintaining relatively high accuracy, improving the model's practical usability on resource-constrained mobile devices and embedded systems.

[0033] Example 3: This invention also presents a lightweight network model construction method for gesture recognition based on surface electromyography (EMG) signals. This invention considers the synergistic effects between muscle groups and constructs a multi-stream feature fusion module based on electrode acquisition locations to extract features from different muscle regions. To combine feature information at different scales, a lightweight multi-scale feature fusion module is constructed, enriching the model's feature representation capabilities with fewer parameters. By constructing a lightweight residual attention module, the feature representation of the input data is learned more effectively than existing attention modules with almost no increase in computation. See also... Figure 1 , Figure 1 This is a flowchart of the implementation of the present invention, which includes the following steps:

[0034] (1) Acquisition of electromyographic signal data: Electromyographic signals of each subject were acquired using 12 Dear Hughes active dual differential wireless electrodes at a sampling rate of 2000 Hz. During the acquisition process, 8 electrodes were evenly distributed on the forearm, while the other 2 electrodes were located on the flexor digitorum superficialis and extensor digitorum superficialis, respectively, and the other 2 electrodes were distributed on the biceps and triceps. During the acquisition process, each hand gesture of each subject was repeated 6 times, each lasting 5 seconds, followed by a 3-second rest. The acquired electromyographic data was saved as a .mat file and stored as the raw electromyographic signal data in the electromyographic signal file.

[0035] (2) Preprocessing of the acquired electromyographic (EMG) signals: The raw EMG signals are obtained by reading the EMG signal file. A digital notch filter and a Butterworth digital filter are used to filter out power frequency interference and obtain the data in the useful frequency band of the EMG signals; see [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram of the overlapping sliding window sampling method of the present invention; the overlapping sliding window is used to segment the electromyographic signal and convert the electromyographic signal data into image data; in the specific experimental process, the repetition order of each gesture is numbered 1-6, and the experimental data generated by the repetitions numbered 1, 3, 4, and 6 are used as the training set X. train Y train The experimental data generated repeatedly, numbered 2 and 5, were used as the test set X. test Y test The gesture recognition accuracy for each subject is defined as the ratio between the number of correctly predicted test samples and the total number of test samples in the experiment; the overall gesture recognition accuracy is defined as the average gesture classification accuracy of all subjects in the experiment.

[0036] (3) Construction of a lightweight network model for surface electromyography (EMG) signal gesture recognition: The lightweight network model for EMG signal gesture recognition constructed in this invention is referred to as the lightweight network model. Between the input and output of the backbone network of the lightweight network model, this invention employs a two-layer lightweight feature fusion network structure based on the EMG acquisition location. This invention considers the synergistic effect between muscle groups in designing the lightweight feature fusion module. The first layer of the lightweight multi-stream feature fusion module consists of two interconnected multi-stream feature fusion modules, referred to as the first multi-stream feature fusion module and the second multi-stream feature fusion module, respectively. Each multi-stream feature fusion module is connected in series with its own fusion module, forming a unified lightweight multi-stream feature fusion module. The second layer includes a lightweight multi-scale feature fusion module, which has two convolutional structures adapted to the first layer. See also... Figure 3 The lightweight multi-scale feature fusion module of the two-layer lightweight feature fusion network structure has two convolutional structures. One convolutional structure includes two convolutional layers, which perform feature fusion with the first multi-stream feature fusion module. The other convolutional structure includes one convolutional layer, which performs feature fusion with the second multi-stream feature fusion module. The output of the fusion module, which is connected to the second multi-stream feature fusion module and contains the convolutional structure of the lightweight multi-scale feature fusion module, is the total output of the backbone network of the lightweight network model. The multi-stream feature fusion module of this invention is composed of a depthwise separable convolutional structure and a lightweight residual attention module connected in series. See [link to documentation]. Figure 5 The lightweight residual attention module is composed of CA attention mechanism and SAM attention mechanism connected in series, and residual connection is used between the input and output of the lightweight residual attention module.

[0037] (4) Obtaining the lightweight network model with the best recognition performance through training: The lightweight network model is trained using preprocessed electromyography data, and its corresponding parameters are saved; First, the training set X is used... train Y train The lightweight network model is trained separately, and the adaptive learning rate for each parameter is calculated using the Adam optimizer, then passed through X. test Y test The recognition performance of the lightweight network model is determined, the optimal parameters are determined layer by layer, and the lightweight network model with the best recognition performance is obtained through training. The final optimal lightweight network model obtained from the training is saved.

[0038] (5) Apply the trained lightweight network model to perform electromyographic gesture recognition: The optimal lightweight network model obtained from the training is used to input the electromyographic gesture signal to be recognized into the lightweight network model. The electromyographic gesture signal data is filtered and denoised to be converted into an electromyographic gesture image corresponding to the electromyographic gesture signal. Then, the electromyographic gesture image is input into the lightweight network model of the present invention to obtain the result with the highest voting probability in the Softmax classifier as the predicted category, thus completing the electromyographic gesture recognition.

[0039] The multi-stream feature fusion module employed in this invention can more comprehensively consider the synergistic effects of muscle groups, improving the performance of gesture recognition. The lightweight residual attention module, with almost no increase in computation, enables the model to perceive different regions of the image more meticulously, better understanding the image content. The lightweight multi-scale feature fusion module, with fewer parameters, enriches the model's feature representation capabilities, enhances the model's understanding and representation of input data, and improves the model's receptive field size, robustness, and generalization ability. By replacing traditional convolutions in the multi-stream feature fusion module with depthwise separable convolutions, while maintaining relatively high accuracy, the number of model parameters and computational burden are significantly reduced, improving the model's practical usability on resource-constrained mobile devices and embedded systems.

[0040] Example 4: Lightweight Network Model for Surface Electromyography (SEMG) Signal Gesture Recognition and its Construction Method. Similar to Examples 1-3, this invention constructs a lightweight network model for SEMG signal gesture recognition, referred to as the lightweight network model. The backbone of the constructed lightweight network model contains a two-layer lightweight feature fusion network structure, including three main modules: a multi-stream feature fusion module, a lightweight residual attention module, and a lightweight multi-scale feature fusion module. Based on the electrode positions for EMG acquisition and considering the synergistic effect between muscle groups, the input data is divided into three parts in each multi-stream feature fusion module. In the multi-stream convolution stage, three multi-stream convolutional network branches are used, corresponding to the three data input streams. Each input stream uses an independent CNN network for feature extraction. After each multi-stream convolution, a batch normalized convolutional module and a lightweight residual attention module are added. The lightweight residual attention module is formed by CA and SAM through residual connections. In the backbone of the lightweight network model, two multi-stream feature fusion modules and two branches of the lightweight multi-scale feature fusion module perform feature fusion. The lightweight multi-scale feature fusion module uses multiple convolutional layers with a kernel size of 3*3 to capture the same characteristics of convolutional layers with kernel sizes of 5*5 and 7*7. The ordinary convolutions of the multi-stream feature fusion module are replaced with depthwise separable convolutions. When performing gesture classification through the classification module, the flattening layer is replaced with a global average pooling layer. Finally, the vector obtained by passing the output of the fully connected layer through the Dropout layer is input into the Softmax layer to obtain the result with the highest voting probability as the predicted category.

[0041] This invention comprehensively considers the synergistic effects of muscle groups through a multi-stream feature fusion module, better focusing on the characteristics of different muscle regions and effectively capturing diverse electromyographic (EMG) signal information. By fusing EMG signal features from multiple locations, this invention more accurately identifies and distinguishes different muscle activities or movement types. The multi-stream feature fusion information better captures subtle differences in movements, improving the accuracy and reliability of movement recognition and enhancing the performance of gesture recognition. The lightweight residual attention module allows the model to focus more intently on task-critical information with almost no increase in computation, improving model performance and generalization ability. The lightweight multi-scale feature fusion module combines feature information from different scales with fewer parameters, improving model performance, enriching the model's feature representation capabilities, enhancing the model's understanding and representation of input data, and increasing the model's receptive field size, robustness, and generalization ability.

[0042] Example 5: Lightweight Network Model and Construction Method for Surface Electromyography Signal Gesture Recognition. Similar to Examples 1-4, the two multi-stream feature fusion modules of the first-layer lightweight network structure of this invention are described in [reference needed]. Figure 3In each multi-stream feature fusion module, the input data is divided into three parts. When the time window length is 200 milliseconds, the number of samples collected by each electrode is 400, i.e., T = 400. This invention considers the positional information of these three input electrodes. Columns 1 to 8 represent equidistant electrodes evenly distributed around the forearm, while columns 9 and 10 contain signals collected from the flexor and superficial extensor digitorum muscles. Furthermore, columns 11 and 12 contain signals from the main muscle activity points, namely the biceps and triceps brachii. Dividing the input data into three parts according to the electrode distribution can be represented as p1∈R 400 ×8×1 p2∈R 400×2×1 p3∈R 400×2×1 Each input stream focuses on features from different muscle regions. This invention constructs two multi-stream feature fusion modules, with each multi-stream convolution followed by a batch-normalized convolution module and a lightweight residual attention module. The first multi-stream feature fusion module has convolution kernels of size 64, composed of depthwise separable convolutions; the second multi-stream feature fusion module has convolution kernels of size 128, composed of depthwise separable convolutions. Assume the input matrix size is D. in The depth of the input feature matrix is ​​M, and the size of the convolution kernel is D. k The depth of the output feature matrix is ​​N, P D P represents the number of parameters in a depthwise separable convolution. S This represents the number of parameters in a regular convolution. (C) D C represents the computational cost of depthwise separable convolution. S This indicates the computational cost of a regular convolution; depthwise convolutions can be separated from regular convolutions.

[0043] Parameter comparison:

[0044]

[0045] Comparison of computational complexity:

[0046]

[0047] A comparison of the number of parameters and computational cost between depthwise separable convolution and ordinary convolution demonstrates the significant reduction in parameters and computational cost achieved by the depthwise separable convolution method of this invention. This reduction is primarily attributed to two key factors: the depth N of the input feature map and the size D of the convolution kernel. k When the kernel size is large or the depth of the input feature map is large, this invention demonstrates significant advantages in reducing parameter storage and computational costs through depth-separable convolution.

[0048] This invention improves the performance and effectiveness of electromyography (EMG) signal processing through a multi-stream feature fusion module, enhancing the ability to recognize, control, and interpret muscle activity. It better focuses on the characteristics of different muscle regions and effectively captures diverse EMG signal information. Multi-stream feature fusion combines signal features from different locations, thereby enhancing the richness of feature representation and enabling the model to more comprehensively understand muscle activity, thus improving gesture recognition performance. By replacing traditional convolutions in the multi-stream feature fusion module with depthwise separable convolutions, the number of model parameters and computational burden are significantly reduced while maintaining relatively high accuracy, improving the model's practical usability on resource-constrained mobile devices and embedded systems.

[0049] Example 6: Lightweight Network Model and Construction Method for Surface Electromyography Signal Gesture Recognition. Similar to Examples 1-5, the lightweight residual attention module of this invention includes two attention mechanisms: coordinate attention mechanism (CA) and spatial attention mechanism (SAM). See [link to relevant documentation]. Figure 5 This invention connects CA and SAM in series and introduces a residual structure to connect the input data residual of CA to the output data of SAM, forming the total output of the lightweight residual attention module RCASM.

[0050] By employing a lightweight residual attention module, lightweight network models can learn the relationships between channels in the input feature map and dynamically adjust the weights of each channel, allowing the network model to focus more on channels important to the current task. This helps improve the network model's ability to distinguish between different features, reduces the impact of redundant information, and improves performance and generalization ability. Furthermore, the lightweight residual attention module allows lightweight network models to learn the spatial structure of the input feature map and dynamically adjust the weights of different spatial locations, enabling the network model to focus more on spatial locations important to the current task. This helps the network model better capture local features in the input data, improving the network model's perceptual ability and accuracy. In summary, the lightweight residual attention module in lightweight network models reduces the number of network model parameters and computational cost while helping the network model learn the feature representation of the input data more effectively, improving the network model's performance and generalization ability.

[0051] Example 7: Lightweight network model and construction method for gesture recognition based on surface electromyography signals. Same as Examples 1-6. For the lightweight multi-scale feature fusion module of this invention, see [link to example 1]. Figure 3In the lightweight multi-scale feature fusion module, multiple Conv layers with a kernel size of 3*3 are used to efficiently capture the same features of 5*5 and 7*7 kernels. The input first passes through two parallel streams. One stream is a multi-stream feature fusion module with a convolutional kernel size of 64. The second stream contains two 3*3 Conv layers stacked sequentially with kernel sizes of 16 and 32, extracting features with a receptive field size of 5. Features from the two streams are fused to ensure that the features have receptive fields of 3 and 5. The output of this layer is then fed into another multi-stream feature fusion module with a convolutional kernel size of 128 and a stride of 3*3 to increase the receptive field size. Specifically, this multi-stream feature fusion module has features with receptive fields of 7 and 5. The output of this multi-stream feature fusion module is then fused again with the output of another 3*3 Conv layer with a kernel size of 32 to ensure multi-scale feature extraction, as it extracts features with receptive fields of 3, 5, and 7.

[0052] By combining feature information from different scales with fewer parameters through a lightweight multi-scale feature fusion module, the model performance can be improved. The multi-scale feature fusion module can capture the size, shape and texture variations of data more comprehensively, improve the network model's ability to understand complex scenes and resist interference, enrich the model's feature expression ability, enhance the model's ability to understand and represent input data, improve the model's receptive field size, robustness and generalization ability, and enable the lightweight model to understand and process input data more comprehensively.

[0053] The lightweight network model for surface electromyography (EMG) signal gesture recognition of this invention can effectively extract diverse EMG signal information, maintaining high recognition accuracy with relatively few parameters and computational load. Due to the non-invasive, non-surgical, and simple operation advantages of EMG signals, this lightweight network model can be used to analyze and interpret patients' EMG signal data, aiding doctors in diagnosing muscle and neuromuscular diseases. Doctors can more accurately assess patients' muscle function and motor control capabilities, and develop personalized treatment plans. This lightweight network model can also be used in industrial robot control systems to achieve real-time adjustment and optimization of robot movements. By capturing the operator's muscle activity signals and inputting them into the network model, the robot can respond to operator commands more intelligently, improving work efficiency and safety. Furthermore, this lightweight network model can serve as a key component in human-machine interface design, enabling EMG-based human-machine interaction. By analyzing the user's EMG signals and applying the results to interface design, a more natural and intelligent human-machine interaction experience can be achieved, improving user convenience and overall experience. The lightweight network model for surface electromyography (EMG) signal gesture recognition of this invention can be applied to consumer electronics products to achieve functions such as intelligent health monitoring and motion tracking. By analyzing the user's EMG signal data and connecting it with smart devices, personalized health management and exercise guidance services can be provided to users, improving their quality of life and health.

[0054] Example 8: A lightweight network model for gesture recognition based on surface electromyography signals and its construction method are the same as in Examples 1-7. To achieve the above objectives, the technical solution adopted by this invention includes the following steps:

[0055] (1) Acquire electromyographic signal data; record the electromyographic signal data corresponding to the human body when performing a hand gesture by using an electrode acquisition device.

[0056] (2) Preprocess the electromyographic signals;

[0057] (2a) A 50Hz digital notch filter and a 30-300Hz Butterworth digital filter were designed in the preprocessing to filter out power frequency interference and obtain data of the useful frequency band of electromyography signal; then, the processed signal was normalized by Min-Max to ensure that the signal values ​​are comparable.

[0058] (2b) Convert the electromyographic signal data into image form. In the sliding window method, the sliding step size s is usually set to be smaller than the window length w. This generates data segments with some overlap, reducing feature loss caused by segmentation; this strategy helps retain more information about the gesture and improves the robustness of subsequent analysis; see [link to relevant documentation]. Figure 4This invention employs an overlapping sliding window technique to segment the previously extracted muscle electromyography signal envelope dataset. This method allows for control of the overlap between data windows by adjusting the window size and stride, and decomposes the signal into muscle activity segments for feature extraction. The window size (w) determines the duration of each window, and the stride (s) defines the time interval between windows, controlling their sliding speed. Smaller strides increase overlap and provide more data for training and evaluation. In the specific experiment, the repetition order of each gesture is numbered 1-6. The experimental data generated by repetitions numbered 1, 3, 4, and 6 are used as the training set, and the experimental data generated by repetitions numbered 2 and 5 are used as the test set. This partitioning method emphasizes the independence of individual test samples between different tests.

[0059] (2c) The gesture recognition accuracy for each subject is defined as the ratio between the number of correctly predicted test samples and the total number of test samples. The overall gesture recognition accuracy is defined as the average gesture classification accuracy of all experimental individuals in the experiment.

[0060] (3) Construct a lightweight network model for feature extraction and gesture classification;

[0061] See Figure 2 The lightweight network model backbone consists of three main modules: a multi-stream feature fusion module, a lightweight residual attention module, and a lightweight multi-scale feature fusion module.

[0062] (3a) Multi-stream feature fusion module. Based on the electrode positions of the EMG acquisition, the input data is divided into three parts in each multi-stream feature fusion module. When the time window length is 200 milliseconds, the number of samples acquired by each electrode is 400, i.e., T = 400. These three inputs can be represented as p1 ∈ R. 400×8×1 p2∈R 400×2×1 p3∈R 400×2×1 In the multi-stream convolution stage, three convolutional network branches are used, corresponding to the three data input streams. Each input stream uses an independent CNN network for feature extraction. Furthermore, a batch normalized convolutional module and a lightweight residual attention module are added after each multi-stream convolution.

[0063] (3b) Lightweight Residual Attention Module. This module includes two attention mechanisms: Coord Attention (CA) and Spatial Attention Mechanism (SAM). Coord Attention (CA) is a novel attention mechanism designed for lightweight networks that embeds positional information into channel attention. CA captures not only cross-channel information but also orientation and positional awareness, helping the model to more accurately locate and identify targets of interest. Furthermore, CA is simple, flexible, efficient, and practical, easily integrated into classic modules with minimal additional computational overhead, thus improving network accuracy. SAM is a technique used in computer vision and deep learning, designed to simulate the human visual system, enabling neural networks to focus more on important regions or features when processing images. The combination of CA and SAM allows the model to perceive different regions of an image more nuancedly, leading to a better understanding of image content. In specific experiments, this invention combines CA and SAM, connecting the input residuals to the output, forming the lightweight residual convolutional attention module RCASM.

[0064] (3c) Lightweight multi-scale feature fusion module;

[0065] Multi-scale feature fusion, by integrating feature information at different levels and scales, enriches the model's feature representation capabilities, enhances its understanding and representation of input data, and improves its receptive field size, robustness, and generalization ability. In convolutional neural networks, small filters like 3x3 are suitable for capturing subtle features in images, such as edges, textures, and fine structures. Large filters like 5x5 and 7x7 are suitable for capturing overall structure and global information in images, such as object shapes and region distribution; however, large filters require more parameters than small filters. Specifically, a Conv layer with a 5x5 kernel requires more than twice the parameters of a Conv layer with the same number of kernels as a 3x3 kernel layer, while a Conv layer with a 7x7 kernel requires more than five times the parameters of a Conv layer with a 3x3 kernel. Therefore, to maintain a lightweight architecture, this invention uses multiple Conv layers with a kernel size of 3*3 to efficiently capture the same characteristics of 5*5 and 7*7 kernels; for example, two Conv layers with a filter size of 3*3 can be used sequentially to extract features with a receptive field size of 5. Similarly, three Conv layers with a kernel size of 3*3 can be used sequentially to extract features with a receptive field size of 7; thus, using multiple Conv layers with a kernel size of 3*3 significantly reduces the number of parameters compared to using a single Conv layer with a kernel size of 5*5 or 7*7.

[0066] (4) Perform gesture classification using the classification module and output the classification results;

[0067] (4a) In order to further reduce the number of parameters and memory storage burden in the model, considering that the number of nodes in the flattening layer is too large and the number of parameters accounts for a large proportion, the present invention adopts the strategy of replacing the flattening layer with the global average pooling layer to design a lighter network model, and can reduce the risk of overfitting.

[0068] (4b) Finally, the vector obtained by passing the output of the fully connected layer through the Dropout layer is input into the Softmax layer to obtain the result with the highest voting probability as the predicted category.

[0069] (5) The model is trained using the preprocessed electromyography data to obtain the network model with the best recognition performance, and its corresponding parameters are saved; firstly, the training set X is used. train Y train The lightweight network model is trained separately, and the adaptive learning rate for each parameter is calculated using the Adam optimizer, then passed through X. test Y test Determine the recognition performance of the network model, determine the optimal parameters layer by layer, and save the optimal network model obtained from the final training.

[0070] (6) Application Model: Input the electromyographic signal to be identified into the network model, filter and denoise the signal data, convert it into an image, and then input the data into the network to obtain the result with the highest voting probability in the Softmax classifier as the predicted category.

[0071] Furthermore, the gesture recognition process of the network is as follows: The data sample X obtained in step 2 is... train Y train As the standard input to the network, the data from each channel is fed into the respective network streams. Then, two multi-stream feature fusion layers and a multi-scale feature fusion layer are used to merge the features output from each network stream, which are then fed into a global average pooling layer for dimensionality transformation. Finally, the classifier receives the output features, performs gesture classification based on the features, and outputs the recognition result.

[0072] The following is a more detailed example to further illustrate the invention.

[0073] Example 9, referring to Figure 1 The present invention includes the following steps:

[0074] Step 1) Acquire electromyographic (EMG) signal data; the specific EMG information acquisition information is as follows: EMG signals were acquired using 12 Delsys Trigno active dual-differential wireless electrodes at a sampling rate of 2000 Hz. During the acquisition process, 8 electrodes were evenly distributed on the forearm, while the other 2 electrodes were located on the flexor and extensor digitorum superficialis muscles, respectively, and the remaining 2 electrodes were distributed on the main activity points of the biceps and triceps muscles. Each gesture was repeated 6 times, each lasting 5 seconds, followed by a 3-second rest.

[0075] Step 2) Preprocessing of electromyographic signals; obtaining the training sample set X train Y train and test sample set X test Y test ;

[0076] Step 2a) In the preprocessing of the original electromyography (EMG) signal, a 50Hz digital notch filter and a 30-300Hz Butterworth digital filter were designed to filter out power frequency interference and obtain data in the useful frequency band of the EMG signal. Next, the processed signal was subjected to Min-Max normalization to ensure comparability between signal values.

[0077] Step 2b) Convert the electromyographic signal data into an image format. By performing a dimensionality-upgrading operation on the electromyographic time-domain image, a new dimensionality-upgraded electromyographic time-domain image is obtained as p∈R. C×l×N We can consider C as the length of the image, l as the width, and N as the number of feature channels. Setting the parameter N=1 indicates that the image is a single-channel feature image, i.e., a grayscale image with only one channel. The electromyography (EMG) signal sequence is divided into multiple data segments using a sliding window method, with each segment recording the changes in the EMG signal over a short period. Specifically, if the sampling frequency of the EMG data is f, the initial acquisition time is t1, and the end time is t2, let the window length of the sliding window be w, and the sliding step size be s (in milliseconds). During the time interval from t1 to t2, slicing can generate m data segments of equal length.

[0078]

[0079] See Figure 4 This invention employs overlapping sliding windows to segment electromyographic signals, such as... Figure 4 As shown, the window size is 200ms and the step size is 50ms. In the specific experiment, the gesture information in the resting state is first removed, and then the repetition order of each gesture is numbered 1-6. The experimental data generated by the repetitions numbered 1, 3, 4, and 6 are used as the training set X. train Y trainThe experimental data generated repeatedly, numbered 2 and 5, were used as the test set X. test Y test .

[0080] Step 2c) The gesture recognition accuracy for each subject is defined as the ratio between the number of correctly predicted test samples and the total number of test samples in the trial; the formula for the gesture classification accuracy (Acc) for each subject is as follows:

[0081]

[0082] Overall Recognition Accuracy (ORA) is defined as the average gesture classification accuracy of all individuals in the experiment, as shown in the following formula:

[0083]

[0084] N represents the total number of subjects.

[0085] Step 3) Construct a lightweight network model for feature extraction and gesture classification;

[0086] See Figure 2 The lightweight network model backbone consists of three main modules: a multi-stream feature fusion module, a lightweight residual attention module, and a lightweight multi-scale feature fusion module. The connections between these modules are as follows: Figure 2 As shown;

[0087] Step 3a) Multi-stream feature fusion module. Based on the electrode locations for EMG acquisition, the input data is divided into three parts in each multi-stream feature fusion module. When the time window length is 200 milliseconds, the number of samples acquired by each electrode is 400, i.e., T = 400. This invention considers the location information of these three input electrodes. Columns 1 to 8 represent equidistant electrodes evenly distributed around the forearm; columns 9 and 10 contain signals acquired from the flexor and superficial extensor digitorum muscles. Furthermore, columns 11 and 12 contain signals from the main muscle activity points, namely the biceps and triceps brachii. The input data is divided into three parts according to the electrode distribution, see [link to documentation]. Figure 3 This can be represented as p1∈R 400×8×1 p2∈R 400×2×1 p3∈R 400×2×1Each input stream focuses on features of different muscle regions. In the multi-stream convolution stage, three convolutional network branches are used, corresponding to the three data input streams, with each input stream using an independent CNN network for feature extraction. This design better focuses on the features of different muscle regions and effectively captures diverse electromyographic (EMG) signal information. Furthermore, a batch-normalized convolutional module and a lightweight residual attention module are added after each multi-stream convolution. The batch-normalized layer helps improve the model's stability and accelerate convergence, while the lightweight residual attention module enables the network to more effectively capture key information in the input data, enhancing the model's expressive ability when extracting deep semantic features. This invention constructs two multi-stream feature fusion modules. The first multi-stream feature fusion module has a convolutional kernel size of 64, composed of depthwise separable convolutions. The second multi-stream feature fusion module has a convolutional kernel size of 128, composed of depthwise separable convolutions. By introducing depthwise separable convolutions into multi-stream feature fusion, it not only better focuses on the features of different muscle regions and effectively captures diverse EMG signal information, but also significantly reduces the number of model parameters and computational burden. Assume the input matrix has size Din, the input feature matrix has depth M, the convolution kernel has size Dk, and the output feature matrix has depth N, P D P represents the number of parameters in a depthwise separable convolution. S This represents the number of parameters in a regular convolution. (C) D C represents the computational cost of depthwise separable convolution. S This indicates the computational cost of a regular convolution; depthwise convolutions can be separated from regular convolutions.

[0088] Parameter comparison:

[0089]

[0090] Comparison of computational complexity:

[0091]

[0092] A comparison of the number of parameters and computational cost between depthwise separable convolution and ordinary convolution demonstrates the significant reduction in parameters and computational cost achieved by the depthwise separable convolution method of this invention. This reduction is primarily attributed to two key factors: the depth N of the input feature map and the size D of the convolution kernel. k When the kernel size is large or the depth of the input feature map is large, this invention demonstrates significant advantages in reducing parameter storage and computational costs through depth-separable convolution.

[0093] Step 3b) Lightweight Residual Attention Module; comprising two attention mechanisms, CA and SAM. In specific experiments, this invention concatenates CA and SAM and introduces a residual structure to form the lightweight residual convolutional attention module RCASM. The specific structure is described in [reference needed]. Figure 5This invention incorporates RCASM after the multi-stream convolution in each multi-stream feature fusion module; the specific structure is described in [reference needed]. Figure 3 .

[0094] Step 3c) Lightweight multi-scale feature fusion module. This invention uses multiple Conv layers with a kernel size of 3*3 to efficiently capture the same features of 5*5 and 7*7 kernels, such as... Figure 2 As shown; the input first passes through two parallel streams. One stream is a multi-stream feature fusion module with a kernel size of 64. The second stream contains two stacked 3*3 Conv layers with kernel sizes of 16 and 32, extracting features with a receptive field size of 5. Features from the two streams are fused to ensure that the features have receptive fields of 3 and 5. Then, the output of this layer is fed into another multi-stream feature fusion module with a kernel size of 128 and a stride of 3*3 to increase the receptive field size. Specifically, this multi-stream feature fusion module has features with receptive fields of 7 and 5. The output of this multi-stream feature fusion module is then fused again with the output of another 3*3 Conv layer with a kernel size of 32 to ensure multi-scale feature extraction, as it extracts features with receptive fields of 3, 5, and 7. Simultaneously, in this module, the input of each module contains the feature maps of the outputs of all previous modules and the feature maps of the model input.

[0095] Step 4) Classify gestures using the classification module and output the classification results;

[0096] Step 4a) In order to further reduce the number of parameters and memory storage burden in the model, considering that the number of nodes in the flattening layer is too large and the number of parameters accounts for a large proportion, this invention adopts the strategy of replacing the flattening layer with a global average pooling layer to design a lighter network model.

[0097] Step 4b) Finally, the output of the fully connected layer of size 128 is passed through a Dropout layer of size 0.3 to obtain a vector, which is then input into a Softmax layer of size 49 to obtain the result with the highest voting probability as the predicted category.

[0098] Step 5) Train the model using the preprocessed electromyography data to obtain the network model with the best recognition performance and save its corresponding parameters; firstly, use the training set X train Y train The lightweight network model is trained separately, and the adaptive learning rate for each parameter is calculated using the Adam optimizer, then passed through X. test Y test Determine the recognition performance of the network model, determine the optimal parameters layer by layer, and save the optimal network model obtained from the final training.

[0099] Step 6) Apply the model. Input the electromyographic signal to be identified into the network model, filter and denoise the signal data, and then input the data into the network to obtain the result with the highest voting probability in the Softmax classifier as the predicted category.

[0100] The technical effects of this invention will be further explained below with reference to experiments and data;

[0101] Example 10: A lightweight network model for gesture recognition based on surface electromyography signals and its construction method, same as Examples 1-9.

[0102] Experimental Environment: All experiments in this invention were implemented using TensorFlow 2.7.4 and Keras version 2.7.0. Training was performed on a GPU equipped with an RTX 2060Ti. The loss function chosen was cross-entropy, and the Adam optimization algorithm was used for model training. A total of 60 training epochs were conducted, with an initial learning rate of 0.001 and a batch size of 64. During training, if the accuracy on the test set did not improve for 10 consecutive epochs, the learning rate was adjusted to 0.0001. Finally, the model with the highest accuracy on the test set was saved.

[0103] Experimental Procedure: In the specific experimental process, electromyographic signal data were sampled with a window size of 200ms and a step size of 50ms. The repetition order of each gesture was numbered 1-6, and the experimental data generated by repetitions numbered 1, 3, 4, and 6 were used as the training set X. train Y train The experimental data generated repeatedly, numbered 2 and 5, were used as the test set X. test Y testGiven the significant individual differences among the subjects, referring to step 2c in Example 9, a method for evaluating the classification performance of the model was adopted in the study, namely, the final accuracy of gesture recognition was obtained by averaging the accuracy of all subjects. The superiority of the lightweight network model was fully demonstrated by comparing the number of parameters and computational cost, as well as the accuracy, of existing studies [1] Wei W, Dai Q, Wong Y, et al. Surface-electromyography-based gesture recognition by multi-view deep learning[J]. IEEE Transactions on Biomedical Engineering, 2019, 66(10): 2964-2973. and [2] Wang Jingwen. Research on surface electromyography gesture recognition based on lightweight neural network[D]. Nanjing University of Posts and Telecommunications, 2022. DOI: 10.27251 / d.cnki.gnjdc.2022.001682.

[0104] Experimental Results and Analysis:

[0105] Table 1 shows a comparison of the present invention with existing electromyography network models (Wei-Net, Wang-Net) in terms of parameter quantity, computational cost, and accuracy.

[0106] Table 1

[0107] <![CDATA[Wei-Net [1] ]]> 0.86 55.4 83.7 <![CDATA[Wang-Net [2] ]]> 1.82 37.06 87.15 This invention 0.13 15.27 87.34

[0108] As shown in Table 1, under the same experimental conditions, the lightweight network model of this invention achieves higher gesture recognition accuracy compared to existing proposed network models, with fewer parameters and less computation. Compared to the Wang-Net network model, the lightweight network model of this invention has only 7% of its parameters and 41% of its computation, while improving recognition accuracy by 0.19%. The technical solution of this invention ensures its advantages in lightweight design and high recognition accuracy in electromyographic gesture recognition, and the experimental results further verify the performance advantages of this invention quantitatively.

[0109] In summary, the lightweight network model and its construction method for surface electromyography (EMG) signal gesture recognition of the present invention solves the problem that current gesture recognition network models cannot effectively capture diverse EMG signal information, resulting in low recognition accuracy. Simultaneously, it addresses the issue that current gesture recognition methods, which improve accuracy by stacking multi-layer network structures and increasing the number of convolutional kernels, require a large number of parameters and computational resources, making them unsuitable for practical applications on resource-constrained devices and mobile devices. The lightweight network model for surface electromyography (EMG) signal gesture recognition of the present invention consists of a backbone network and a classification module connected in series. Between the input and output of the backbone network, there is a two-layer lightweight feature fusion network structure. The first layer of lightweight multi-stream feature fusion module is composed of two interconnected multi-stream feature fusion modules, referred to as the first multi-stream feature fusion module and the second multi-stream feature fusion module, respectively. The second layer has a lightweight multi-scale feature fusion module, which has two convolutional structures adapted to the first layer. One convolutional structure performs feature fusion with the first multi-stream feature fusion module, and the other convolutional structure performs feature fusion with the second multi-stream feature fusion module. The output of the fusion module connected to the second multi-stream feature fusion module is the total output of the lightweight network model backbone network. The present invention discloses a lightweight network model construction method for surface electromyography (EMG) signal gesture recognition. The implementation steps are as follows: acquiring raw EMG signal data; preprocessing the raw EMG signals; constructing a lightweight network model for EMG signal gesture recognition; training the lightweight network model using the preprocessed EMG data to obtain the optimal lightweight network model; and inputting the EMG signal to be recognized into the lightweight network model to obtain the EMG signal gesture type. This invention constructs a lightweight network model for surface EMG signal gesture recognition. A multi-stream feature fusion module is designed based on the EMG acquisition location and the synergistic effect between muscle groups. A lightweight residual attention module is also added to the multi-stream feature fusion module, which adds almost no additional computational overhead, allowing the model to extract features more effectively. By combining the lightweight multi-scale feature fusion module and the multi-stream feature fusion module, the model can capture multi-scale features with reduced parameter count, enhancing feature propagation and improving the accuracy of EMG gesture recognition. Introducing depthwise separable convolution into the multi-stream feature fusion module significantly reduces the number of parameters and computational load of the model. This invention can be used in clinical medicine, industrial robot control, human-machine interface design, rehabilitation medicine, and consumer electronics.

Claims

1. A lightweight network system for gesture recognition based on surface electromyography signals, comprising a backbone network and a classification module connected in series, wherein two multi-stream convolutional networks are connected in series between the input and output of the backbone network, characterized in that, Between the input and output of the lightweight network system backbone, a two-layer lightweight feature fusion network structure is established. The first layer's lightweight multi-stream feature fusion module consists of two interconnected multi-stream feature fusion modules, referred to as the first multi-stream feature fusion module and the second multi-stream feature fusion module. Each multi-stream feature fusion module is connected in series with its own fusion module, forming a unified lightweight multi-stream feature fusion module. The second layer contains a lightweight multi-scale feature fusion module, which has two convolutional structures adapted to the first layer. One convolutional structure performs feature fusion with the first multi-stream feature fusion module, and the other convolutional structure... The second multi-stream feature fusion module performs feature fusion; the output of the fusion module connected to the second multi-stream feature fusion module is the total output of the lightweight network system backbone network; the multi-stream feature fusion module is composed of a depthwise separable convolutional structure and a lightweight residual attention module connected in series, the lightweight residual attention module is composed of CA attention mechanism and SAM attention mechanism connected in series, and residual connections are used between the input and output of the lightweight residual attention module; the two-layer lightweight feature fusion network structure in the lightweight network system backbone network includes three main modules: multi-stream feature fusion module, lightweight residual attention module, and lightweight multi-scale feature fusion module; Based on the electrode positions acquired by EMG, the input data is divided into three parts in each multi-stream feature fusion module. During the multi-stream convolution stage, three branches of a multi-stream convolutional network are used, corresponding to the three data input streams. Each input stream uses an independent CNN network for feature extraction. After each multi-stream convolution, a batch normalized convolutional module and a lightweight residual attention module are added. The lightweight residual attention module is formed by CA and SAM through residual connections. In the backbone network of the lightweight network system, the two multi-stream feature fusion modules and the two branches of the lightweight multi-scale feature fusion module perform feature fusion. The lightweight multi-scale feature fusion module uses multiple convolutional layers with a kernel size of 3*3 to capture the same characteristics of convolutional layers with kernel sizes of 5*5 and 7*7. The ordinary convolution of the multi-stream feature fusion module is replaced with a depthwise separable convolution. When performing gesture classification through the classification module, the flattening layer is replaced with a global average pooling layer. Finally, the vector obtained by passing the output of the fully connected layer through the Dropout layer is input into the Softmax layer to obtain the result with the highest voting probability as the predicted category.

2. A method for constructing a lightweight network system for gesture recognition based on surface electromyography signals, characterized in that, The steps include the following: (1) Acquiring electromyographic signal data: Electromyographic signals of each subject were collected by 12 Dear Hughes active dual differential wireless electrodes at a sampling rate of 2000 Hz. During the acquisition process, 8 electrodes were evenly distributed on the forearm, while the other 2 electrodes were located on the flexor digitorum superficialis and extensor digitorum superficialis respectively, and the other 2 electrodes were distributed on the biceps and triceps muscles at their activity positions. During the data collection process, each subject's gesture was repeated 6 times, each time lasting 5 seconds, followed by a 3-second rest. The collected electromyography (EMG) data was saved as a .mat file as the raw EMG signal data. (2) Preprocessing of the acquired electromyographic signals: The original electromyographic signals were obtained by reading the electromyographic signal file. A digital notch filter and a Butterworth digital filter were used to filter out power frequency interference and obtain the data of the useful frequency band of the electromyographic signals. The electromyographic signals were segmented using an overlapping sliding window and the electromyographic signal data was converted into image data. In the specific experimental process, the repetition order of each gesture was numbered 1-6, and the experimental data generated by the repetitions numbered 1, 3, 4, and 6 were used as the training set. , The experimental data generated repeatedly for numbers 2 and 5 were used as the test set. , The gesture recognition accuracy for each subject is defined as the ratio between the number of correctly predicted test samples and the total number of test samples in the experiment; the overall gesture recognition accuracy is defined as the average gesture classification accuracy of all subjects in the experiment. (3) Constructing a lightweight network system for surface electromyography (EMG) signal gesture recognition: The lightweight network system for EMG signal gesture recognition is referred to as the lightweight network system. Between the input and output of the backbone network of the lightweight network system, there is a two-layer lightweight feature fusion network structure. The first layer of lightweight multi-stream feature fusion module is composed of two interconnected multi-stream feature fusion modules, referred to as the first multi-stream feature fusion module and the second multi-stream feature fusion module, respectively. Each multi-stream feature fusion module is connected in series with its own fusion module, which are connected together to form the whole lightweight multi-stream feature fusion module. The second layer is equipped with a lightweight multi-scale feature fusion module. This lightweight multi-scale feature fusion module is equipped with two convolutional structures adapted to the first layer. One of the convolutional structures is connected to the first multi-stream feature fusion module. The first multi-stream feature fusion module performs feature fusion, and another convolutional structure performs feature fusion with the second multi-stream feature fusion module. The output of the fusion module connected to the second multi-stream feature fusion module is the total output of the lightweight network system backbone. The multi-stream feature fusion module is composed of a depthwise separable convolutional structure and a lightweight residual attention module. The lightweight residual attention module is composed of a coordinate attention mechanism (CA) and a spatial attention mechanism (SAM) connected in series, and residual connections are used between the input and output of the lightweight residual attention module. The constructed lightweight network system backbone contains a two-layer lightweight feature fusion network structure, including three main modules: multi-stream feature fusion module, lightweight residual attention module, and lightweight multi-scale feature fusion module. Based on the electrode positions acquired by EMG, the input data is divided into three parts in each multi-stream feature fusion module. During the multi-stream convolution stage, three branches of a multi-stream convolutional network are used, corresponding to the three data input streams. Each input stream uses an independent CNN network for feature extraction. After each multi-stream convolution, a batch normalized convolutional module and a lightweight residual attention module are added. The lightweight residual attention module is formed by CA and SAM through residual connections. In the backbone of the lightweight network system, the two multi-stream feature fusion modules and the two branches of the lightweight multi-scale feature fusion module perform feature fusion. The lightweight multi-scale feature fusion module uses multiple convolutional layers with a kernel size of 3*3 to capture the same characteristics of convolutional layers with kernel sizes of 5*5 and 7*7. The ordinary convolutions in the multi-stream feature fusion module are replaced with depthwise separable convolutions. When performing gesture classification through the classification module, the flattening layer is replaced with a global average pooling layer. Finally, the vector obtained by passing the output of the fully connected layer through the Dropout layer is input into the Softmax layer to obtain the result with the highest voting probability as the predicted category. (4) Obtaining a lightweight network system with the best recognition performance through training: The lightweight network system is trained using preprocessed electromyography data, and its corresponding parameters are saved; First, the training set is used , The lightweight network system is trained separately, and the adaptive learning rate for each parameter is calculated using the Adam optimizer. , Determine the recognition performance of the lightweight network system, determine the optimal parameters layer by layer, and save the optimal lightweight network system obtained from the final training. (5) Apply the trained lightweight network system to perform electromyographic gesture recognition: The optimal lightweight network system obtained from the training is used to input the electromyographic gesture signal to be recognized into the lightweight network system. The electromyographic gesture signal data is filtered and denoised to be converted into the electromyographic gesture image corresponding to the electromyographic gesture signal. Then, the electromyographic gesture image is input into the lightweight network system to obtain the result with the highest voting probability in the Softmax classifier as the predicted category.

3. The method for constructing a lightweight network system for gesture recognition based on surface electromyography signals according to claim 2, characterized in that, The multi-stream feature fusion module described in step (3) includes: In each multi-stream feature fusion module, the input data is divided into three parts. When the time window length is 200 milliseconds, each electrode collects 400 samples, i.e., T=400. Columns 1 to 8 represent equidistant electrodes evenly distributed around the forearm, while columns 9 and 10 contain signals collected from the flexor and superficial extensor digitorum muscles. Furthermore, columns 11 and 12 contain signals from the main muscle activity points, namely the biceps and triceps brachii. Dividing the input data into three parts according to the electrode distribution can be represented as follows: , , Each input stream focuses on features from different muscle regions; two multi-stream convolutional modules are constructed, each followed by a batch-normalized convolutional module and a lightweight residual attention module; the first multi-stream feature fusion module has a kernel size of 64 and consists of ordinary convolutions; the second multi-stream feature fusion module has a kernel size of 128 and consists of depthwise separable convolutions; assuming the size of the input matrix is... The depth of the input feature matrix is ​​M, and the size of the convolution kernel is... The depth of the output feature matrix is ​​N. A comparison of the number of parameters between depthwise separable convolution and ordinary convolution is provided. ； Comparison of computational complexity: ； In the formula The number of parameters representing a depthwise separable convolution. This represents the number of parameters in a regular convolution. This represents the computational cost of depthwise separable convolution. This indicates the computational cost of a regular convolution.

4. The method for constructing a lightweight network system for gesture recognition based on surface electromyography signals according to claim 2, characterized in that, In step (3), a residual connection is used between the input and output of the lightweight residual attention module. Specifically, the coordinate attention mechanism (CA) and the spatial attention mechanism (SAM) are connected in series, and a residual structure is introduced to connect the input residual of the attention module to the output, thus forming the lightweight residual attention module (RCASM).

5. The method for constructing a lightweight network system for gesture recognition based on surface electromyography signals according to claim 2, characterized in that, The lightweight multi-scale feature fusion module described in step (3) includes: In the lightweight multi-scale feature fusion module, multiple Conv layers with a kernel size of 3*3 are used to efficiently capture the same features of 5*5 and 7*7 kernels. The input first passes through two parallel streams. One stream is the first multi-stream feature fusion module with a convolutional kernel size of 64. The second stream contains two 3*3 Conv layers stacked sequentially with convolutional kernel sizes of 16 and 32, extracting features with a receptive field size of 5. The features from the two streams are fused to ensure that the features have receptive fields of 3 and 5. Then, the output of this layer is fed into the second multi-stream feature fusion module with a convolutional kernel size of 128 and a stride of 3*3 to increase the size of the receptive field. Specifically, the second multi-stream feature fusion module has features with receptive fields of 7 and 5. Finally, the output of the second multi-stream feature fusion module is fused again with the output of another 3*3 Conv layer with a convolutional kernel size of 32 to ensure multi-scale feature extraction.

Images