Surface electromyography signal classification method based on spatial attention pruning capsule network

Abstract

The application discloses a surface electromyogram signal classification method based on a spatial attention pruning capsule network, and comprises the following steps: collecting a surface electromyogram signal, pre-processing the surface electromyogram signal, and obtaining a target electromyogram signal; performing two-dimensionalization on discrete features of the target electromyogram signal based on Cartesian product operation, and obtaining a two-dimensional feature relationship graph; obtaining spatial attention information based on the two-dimensional feature relationship graph; and constructing a spatial attention pruning capsule network model, inputting two-dimensional features into the spatial attention pruning capsule network model for feature classification, and obtaining a classification result. The application improves the recognition accuracy of the model, reduces the training time, meets the short delay requirement of surface electromyogram signal gesture action recognition in a real situation, and can adapt to sample recognition of a large data scale.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of signal classification, and particularly relates to a surface electromyogram signal classification method based on a spatial attention pruning capsule network. BACKGROUND

[0002] When human beings enter the Internet era from the PC era, mobile Internet brings us a large amount of explosive information, and multi-attribute text data such as medical data increase sharply, and the contradiction between information overload and knowledge deficiency is increasingly prominent. Therefore, how to efficiently process massive multi-attribute information has become a key problem in current research. However, in the face of exponentially growing discrete feature attribute data, pure manual annotation and classification is unrealistic, so machine intelligent classification is imminent.

[0003] Large-scale multi-classification data sets are generally composed of multiple discrete feature attributes. These discrete features are usually one-dimensional vectors, but there is a certain correlation between each discrete feature. In practical applications, simple linear classification or regression prediction often cannot distinguish most complex samples. In the face of a large number of nonlinear problems today, if only the quantitative relationship of several discrete features is simply considered, the correlation between different independent variables is easily ignored. In order to cope with nonlinear large-scale data sets such as surface electromyogram signals, in feature engineering, one-order discrete features are often combined with each other (multiplied or Cartesian product) to derive synthetic features to learn highly complex models. Synthetic features to some extent represent the potential mutual relationship of different features in the original feature space, and are most likely to become new features with strong classification ability, such as multiplying or dividing several features to construct new features in the feature space in a multivariate regression model. However, not all synthetic features can have a positive impact on the model, so existing research mainly focuses on searching the feature space to obtain the best synthetic feature, or finding a synthetic feature with high correlation with the target, and finally classifying the data set through machine learning method. However, pre-annotating all discrete features and synthetic features manually is time-consuming and laborious, but if all types of features cannot be evaluated at the same time, the correlation between features is also easily ignored. Therefore, it is imperative to use machines to classify and annotate data sets.

[0004] Therefore, the existing technology mainly focuses on using machine learning and convolutional neural networks to solve classification problems in the recognition of surface electromyogram signals. However, in actual surface electromyogram signal application scenarios, the model needs to consider multiple factors such as the spatiality and correlation of features. SUMMARY

[0005] In order to overcome the defects that the prior art cannot well obtain the spatial correlation between features, the application provides a surface electromyogram signal classification method based on a spatial attention pruning capsule network, which sends the two-dimensionalized discrete features through Cartesian product operation into a spatial attention based capsule network for training, simultaneously, increases the regularization loss to constrain the spatial saliency attention weight to increase the attention weight of the network to the central region, and removes the low-level capsules with low contribution to high-level capsules through dynamic pruning operation to simplify the model and reduce the calculation cost.

[0006] To achieve the above object, the application provides the following scheme: a surface electromyogram signal classification method based on a spatial attention pruning capsule network, comprising:

[0007] Collecting a surface electromyogram signal, pre-processing the surface electromyogram signal to obtain a target electromyogram signal;

[0008] Two-dimensionalizing the discrete features of the target electromyogram signal based on Cartesian product operation to obtain a two-dimensional feature relationship graph; obtaining spatial attention information based on the two-dimensional feature relationship graph;

[0009] Constructing a spatial attention pruning capsule network model, inputting the two-dimensional features into the spatial attention pruning capsule network model for feature classification to obtain a classification result.

[0010] Preferably, the pre-processing of the surface electromyogram signal includes feature extraction of the surface electromyogram signal based on window analysis method to obtain a feature matrix.

[0011] The feature extraction includes time domain feature extraction, frequency domain feature extraction and time-frequency domain feature extraction.

[0012] Preferably, before the two-dimensionalization of the discrete features of the target electromyogram signal based on Cartesian product operation, it further includes Cartesian product operation of the discrete features to convert the data from one-dimensional feature vector to two-dimensional matrix; the two-dimensional matrix includes composite information of the discrete features and the synthetic features.

[0013] Preferably, the process of two-dimensionalizing the discrete features of the target electromyogram signal based on Cartesian product operation includes feature combination of the feature vector through Cartesian product operation to convert the data from one-dimensional feature vector to two-dimensional feature relationship graph.

[0014] Preferably, the process of obtaining the spatial attention information based on the two-dimensional feature relationship graph comprises: performing overlapping max-pooling processing and overlapping average-pooling processing on the two-dimensional feature relationship graph to filter key position information from different channels; determining spatial saliency attention weights by a spatial attention module based on the key position information; filtering the two-dimensional feature relationship graph according to the spatial saliency attention weights to obtain a saliency feature map; and obtaining the spatial attention information based on the saliency feature map.

[0015] Preferably, the process of constructing the spatial attention pruning capsule network model comprises: taking a capsule network as a main network, constructing the spatial attention pruning capsule network model based on a spatial attention module of a pooling idea, a regularization damage, and a dynamic pruning mechanism.

[0016] Preferably, the process of inputting the two-dimensional features into the spatial attention pruning capsule network model for feature classification comprises: inputting the spatial attention information into the spatial attention pruning capsule network model, extracting abstract features from the saliency feature map, finding the correlation of features with labels through dynamic routing, obtaining the mutual relationship of features at different positions in the same channel and the spatial feature relationship between different channels, and classifying features according to the mutual relationship of features at different positions in the same channel and the spatial feature relationship between different channels to obtain a classification result.

[0017] The present application discloses the following technical effects:

[0018] The surface electromyographic signal classification method based on the spatial attention pruning capsule network provided by the present application uses Cartesian product to two-dimensionize discrete one-dimensional electromyographic features, obtains the internal relationship between features on the basis of retaining discrete features, considers that the generated high-dimensional regular data has problems such as information sparseness and excessive parameters, combines a spatial attention mechanism based on a pooling idea on the basis of a capsule network to mine saliency scores of different features, and thereby enhances or weakens the representation, and proposes a spatial attention pruning capsule network by combining a regularization and a dynamic routing pruning mechanism. In this way, on the one hand, the spatial saliency attention weight matrix has good sparseness, and on the other hand, the dynamic routing link with a less pruning contribution degree is pruned and the back propagation of the link is prohibited, which can effectively reduce the memory requirement of model parameters. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0020] Figure 1 A method flowchart for embodiments of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0022] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0023] As shown in the drawings, Figure 1 The present application provides a surface electromyogram signal classification method based on a spatial attention pruning capsule network. The method extracts time domain features, frequency domain features and time-frequency domain features of electromyogram signals, and converts the features extracted from the electromyogram signals into two dimensions to facilitate training in the network. Spatial attention information is obtained by convolution calculation after maximum pooling and average pooling, and then sent to the capsule network for training. The spatial attention is improved through the spatial attention module, and the spatial features are obtained through the capsule network for classification. The attention to the central features is improved by adding a regularization operation and a pruning mechanism, thereby improving the classification accuracy and reducing the time cost.

[0024] The surface electromyogram signal classification method based on the spatial attention pruning capsule network comprises,

[0025] Collecting a surface electromyogram signal, preprocessing the surface electromyogram signal to obtain a target electromyogram signal;

[0026] Based on Cartesian product operation, the discrete features of the target electromyogram signal are two-dimensionalized to obtain a two-dimensional feature relationship graph. Spatial attention information is obtained based on the two-dimensional feature relationship graph.

[0027] A spatial attention pruning capsule network model is constructed, and the two-dimensional features are input into the spatial attention pruning capsule network model for feature classification to obtain a classification result.

[0028] The preprocessing process of the surface electromyogram signal comprises feature extraction of the surface electromyogram signal based on window analysis to obtain a feature matrix.

[0029] The feature extraction comprises time domain feature extraction, frequency domain feature extraction and time-frequency domain feature extraction.

[0030] Before the two-dimension of the discrete features of the target electromyographic signal is performed based on the Cartesian product operation, further comprising: performing Cartesian product operation on the discrete features, and converting the data from one-dimensional feature vector to two-dimensional matrix; the two-dimensional matrix comprises the composite information of the discrete features and the synthetic features.

[0031] The process of two-dimension of the discrete features of the target electromyographic signal based on the Cartesian product operation comprises: performing feature combination on the feature vector by using the Cartesian product operation, and converting the data from one-dimensional feature vector to two-dimensional feature relationship graph.

[0032] The process of obtaining the spatial attention information based on the two-dimensional feature relationship graph comprises: performing overlapping max-pooling processing and overlapping average-pooling processing on the two-dimensional feature relationship graph to filter key position information from different channels; determining spatial saliency attention weight by a spatial attention module based on the key position information; filtering the two-dimensional feature relationship graph according to the spatial saliency attention weight to obtain a saliency feature map; and obtaining the spatial attention information based on the saliency feature map.

[0033] The process of constructing the spatial attention pruning capsule network model comprises: taking a capsule network as a main network, constructing the spatial attention pruning capsule network model based on a spatial attention module of the pooling idea, combining a regularization damage and a dynamic pruning mechanism.

[0034] The process of inputting the two-dimensional features into the spatial attention pruning capsule network model for feature classification comprises: inputting the spatial attention information into the spatial attention pruning capsule network model, extracting abstract features from the saliency feature map, finding the relationship between features and labels through dynamic routing, obtaining the mutual relationship of features at different positions in the same channel and the spatial feature relationship between different channels; and performing feature classification according to the mutual relationship of features at different positions in the same channel and the spatial feature relationship between different channels to obtain a classification result.

[0035] Embodiment one

[0036] As shown in Figure 1 The present application provides a surface electromyographic signal classification method based on spatial attention pruning capsule network, which two-dimensions the features obtained by feature extraction of electromyographic signal, obtains spatial attention information through max-pooling and average-pooling and convolution calculation, and then sends it into the capsule network for training, and improves the attention to the central features by adding regularization and pruning mechanism, improves the classification accuracy and reduces the training and prediction time.

[0037] Further, the classification method specifically comprises the following steps:

[0038] Step 1: The surface electromyography signals measured for all channels are processed using a window analysis method to obtain the surface electromyography signals obtained by the recording electrode; wherein w represents the window length, t represents the increment interval, and τ represents the processing delay of the feature extraction and classification operation; after each interval t, the signal with a time length of w is sequentially extracted as a whole to obtain a feature matrix; the following formula is defined:

[0039]

[0040] wherein α is an operator, including a feature calculation formula, is the electromyography information value measured for all channels in the mth sample of the kth action, that is, the electromyography signal value from the (m-1)th time node to the (m-1)th+w time node, represents the calculation of different feature values of each channel information in the mth sample of the kth action using the feature calculation formula, forming a feature matrix

[0041] Step 2: The extracted discrete features are subjected to Cartesian product operation, and the data is converted from one-dimensional vector to two-dimensional matrix, so that the matrix contains the composite information of discrete features and synthetic features:

[0042] x = [x1, x2,..., x i ,...,x γ ] (2)

[0043] wherein x i is a single discrete feature.

[0044] Since the discrete features themselves contain the characteristics of the data, after obtaining the normalized discrete feature vector x, a 1 vector is added to x to construct a γ+1 dimensional feature vector f, which is used to increase the selection of discrete features by the discrete feature two-dimensionalization method:

[0045] f = [1, x1, x2,..., x γ ] (3)

[0046] Step 3: After obtaining the required feature vectors x and f, Cartesian product operation is adopted to combine the two feature vectors, thereby converting the one-dimensional feature vector into a two-dimensional feature relationship graph. The specific conversion formula is as follows:

[0047] F = D(f, x, α, β) = G(α·(f×x) β ) (4)

[0048]

[0049]

[0050] where a and b are transformation parameters; x is the Cartesian product operator; F is the feature map x, f is the two-dimensional feature map obtained by the Cartesian product operation; G is the nonlinear activation function sigmoid, which is used to limit the initial data between [0, 1].

[0051] Step 4: In order to collect as many important clues of the feature map as possible, the model uses both overlap max pooling (OverlapMaxPool) and overlap mean pooling (OverlapMeanPool) to process the feature map to infer more accurate saliency scores. The specific saliency score calculation formula is as follows:

[0052] P = Q c (F) (7)

[0053]

[0054] where Qc is the ordinary convolution layer; P is the MxMxC-dimensional feature map obtained after preliminary feature detection by the ordinary convolution layer, and C is the channel number; P mn is the channel vector of the feature map P located at the (m, n) position; is the weight transformation vector of the fully connected layer located at the (m, n) position, which is used to fuse feature information from different channels, that is, |umnT|≡C; b mn represents the bias of the fully connected layer located at the (m, n) position; R mn is the information of the fused feature R located at the (m, n) position after nonlinear transformation. After fusing information from different channels, the feature map is simultaneously subjected to overlap max pooling and overlap mean pooling to screen information of key positions:

[0055] A = G(Q 5×5 ([MaxPool(R);AvgPool(R)])) = G(Q 5×5 ([R max ;R mean ])) (9)

[0056] I = P o A (10)

[0057] where Q 5×5 represents a convolution layer with a receptive field of 5x5, which is used to concatenate the context descriptor; represents the spatial saliency attention weight of different positions of the feature map; I is the feature map after feature saliency screening; G is still the activation function sigmoid; o is the Hadamard product, which is used for the multiplication of the same order matrices;

[0058] Step 5: The information filtered by the spatial attention module is sent to the capsule network to further extract abstract features from the saliency feature map. By dynamically routing the features to find their connection with the labels, not only can the mutual relationship between different location features in the same channel be obtained, but also the spatial feature relationship between different channels can be captured, making the deep abstract features more representative. To solve the problem of long iteration time in the dynamic routing of the capsule network, the number of coupling coefficients is reduced, and a non-dynamic routing pruning mechanism is adopted. The specific calculation formula of the dynamic routing with pruning mechanism is as follows:

[0059] U' j|i = W ij U i (11)

[0060]

[0061]

[0062]

[0063]

[0064] b ij = b ij + U' j|i · v j (16)

[0065] where W ij is the connection weight between the i-th low-level capsule and the j-th high-level capsule; U' j|i is the prediction vector of the i-th low-level capsule to the j-th high-level capsule; c ij is the coupling coefficient determined by the dynamic routing; b ij is the logarithmic prior probability of the i-th low-level capsule coupling to the j-th high-level capsule.

[0066] Step 6: In order to maximize the efficiency of distinguishing the importance of the central region synthesis features, and thus accurately identify the sample labels, the attention weight regularization loss is added, i.e.:

[0067]

[0068] where λ A is the weight loss parameter, used to measure the influence of the spatial saliency attention weight loss.

[0069] Example: The surface electromyography signals used in the study were collected by ELONXI electromyography acquisition instrument developed by the team of the University of Portsmouth, UK. The device supports a maximum of 16 bipolar channels, with a sampling resolution of 24 bits and a sampling frequency between 1000 Hz and 2000 Hz. The experimental data set contains a total of 8 subjects' surface electromyography signals measured at 6 different time periods. In each time period, each subject demonstrates 5 gesture actions respectively, and the surface electromyography signals of each gesture action are detected and recorded by 16 bipolar electrodes (channels). In order to exclude the transition state between two gestures, the middle 10 seconds of the surface electromyography signals of each gesture action are marked as steady state signals. The sampling frequency of the surface electromyography signals is set to 1 kHz, so the raw surface electromyography signal data size of each gesture action is 10000x16. The raw surface electromyography signal size of the data set is 8x6x5x10000x16, i.e. 240x10000x16. The number of channels n p = 16, and the number of features n f = 14 are selected. The window length w = 300 and the increment interval t = 50 in the window analysis method are set. The surface electromyography signals measured by the same subject in the morning and afternoon are used in the experiment to simulate the electrode offset situation, where the segmented surface electromyography signal data measured in the morning is used as the training set (i.e. the training set size is 2925x300x16), and the segmented surface electromyography signal data measured in the afternoon is used as the test set (i.e. the test set size is 975x300x16). Table 1 below shows the accuracy of surface electromyography signals in different models for identifying different people, and it can be seen that the accuracy of the spatial attention capsule network proposed in the present application for identifying surface electromyography signals of different people has been improved compared with other models and methods in the table.

[0070] Table 1

[0071]

[0072] Note: FE is feature extraction; TD is two-dimensionalization of discrete features.

[0073] Table 2 shows the training time and recognition time required by different models, and it can be seen that the spatial attention capsule network proposed in the present application has improved the time complexity compared with the network without the pruning mechanism.

[0074] Table 2

[0075]

[0076]

[0077] Note: FE is feature extraction

[0078] From the experimental results, the spatial attention pruning capsule network (TDACAPS-P) based on the two-dimensionalization of discrete feature combined with regularization and dynamic routing pruning mechanism has excellent recognition ability, and also verifies the effectiveness of regularization and dynamic routing pruning mechanism. The introduction of the two mechanisms effectively improves the recognition accuracy of the model, reduces the training time, meets the short delay requirement of surface electromyographic signal gesture recognition in real situation, and can adapt to sample recognition with large data size.

[0079] The application two-dimensionalizes the discrete features of the surface electromyographic signal through Cartesian product operation, so as to be sent into the network for training. In view of the information sparsity and excessive parameters of the two-dimensional feature relationship graph, the capsule network is used as the main network, and the spatial attention module based on the pooling idea is innovatively combined to dynamically estimate the saliency score of different regions of the feature layer, so as to retain information to improve the recognition accuracy of the model. The regularization and pruning mechanism are combined on the spatial attention capsule network to improve the difficulty of the spatial attention module to fall into the unified strengthening representation and the defect of the long iteration time of the dynamic routing of the capsule network. The accuracy of gesture recognition is improved, and the time cost of network training and recognition is reduced.

[0080] The above-described embodiments are only preferred modes of the application and do not limit the scope of the application. Without departing from the design spirit of the application, various modifications and improvements of the technical solutions of the application made by those skilled in the art shall fall within the protection scope of the claims of the application.

Claims

1. A surface electromyography signal classification method based on spatial attention pruned capsule network, characterized by: include: Collecting surface electromyographic signals, preprocessing the surface electromyographic signals, and obtaining target electromyographic signals; Converting the discrete features of the target electromyographic signal into two dimensions based on a Cartesian product operation to obtain a two-dimensional feature relationship graph; Obtaining spatial attention information based on the two-dimensional feature relationship graph; Constructing a spatial attention pruning capsule network model, inputting the spatial attention information into the spatial attention pruning capsule network model for feature classification, and obtaining a classification result; Before converting the discrete features of the target electromyographic signal into two dimensions based on the Cartesian product operation, the method further includes performing a Cartesian product operation on the discrete features to convert the data from a one-dimensional feature vector into a two-dimensional matrix; the two-dimensional matrix includes composite information of the discrete features and the synthetic features; The process of converting the discrete features of the target electromyographic signal into two dimensions based on the Cartesian product operation includes: combining the feature vectors using the Cartesian product operation to convert the data from the one-dimensional feature vector into a two-dimensional feature relationship graph; The process of obtaining the spatial attention information based on the two-dimensional feature relationship graph includes performing overlapping maximum pooling and overlapping average pooling on the two-dimensional feature relationship graph to filter key position information from different channels; and determining a spatial saliency attention weight based on the key position information through a spatial attention module; Filtering the two-dimensional feature relationship graph according to the spatial saliency attention weight to obtain a saliency feature graph; and obtaining spatial attention information based on the saliency feature graph; The process of constructing the spatial attention pruned capsule network model includes taking the capsule network as the main network, a spatial attention module based on the pooling idea, and combining regularization damage and dynamic pruning mechanisms to obtain the spatial attention pruned capsule network model; The process of preprocessing the surface electromyography signal includes extracting features of the surface electromyography signal based on a window analysis method to obtain a feature matrix; The feature extraction includes time domain feature extraction, frequency domain feature extraction and time-frequency domain feature extraction; The process of inputting the two-dimensional features into the spatial attention pruned capsule network model for feature classification includes inputting the spatial attention information into the spatial attention pruned capsule network model, extracting abstract features from the saliency feature map, finding the connection between features and labels through dynamic routing, obtaining the mutual relationship between features at different positions in the same channel and the spatial feature relationship between different channels; and performing feature classification based on the mutual relationship between features at different positions in the same channel and the spatial feature relationship between different channels to obtain a classification result.

Images