A badminton swing action recognition method based on surface electromyography signals

Abstract

A badminton swing action recognition method based on surface electromyogram signal, comprising: dividing badminton swing action; collecting surface electromyogram signal using MYO; obtaining active section by pre-processing electromyogram signal; transforming active window electromyogram signal into wavelet time-frequency graph; sending time-frequency graph into CNN network of fusion space and channel attention mechanism, completing extraction of spatial features, sending spatial features restored into time sequence into ConvLSTM network, completing extraction of time sequence features, sending features into linear network for integration, and finally obtaining badminton swing action recognition result using Softmax function. The application effectively reduces decoding time and improves recognition accuracy.

Description

Technical Field

[0001] This invention belongs to the field of sports and biosignal recognition, and particularly relates to a method for recognizing badminton racket swing movements based on surface electromyography signals. Background Technology

[0002] Surface electromyography (EMG) signals are weak electrical signals collected on the skin surface of the human body using non-invasive electrodes. They are potential differences that accompany muscle movement.

[0003] Different movements produce different electromyographic (EMG) signals. When a person prepares to exercise, the brain sends signals, which are then transmitted through neurons to control the limbs to perform the movement. EMG signals respond more directly to movement intentions and are easier to collect. Therefore, EMG signals can be used to identify different limb movements and help monitor the standard of physical activity.

[0004] Currently, electromyography (EMG) signal recognition is broadly categorized into machine learning and deep learning. Both approaches have limitations. Machine learning requires extracting a large number of features, which is time-consuming and labor-intensive. Deep learning preprocessing does not simplify EMG signals, resulting in excessively long training times to achieve high recognition accuracy, and it also places higher demands on computing power. Summary of the Invention

[0005] To address the technical challenges of complex feature extraction and low recognition accuracy in traditional classification methods, this invention proposes a badminton racket swing recognition method based on surface electromyography (EMG) signals. By utilizing wavelet time-frequency maps of EMG signals and incorporating a CNN-ConvLSTM model with an attention fusion mechanism, the decoding time is effectively reduced while improving recognition accuracy.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A method for badminton racket swing motion recognition based on surface electromyography signals, characterized in that the method includes the following steps:

[0008] Step 1) Divide the badminton shuttlecock receiving action into five types and collect electromyographic signals from eight channels of the upper limb;

[0009] Step 2) Preprocessing of surface electromyography signals;

[0010] Step 3) Use a sliding window to form labeled data pairs, use db5 wavelets to calculate wavelet coefficients for the electromyographic signals in each window, and convert them into wavelet time-frequency maps;

[0011] Step 4) The wavelet time-frequency map and corresponding labels are fed into a convolutional neural network based on the fusion attention mechanism to obtain local spatial features. The local spatial features are then expanded and fed into a ConvLSTM layer to obtain sequence features. The sequence features are then fed into a fully connected layer to complete feature extraction and integration. Finally, a Softmax layer is used to obtain the predicted probabilities of the five badminton swing actions. The badminton swing action corresponding to the highest probability is taken as the recognition result, thus completing the recognition process.

[0012] Furthermore, in step 1), the badminton swing action is specifically divided into lower left backhand swing, middle left backhand swing, upper forehand swing, middle right forehand swing, and lower right forehand swing.

[0013] In step 1), the data acquisition process specifically includes:

[0014] Wear the MYO armband on the biceps brachii area of ​​your right upper arm;

[0015] sEMG signals were acquired at a sampling frequency of 200Hz;

[0016] Each shot is repeated 20 times, with a 3-second interval between each shot.

[0017] In step 2), the data preprocessing includes three steps in sequence: electromyography signal synchronization, wavelet denoising, and active segment extraction.

[0018] Furthermore, the electromyographic signals in the data preprocessing are synchronized and truncated according to the time axis, so that the electromyographic signals of the eight channels are on the same time axis.

[0019] Furthermore, in the data preprocessing, wavelet denoising is performed. The db5 wavelet is used to decompose the synchronized electromyography (EMG) signal to obtain high-frequency and low-frequency components. Thresholding is applied to the high-frequency components, and wavelet reconstruction is performed on each processed component to remove baseline noise from the EMG signal, resulting in a signal in the 20Hz-150Hz frequency band.

[0020] a n-1 =G * a n +H * d n (1)

[0021] Where a is the low-frequency coefficient obtained by passing the electromyographic signal through the low-pass filter G, and d is the high-frequency coefficient obtained by passing the electromyographic signal through the high-pass filter H, G * H is the conjugate matrix of the low-pass filter G. * Let H be the conjugate matrix of the high-pass filter H.

[0022] Furthermore, in the data preprocessing, the active segment extraction first involves fully rectifying and weighting the electromyographic signals from the eight channels, then smoothing the signals using a moving average method, and finally setting a threshold to extract the active segments.

[0023] In step 3), the sliding window is set to 300ms with a step size of 60ms, ensuring that adjacent windows overlap by 80%. Wavelet transform is performed on the electromyography signals within each window to obtain the wavelet coefficients.

[0024]

[0025] Where τ is the translation amount, s is the scale, x(t) is the surface electromyography signal to be transformed, s is changed to adjust the width of the wavelet, and the translation amount τ determines the time. This means that every result obtained will be normalized.

[0026] Finally, the wavelet time-frequency plots of the actual frequency sequence f and the time sequence t are combined.

[0027] In step 4), a lightweight attention module (CBAM) is added to the neural network, which can perform attention in both channel and spatial dimensions. The probability of classifying the wavelet time-frequency graph into the lower left backhand swing state, the middle left backhand swing state, the upper forehand swing state, the middle right forehand swing state, and the lower right forehand swing state is as follows:

[0028]

[0029] Where h(x,y) i ) represents the i eigenvalues ​​obtained by matrix operations on the degraded feature matrix.

[0030] In step 4), the ReLU activation function is used, gradient descent is used to adjust the network parameters, and the cross-entropy loss function is adopted.

[0031]

[0032] Where z is the label value predicted by the model, and y is the actual label value.

[0033] This invention segments badminton swing motions; uses MYO to collect surface electromyography (EMG) signals; preprocesses the EMG signals to obtain active segments; transforms the active window EMG signals into wavelet time-frequency maps; feeds the time-frequency maps into a CNN network that integrates spatial and channel attention mechanisms to extract spatial features; restores the spatial features to a time series and feeds them into a ConvLSTM network to extract time series features; integrates the features into a linear network; and finally uses the Softmax function to obtain the badminton swing motion recognition result.

[0034] The beneficial effects of this invention are mainly reflected in the following aspects: converting traditional electromyographic signals into wavelet time-frequency maps, making full use of the characteristics of electromyographic signals; using a sliding window for real-time acquisition and recognition; employing wavelet denoising for high signal-to-noise ratio; and using a CNN-ConvLSTM neural network with an attention mechanism for recognition, which combines the characteristics of traditional physical feature extraction and neural networks, effectively reducing decoding time and improving recognition accuracy. Attached Figure Description

[0035] Figures 1(a)-1(f) The present invention describes five swing motions and resting states, wherein Figure 1(a) is a left-lower backhand swing, Figure 1(b) is a left-middle backhand swing, Figure 1(c) is a forehand swing, Figure 1(d) is a right-middle forehand swing, Figure 1(e) is a right-lower forehand swing, and Figure 1(f) is a resting state.

[0036] Figure 2(a) shows the MYO arm ring used for data collection in this invention.

[0037] Figure 2(b) shows the position of the MYO armband when collecting data according to the present invention.

[0038] Figure 3 This is the experimental flowchart of the present invention.

[0039] Figure 4 This is an electromyography (EMG) of the lower left backhand swing motion collected by this invention.

[0040] Figure 5 Flowchart of electromyography data preprocessing.

[0041] Figure 6 This is a wavelet amplitude frequency diagram of a certain time window of the present invention.

[0042] Figure 7 This is a diagram of the CNN-ConvLSTM model architecture that incorporates an attention mechanism.

[0043] Figure 8 This is a schematic diagram of the ConvLSTM network of the present invention. Detailed Implementation

[0044] The present invention will now be further described with reference to the accompanying drawings.

[0045] Reference Figures 1(a)-1(f) , Figure 2(a)-Figure 2(b) , Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7 and Figure 8 A badminton racket swing motion recognition method based on surface electromyography signals, the method comprising the following steps:

[0046] Step 1) as Figures 1(a)-1(f) As shown, the badminton receiving motion is divided into five types: left-lower backhand swing (Figure 1(a)), left-middle backhand swing (Figure 1(b)), forehand swing (Figure 1(c)), right-middle forehand swing (Figure 1(d)), and right-lower forehand swing (Figure 1(e)). The MYO device shown in Figure 2(a) is worn in the position shown in Figure 2(b) to capture the bioelectrical changes in the user's arm muscles. Figure 3 As shown in Figure 1(f), upper arm electromyography (EMG) signals were collected for five different racket-swinging motions at a sampling frequency of 200 Hz. Each discrete swing motion was repeated 20 times, with a 3-second rest period as shown in Figure 1(f) after each repetition, resulting in N sampling points. The acquisition device has 8 electrode channels, therefore the data acquisition dimension is N×8. The eight-channel EMG data for the lower left swing motion are shown below. Figure 4 As shown.

[0047] Step 2) According to Figure 5 The preprocessing flow shown processes electromyographic data, which includes three steps in sequence: electromyographic signal synchronization, wavelet denoising, and active segment extraction.

[0048] Electromyography (EMG) signals are synchronized by processing the eight-channel EMG signals acquired according to the time axis, so that the EMG signals of the eight channels are on the same time axis.

[0049] Wavelet denoising was performed using the db5 wavelet to decompose the synchronized electromyography (EMG) signal into high-frequency and low-frequency components. Then, based on the characteristics of the EMG signal and noise at different scales, appropriate rules were selected, and the characteristics of the decomposed wavelet coefficients were used to denoise the EMG signal at each scale. The processed components were then reconstructed using wavelets to obtain the signal in the 20Hz-150Hz frequency band.

[0050] a n-1 =G * a n +H * d n (1)

[0051] Where a is the low-frequency coefficient obtained by passing the electromyographic signal through the low-pass filter G, and d is the high-frequency coefficient obtained by passing the electromyographic signal through the high-pass filter H, G * H is the conjugate matrix of the low-pass filter G. * Let H be the conjugate matrix of the high-pass filter H.

[0052] The electromyographic signal after removing baseline noise was obtained.

[0053] For activity segment extraction, the absolute values ​​of the denoised eight-channel electromyography (EMG) signals are first calculated, and then the eight-channel moving average absolute value (AMA) is calculated:

[0054]

[0055] Where m represents the number of channels in sEMG, and n represents the number of channels before the nth sampling point. i,j This represents the i-th sampling point of the j-th channel, with a window length of 64ms.

[0056] The activity segment is considered to be at its start point when three consecutive amplitude values ​​(AMA) are greater than a set threshold S, and at its end point when they are less than E. To prevent the end point from being detected before the start point is detected, S ≥ E is set. The start and end points are obtained and stored in a one-dimensional matrix. Finally, the activity segment of the denoised eight-channel electromyography signal is extracted based on the start and end points.

[0057] Step 3) Use a sliding window set to 300ms and a sliding step size of 60ms, ensuring 80% overlap between adjacent windows. Use the db5 wavelet to perform wavelet transform on the electromyography signals within each window to obtain the wavelet coefficients.

[0058]

[0059] Where τ is the translation amount, s is the scale, x(t) is the surface electromyography signal to be transformed, s is changed to adjust the width of the wavelet, and the translation amount τ determines the time. This means that every result obtained will be normalized.

[0060] Finally, combining the actual frequency sequence f and the time series t, the following diagram is drawn: Figure 5 The wavelet time-frequency diagram is shown.

[0061] Step 4) According to Figure 7 As shown, a CNN-ConvLSTM model with an attention mechanism is established. The first part of the model is a CNN module containing a ReLU activation function. To prevent overfitting, its output is connected to a BN layer and max pooling is performed, which reduces dimensionality while preserving information.

[0062] The second part of the model is the CBAM attention mechanism module, which sequentially adds features to the channel attention mechanism and the spatial attention mechanism;

[0063] The channel attention mechanism module specifically includes:

[0064] The input feature map F of size H×W×C is processed by global max pooling based on width and global average pooling based on height to obtain two 1×1×C feature maps, where H is the height of feature map F, W is the width of feature map F, and C is the number of channels.

[0065] They are then fed into a two-layer neural network. The first layer has C / r neurons, where r is the reduction rate and ReLU is the activation function. The second layer has C neurons. This two-layer neural network is shared.

[0066] Then, the two features output by the neural network are summed, and then the weights are normalized by sigmoid activation to generate the final channel weights M. c :

[0067]

[0068] Where σ is the sigmoid function, The weights W0 and W1 of the MLP are shared, and the ReLU activation function precedes W0.

[0069] Finally, the channel weights are multiplied by the input feature map F to generate the input features required by the spatial attention mechanism module.

[0070] The spatial attention mechanism module specifically includes:

[0071] The transpose of the feature map F output by the channel attention mechanism module is used as the input feature map of this module;

[0072] First, perform a channel-based global max pooling and global average pooling to obtain a two-dimensional H×W×1 feature map;

[0073] Then, this two-dimensional feature map is stitched together based on the channels, and the channel information is fused through a 7×7 convolution operation to reduce the dimension to 1 channel, i.e. H×W×1;

[0074] The spatial weights are then normalized using the sigmoid activation function to generate the final spatial weights M. s :

[0075]

[0076] Where σ is the sigmoid activation function, f 7×7 The convolution kernel is 7x7 in size.

[0077] Finally, the spatial weights are multiplied by the input feature map of the module to obtain the final generated features.

[0078] The third part of the model is the ConvLSTM module, which expands the obtained features, restores the time series, and feeds it into the ConvLSTM module to extract sequence features. Figure 8 As shown, details of the hidden state update:

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] Where * denotes the convolution operation, and σ denotes the sigmoid activation function. This is the vector of the current element in the time series. This is the hidden state output by the previous node. This represents the hidden state of this node.

[0085] The output features are processed through a linear layer to extract effective information, and then a softmax layer is used to obtain a wavelet time-frequency map. The probabilities of classifying the wavelet time-frequency map into the following states are: lower left backhand swing, middle left backhand swing, top forehand swing, middle right forehand swing, and lower right forehand swing.

[0086]

[0087] Where h(x,y) i ) represents the i eigenvalues ​​obtained by matrix operations on the degraded feature matrix.

[0088] The badminton receiving action with the highest probability is used as the recognition result to train the network model. The Adam optimizer is used to optimize the network parameters through gradient descent, and the cross-entropy loss function is used.

[0089]

[0090] Where z is the label value predicted by the model, and y is the actual label value.

[0091] Finally, the trained network model is used to identify the swing motion.

[0092] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for badminton racket swing motion recognition based on surface electromyography signals, characterized in that, The method includes the following steps: Step 1) Divide the badminton shuttlecock receiving action into five types and collect electromyographic signals from eight channels of the upper limb; Step 2) Surface electromyography signal preprocessing, which includes three steps in sequence: electromyography signal synchronization, wavelet denoising, and active segment extraction; Step 3) Use a sliding window to form labeled data pairs, use db5 wavelets to calculate wavelet coefficients for the electromyographic signals in each window, and convert them into wavelet time-frequency maps; Step 4) The wavelet time-frequency map and corresponding labels are fed into a convolutional neural network based on the fusion attention mechanism to obtain local spatial features. The local spatial features are then expanded and fed into a ConvLSTM layer to obtain sequence features. The sequence features are then fed into a fully connected layer to complete feature extraction and integration. Finally, a Softmax layer is used to obtain the predicted probabilities of the five badminton swing actions. The badminton swing action corresponding to the highest probability is taken as the recognition result, thus completing the recognition process.

2. The badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1, characterized in that: In step 1), the badminton swing action is specifically divided into the lower left backhand swing, the middle left backhand swing, the upper forehand swing, the middle right forehand swing, and the lower right forehand swing.

3. A badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1 or 2, characterized in that: In step 1), the acquisition process specifically includes: wearing the MYO armband on the biceps brachii of the right upper arm; acquiring sEMG signals at a sampling frequency of 200Hz; repeating each discrete hitting action 20 times, with a 3-second interval between each action.

4. A badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1 or 2, characterized in that: In step 2), the synchronized electromyographic signals in the data preprocessing are specifically: based on the time axis, the electromyographic signals of the eight channels are placed on the same time axis.

5. A badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1 or 2, characterized in that: In step 2), the wavelet denoising in data preprocessing specifically involves: using the db5 wavelet to perform wavelet decomposition on the synchronized electromyography (EMG) signal to obtain high-frequency and low-frequency components; performing thresholding on the high-frequency components; and then reconstructing each processed component using wavelets to remove baseline noise from the EMG signal, resulting in a signal in the 20Hz-150Hz frequency band. （1） in, a Electromyography signals are passed through a low-pass filter. The obtained low-frequency coefficients, d Electromyography signals are passed through a high-pass filter. The obtained high-frequency coefficients, For low-pass filter The conjugate matrix, For high-pass filters The conjugate matrix.

6. A badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1 or 2, characterized in that: In step 2), the active segment extraction in data preprocessing is specifically as follows: first, the electromyographic signals of the eight channels are fully rectified and weighted to obtain an average; then, the moving average method is used to smooth the signal; and finally, a threshold is set to extract the active segment.

7. A badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1 or 2, characterized in that: In step 3), the sliding window is set to 300ms with a step size of 60ms, ensuring that adjacent windows overlap by 80%; wavelet transform is performed on the electromyography signals within each window to obtain the wavelet coefficients. （2） in, The translation amount, s It is a scale. To change the surface electromyography signal that needs to be transformed, s Adjust the wavelet width and translation amount. Decide on the timing. This means that every result obtained will be normalized; Finally, combined with the actual frequency sequence f and time series t The wavelet time-frequency plot is obtained.

8. A badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1 or 2, characterized in that: In step 4), a lightweight attention module (CBAM) is added to the neural network, which can perform attention in both channel and spatial dimensions; the probability of classifying the wavelet time-frequency graph into the lower left backhand swing state, the middle left backhand swing state, the upper forehand swing state, the middle right forehand swing state, and the lower right forehand swing state is as follows: （3） in, The feature matrix obtained after degradation is subjected to matrix operations. i Each feature value.

9. A badminton racket swing motion recognition method based on surface electromyography signals as described in claim 1 or 2, characterized in that: In step 4), a ReLU activation function is used, gradient descent is used to adjust the network parameters, and the cross-entropy loss function is adopted. （4） in, z The label values ​​predicted by the model. y These are the actual label values.

Images