A Tai Chi Movement Recognition Method Based on Multi-rate Compressed Convolution

Abstract

This invention discloses a Tai Chi movement recognition method based on multi-rate compressive convolution. First, electromyographic (EMG) data of Tai Chi movements is collected and preprocessed. Then, temporal features of the source signals are extracted to construct a feature matrix. A multi-rate compressive convolution algorithm is used to extract cross-channel compressed feature sequences, which are then fused to obtain a multi-dimensional fused feature matrix. Temporal relationship features are then obtained through a long short-term memory (LSTM) network. Finally, a fully connected layer is used to obtain the movement recognition result, thus completing the Tai Chi movement recognition. This invention achieves multi-scale cross-channel feature extraction of EMG signals through multi-rate dilated convolution, overcoming the limitation of ordinary convolution which can only extract spatial relationships between adjacent channels. It can compress the information contained in EMG signal sequences of arbitrary sizes into a one-dimensional compressed feature sequence to extract highly fused features, reducing data redundancy, the number of parameters, and model computation, thereby improving the accuracy of Tai Chi movement recognition.

Description

Technical Field

[0001] This invention belongs to the field of action recognition technology, specifically relating to a method for recognizing Tai Chi movements based on multi-rate compressed convolution. Background Technology

[0002] Tai Chi is a traditional Chinese martial art that can improve physical health and cultivate character. During interactive Tai Chi rehabilitation training, issues such as improper form and poor coordination may arise, requiring guidance. With the application of deep learning and machine learning, surface electromyography (EMG) signals are frequently used for pattern recognition of limb movements. The paper “Learning Effective Spatial-Temporal Features for sEMG Armband-Based Gesture Recognition. IEEE INTERNET OF THINGS JOURNAL. 2020, 7(8): 6979-6992” uses MEMD to decompose the signal into multiple levels, which together with the original signal form a three-dimensional data matrix, and extracts and classifies the features through CRNN; the paper “sEMG-Based Gesture Recognition Using Deep Learning From Noisy Labels. IEEE Biomed. Health Informatics 26(9): 4462-4473(2022)” uses the multi-resolution spectrum of multi-layer wavelet decomposition to construct a rich fusion feature map, and uses a CNN model to explain the high redundancy structure in the formed feature image, thereby achieving the purpose of hand movement recognition; the paper “Surface Electromyography Image Driven Torque Estimation of Multi-DoF Wrist Movements. IEEE Transactions on Industrial Electronics, 2021, pp(99): 1-1. "Hand action recognition is achieved by acquiring HD-EMG signals and extracting the relative global measurement of the potential physiological process of instantaneous muscle activity through the evolutionary model of CNN; The literature "Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electroromyographic signals. JOURNAL OF NEURAL ENGINEERING. 2021, 18(2)" constructs a three-dimensional data matrix by overlapping windowing and then uses 3D-CNN to extract spatial features to achieve action recognition.

[0003] However, Tai Chi practice requires the coordination of all muscles in the body. This coordination is reflected in the relationship between channels in electromyography (EMG). Existing machine learning methods and methods combining convolution and recurrent networks usually ignore channel relationships or only consider the spatial relationship between adjacent channels or under a fixed expansion rate, ignoring the problem of multiple coordination modes of upper and lower limb muscles, which affects the accuracy of recognition results. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a Tai Chi movement recognition method based on multi-rate compressed convolution. This method takes into account the relationships between different channels of electromyographic signals and the inherent characteristics of the signals themselves, thereby reducing data dimensionality, minimizing data redundancy, and improving the accuracy of Tai Chi movement recognition.

[0005] The technical solution of this invention is: a method for recognizing Tai Chi movements based on multi-rate compressed convolution, the specific steps of which are as follows:

[0006] S1. Collect electromyographic data of Tai Chi movements and perform data preprocessing;

[0007] S2. Extract the time-domain features of the source signal and construct the feature matrix;

[0008] S3. Extract cross-channel compressed feature sequences using a multi-rate compression convolution algorithm;

[0009] S4. Based on the source data obtained in step S3 and the compressed feature sequences of four time-domain features, a multi-dimensional fusion feature matrix is ​​obtained by fusing them. Then, the temporal relationship features are obtained through a long short-term memory network. Finally, the action recognition result is obtained through a fully connected layer, thus completing the Tai Chi action recognition.

[0010] Furthermore, step S1 is specifically as follows:

[0011] The surface electromyography device, namely electrode patches, collects electromyographic signals from the upper limbs, lower limbs, and trunk of the instructor when performing various movements of Tai Chi.

[0012] Each electrode patch is fixed at the corresponding position of the upper limb muscles, lower limb muscles, and trunk muscles, for a total of C electrode patches;

[0013] After confirming that the device is securely fitted and connected to the data acquisition system, the instructor begins practicing each movement; each piece of data collected has C channels.

[0014] The collected data is then preprocessed, as follows:

[0015] First, perform a low-pass filter:

[0016] A fourth-order Butterworth filter is used to perform low-pass filtering on the signal. The empirical value of the cutoff frequency is 5Hz, which should be adjusted appropriately for different devices and specific application scenarios.

[0017] Then perform notch rectification:

[0018] The signal is filtered by a notch filter. The notch frequency is set according to the electronic noise frequency of the acquisition device. The default setting is 50Hz and multiples thereof. The frequency is adjusted appropriately for different devices and specific application scenarios, and the notch nodes are smoothed.

[0019] The preprocessed electromyographic data is referred to as the source signal, denoted as S, with dimensions C×T. The data sequence of the c-th channel of S is represented as Sc. c c∈(1,C), where the length of each channel sequence is T, and C represents the total number of channels in the data sequence.

[0020] Furthermore, step S2 is specifically as follows:

[0021] For each S c ,c∈(1,C), use a window of length L to generate the feature sequence.

[0022] S21. Fill the source signal of length T with zero values ​​of length (L / 2)-1 before it and fill the source signal of length L / 2 after it to form an extended signal sequence of length T+L-1.

[0023] S22. Start windowing from the beginning of the expanded sequence, with a length of L;

[0024] S23. Calculate the characteristic values ​​of the signal within the window, expressed as: c∈(1,C);

[0025] Where 'c' represents the channel number and 'type' represents the feature type, including four features: absolute mean (MAV), root mean square (RMS), waveform length (WL), and standard deviation (SD).

[0026] The absolute mean (MAV) of the signal within the window of length L in channel c reflects the local intensity information of Tai Chi movements. The expression is calculated as follows:

[0027]

[0028] Among them, S c (t) represents the signal value of the t-th sampling point in the data sequence of the source signal channel c, and L represents the length of the window, that is, the number of sampling points in the sequence corresponding to the window;

[0029] The root mean square (RMS) of the signal within the window of length L in channel c reflects the energy information contained in the electromyographic signal during Tai Chi movements. The expression is calculated as follows:

[0030]

[0031] The waveform length WL of the signal within the window of length L in channel c represents the cumulative length of the electromyographic signal waveform, reflecting the fluctuation and duration of Tai Chi movements. The expression is calculated as follows:

[0032]

[0033] The standard deviation SD of the signal within the window of length L in channel c represents the dynamic component of the signal energy, reflecting the energy information contained in Tai Chi movements. The expression is calculated as follows:

[0034]

[0035] in, This represents the average value of the electromyographic signal within the window.

[0036] S24. Move the window backward with a step size of 1. Repeat steps S23 and S24 until the window contains all the values ​​of the sequence, obtaining S. c Feature sequences

[0037] Where type∈{MAV, RMS, WL, SD}.

[0038] S25. Analyze the feature sequences of each channel. c∈(1,C) are combined to form a characteristic matrix S type The dimension is C×T.

[0039] In summary, each source signal S yields four characteristic matrices, which are represented by the absolute mean matrix S0. MAV Root mean square matrix S RMS Waveform length matrix S WL Standard deviation matrix S SD .

[0040] Furthermore, step S3 is specifically as follows:

[0041] The multi-rate compressed convolution (CConv) algorithm uses two convolutional layers to process the source signal S and the temporal feature matrix S extracted in step S2, respectively. MAV S RMS S WL S SD Compressed convolution is performed to obtain a one-dimensional vector;

[0042] Specifically, S undergoes multi-rate compressed convolution, as follows:

[0043] (1) Expand the source signal S with dimension C×T;

[0044] Given the expansion ratio d of the dilated convolution, calculate the kernel size k (rounded up) using the formula C = (k-1)d + k, where k ≥ d. Then, use the values ​​of d and k to calculate the number of rows C of the expanded matrix. ′ , denoted as C ′ = (k-1)d+k.

[0045] Fill the rows with the signal sequence values ​​in order to achieve C. ′ The rows are filled by filling the first row with the values ​​of the first row of the original sequence, the second row with the values ​​of the second row of the original sequence, and so on; resulting in a size of C. ′ An extended matrix of ×T;

[0046] By obtaining synergistic information on various muscle (group) combinations through different expansion rates, a set of d values ​​(d1, d2, ..., d) is obtained. Z A set of k values ​​(k1, k2, ..., k Z ), and a set of C ′ (C ′ 1,C ′ 2,…,C ′ z ).

[0047] Where Z represents the number of different expansion rates.

[0048] (2) Extract compressed feature sequences through compressed convolution;

[0049] Compressed convolutions consist of two layers of convolutional kernels. The first layer is a dilated convolution with a kernel size of k. z (z∈(1,Z)), with a spread rate of d z The stride is 1, and the second layer is a regular convolution with a expansion rate of 1 and a kernel size of k. z The step size is 1;

[0050] The single-step convolution formula for each layer is: ∑ i ∑ j k ij s i ′ j k ij s represents the value in the i-th row and j-th column of the convolution kernel. i ′ j This represents the value in the i-th row and j-th column within the coverage area of ​​the convolution kernel in the expanded matrix.

[0051] The formula for two-layer convolution is as follows:

[0052]

[0053] Where T represents the transpose of the matrix, i represents the i-th convolutional layer (i = 1, 2), S′ represents the padded data matrix, W represents the weight parameter matrix, b represents the offset parameter matrix, and O represents the final output matrix. The weight values ​​of the two weight matrices can be different; f(·) represents the activation function, using the Leaky ReLU function, whose formula is as follows:

[0054]

[0055] Here, 'a' represents a hyperparameter.

[0056] By compressing the convolution, the expansion rate d is obtained. z Compressed feature sequence under (z∈(1,Z)) The size is 1×M z .

[0057] Among them, M z =T-(k z -1)d z The feature sequence after compression convolution contains compressed features of the electromyographic signal (or temporal feature) sequence of Tai Chi.

[0058] Different compression feature sequences can be obtained under different expansion rates, that is, for d1, d2, ..., d Z and their corresponding k1,k2,…,k Z We can obtain lengths M1, M2, ..., M Z The compressed feature sequence, take the max z M z Let M be the length of the longest compressed feature sequence.

[0059] Using M as a reference, all compressed feature sequences with insufficient length are padded with zeros to obtain multiple compressed feature sequences of the same length. The compressed feature sequences obtained from all the compressive convolutions at different expansion rates are then summed to obtain the final compressed feature sequence of the source signal S, denoted as […]. The dimensions are 1×M.

[0060] In summary, similarly, we can obtain the following for the time-domain characteristic matrix S. MAV S RMS S WL S SD Perform multi-rate compressed convolution to obtain compressed feature sequences.

[0061] Furthermore, step S4 is specifically as follows:

[0062] The result obtained in step S3 and The integration yields a multi-dimensional fusion feature matrix. Its dimensions are 5×M, The LSTM model is input to extract temporal relationship features. A fully connected layer and softmax are added after the LSTM model to obtain the final prediction value of the model, thus obtaining the action recognition result and completing the Tai Chi action recognition.

[0063] The beneficial effects of this invention are as follows: The method of this invention first collects electromyographic (EMG) data of Tai Chi movements and performs data preprocessing. Then, it extracts the temporal features of the source signals, constructs a feature matrix, extracts cross-channel compressed feature sequences through multi-rate compressive convolution (CConv) algorithm, fuses them to obtain a multi-dimensional fused feature matrix, obtains temporal relationship features through a long short-term memory network, and finally obtains the action recognition result through a fully connected layer, thus completing the Tai Chi movement recognition. This invention's method achieves multi-scale cross-channel feature extraction of EMG signals through multi-rate dilated convolution, breaking the limitation of ordinary convolution which can only extract spatial relationships between adjacent channels. It can compress the information contained in EMG signal sequences of arbitrary size into a single-unit feature block to extract its highly fused features, reducing data redundancy, the number of parameters, and model computation, thereby improving the accuracy of Tai Chi movement recognition. Attached Figure Description

[0064] Figure 1 This is a flowchart of a Tai Chi movement recognition method based on multi-rate compressed convolution according to the present invention.

[0065] Figure 2 This is a flowchart of the compressed convolution algorithm in an embodiment of the present invention. Detailed Implementation

[0066] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0067] like Figure 1 The flowchart of a Tai Chi movement recognition method based on multi-rate compressed convolution of the present invention is shown below. The specific steps are as follows:

[0068] S1. Collect electromyographic data of Tai Chi movements and perform data preprocessing;

[0069] S2. Extract the time-domain features of the source signal and construct the feature matrix;

[0070] S3. Extract cross-channel compressed feature sequences using the multi-rate compressed convolution (CConv) algorithm;

[0071] S4. Based on the source data obtained in step S3 and the compressed feature sequences of four time-domain features, a multi-dimensional fusion feature matrix is ​​obtained by fusing them. Then, the temporal relationship features are obtained through a long short-term memory network. Finally, the action recognition result is obtained through a fully connected layer, thus completing the Tai Chi action recognition.

[0072] In this embodiment, step S1 is specifically as follows:

[0073] The surface electromyography device, namely electrode patches, collects electromyographic signals from the upper limbs, lower limbs, and trunk of the instructor when performing various movements of Tai Chi.

[0074] Each electrode patch is fixed to the corresponding position of the upper limb muscles such as biceps brachii, triceps brachii, brachialis, brachioradialis, extensor carpi brachii longus, extensor carpi brachii brevis, flexor carpi radialis, palmaris longus, pronator teres, and flexor carpi ulnaris; the lower limb muscles such as tensor fasciae latae, gluteus maximus, rectus femoris, vastus lateralis, vastus medialis, biceps femoris, tibialis anterior, lateral gastrocnemius, and medial gastrocnemius; and the trunk muscles such as trapezius and erector dorsalis. A total of C electrode patches are used.

[0075] After confirming that the device is securely fitted and connected to the data acquisition system, the instructor begins practicing each movement; each piece of data collected has C channels.

[0076] The collected data is then preprocessed, as follows:

[0077] First, perform a low-pass filter:

[0078] A fourth-order Butterworth filter is used to perform low-pass filtering on the signal. The empirical value of the cutoff frequency is 5Hz, which should be adjusted appropriately for different devices and specific application scenarios.

[0079] Then perform notch rectification:

[0080] The signal is filtered by a notch filter. The notch frequency is set according to the electronic noise frequency of the acquisition device. The default setting is 50Hz and multiples thereof. The frequency is adjusted appropriately for different devices and specific application scenarios, and the notch nodes are smoothed.

[0081] The preprocessed electromyographic data is referred to as the source signal, denoted as S, with dimensions C×T. The data sequence of the c-th channel of S is represented as Sc. c c∈(1,C), where the length of each channel sequence is T, and C represents the total number of channels in the data sequence.

[0082] In this embodiment, step S2 is specifically as follows:

[0083] Temporal feature extraction is performed on each channel sequence separately to obtain a one-dimensional feature sequence. These one-dimensional feature sequences are then integrated together according to channel arrangement to form a two-dimensional feature matrix; for each S... c For c∈(1,C), a feature sequence is generated using a window of length L (generally selected to be the number of data points within 200ms, ensuring effective features while reducing redundancy), as follows:

[0084] S21. To ensure the regularity of the data size after feature extraction, the source signal of length T is padded with zero values ​​of length (L / 2)-1 and padded with zero values ​​of length L / 2, so that feature extraction is not disturbed and the feature sequence is consistent with the length of the source signal sequence, forming an extended signal sequence of length T+L-1.

[0085] S22. Start windowing from the beginning of the expanded sequence, with a length of L;

[0086] S23. Calculate the characteristic values ​​of the signal within the window, expressed as: c∈(1,C);

[0087] Where 'c' represents the channel number and 'type' represents the feature type, including four features: absolute mean (MAV), root mean square (RMS), waveform length (WL), and standard deviation (SD).

[0088] The absolute mean (MAV) of the signal within a window of length L in channel c reflects the local intensity information of Tai Chi movements. It is calculated by averaging the absolute values ​​of each signal, as shown in the following expression:

[0089]

[0090] Among them, S c (t) represents the signal value of the t-th sampling point in the data sequence of the source signal channel c, and L represents the length of the window, that is, the number of sampling points in the sequence corresponding to the window;

[0091] The root mean square (RMS) of the signal within the window of length L in channel c reflects the energy information contained in the electromyographic signals during Tai Chi movements. This is achieved by calculating the square root of the mean square of each signal, as expressed below:

[0092]

[0093] The waveform length WL of the signal within the window of length L in channel c represents the cumulative length of the electromyographic signal waveform, reflecting the fluctuation and duration of Tai Chi movements. It is the sum of the cumulative relative differences of the signals, and the expression is calculated as follows:

[0094]

[0095] The standard deviation SD of the signal within the window of length L in channel c represents the dynamic component of the signal energy, reflecting the energy information contained in Tai Chi movements. It is the square root of the average distance between each signal value and the arithmetic mean of the signals within that segment. The expression is calculated as follows:

[0096]

[0097] in, This represents the average value of the electromyographic signal within the window.

[0098] S24. Move the window backward with a step size of 1. Repeat steps S23 and S24 until the window contains all the values ​​of the sequence, obtaining s. c Feature sequences

[0099] Where type∈{MAV, RMS, WL, SD}.

[0100] S25. Analyze the feature sequences of each channel. c∈(1,C) are combined to form a characteristic matrix S type The dimension is C×T.

[0101] In summary, each source signal S yields four characteristic matrices, which are represented by the absolute mean matrix S0. MAV Root mean square matrix S RMS Waveform length matrix S WL Standard deviation matrix S SD .

[0102] like Figure 2 As shown, in this embodiment, step S3 is specifically as follows:

[0103] The multi-rate compressed convolution (CConv) algorithm uses two convolutional layers to process the source signal S and the temporal feature matrix S extracted in step S2, respectively. MAV S RMS S WL S SD Compressed convolutions yield one-dimensional vectors. These two convolutional layers have the same kernel size, different expansion rates, and can have different parameters.

[0104] Specifically, S undergoes multi-rate compressed convolution, as follows:

[0105] (1) Expand the source signal S with dimension C×T;

[0106] Given the expansion ratio d of the dilated convolution, calculate the kernel size k (rounded up) using the formula C = (k-1)d + k, where k ≥ d. Then, use the values ​​of d and k to calculate the number of rows C of the expanded matrix. ′ , denoted as C ′ = (k-1)d+k.

[0107] Fill the rows with the signal sequence values ​​in order to achieve C. ′ The rows are filled by filling the first row with the values ​​of the first row of the original sequence, the second row with the values ​​of the second row of the original sequence, and so on; resulting in a size of C. ′ An extended matrix of ×T;

[0108] The source signal S is expanded so that all data can be covered by the convolution kernel, thus avoiding the loss of Tai Chi movement information.

[0109] By obtaining synergistic information on various muscle (group) combinations through different expansion rates, a set of d values ​​(d1, d2, ..., d) is obtained. Z A set of k values ​​(k1, k2, ..., k Z ), and a set of C ′ (C ′ 1,C ′ 2,…,C ′ z ).

[0110] Where Z represents the number of different expansion rates.

[0111] (2) Extract compressed feature sequences through compressed convolution;

[0112] Compressed convolutions consist of two layers of convolutional kernels. The first layer is a dilated convolution with a kernel size of k. z (z∈(1,Z)), with a spread rate of d z The stride is 1, and the second layer is a regular convolution with a expansion rate of 1 and a kernel size of k. z The step size is 1;

[0113] The single-step convolution formula for each layer is: ∑ i ∑ j k ij s ′ ′ j k ij s represents the value in the i-th row and j-th column of the convolution kernel. i ′ j This represents the value in the i-th row and j-th column within the coverage area of ​​the convolution kernel in the expanded matrix.

[0114] The formula for two-layer convolution is as follows:

[0115]

[0116] Where T represents the transpose of the matrix, i represents the i-th convolutional layer (i = 1, 2), S′ represents the padded data matrix, W represents the weight parameter matrix, b represents the offset parameter matrix, and O represents the final output matrix. The weight values ​​of the two weight matrices can be different; f(·) represents the activation function, using the Leaky ReLU function, whose formula is as follows:

[0117]

[0118] Here, 'a' represents a hyperparameter that determines the slope of the activation region of the activation function, and thus affects the backpropagation of the deep model.

[0119] By compressing the convolution, the expansion rate d is obtained. z Compressed feature sequence under (z∈(1,Z)) The size is 1×M z .

[0120] Among them, M z =T-(k z -1)d z The feature sequence after compression convolution contains compressed features of the electromyographic signal (or temporal feature) sequence of Tai Chi.

[0121] Different compression feature sequences can be obtained under different expansion rates, that is, for d1, d2, ..., d z and their corresponding k1,k2,…,k Z We can obtain lengths M1, M2, ..., M Z The compressed feature sequence, take the max z M z Let M be the length of the longest compressed feature sequence.

[0122] Using M as a reference, all compressed feature sequences with insufficient length are padded with zeros to obtain multiple compressed feature sequences of the same length. The compressed feature sequences obtained from all the compressive convolutions at different expansion rates are then summed to obtain the final compressed feature sequence of the source signal S, denoted as […]. The dimensions are 1×M.

[0123] In summary, similarly, we can obtain the following for the time-domain characteristic matrix S. MAV S RMS S WL S SD Perform multi-rate compressed convolution to obtain compressed feature sequences.

[0124] Using multi-rate dilated convolutions can extract relationships between channels of different spans, rather than being limited to adjacent channels or fixed channels. Using two convolutional layers can fully extract spatial relationship features while reducing data redundancy.

[0125] In this embodiment, step S4 is specifically as follows:

[0126] The result obtained in step S3 and The integration yields a multi-dimensional fusion feature matrix. Its dimensions are 5×M, The LSTM (Long Short Term Memory) model is input to extract temporal relationship features. A fully connected layer and softmax are added after the LSTM model to obtain the final prediction value of the model, thus obtaining the action recognition result and completing the Tai Chi action recognition.

[0127] In summary, the method of this invention extracts multiple temporal features from the source signal, extracts cross-channel features from the temporal feature sequence and the source data sequence respectively through multi-rate compression convolution, and obtains different compressed feature sequences. The compressed feature sequences are integrated to further extract temporal features and realize Tai Chi movement recognition. Multi-rate compression convolution takes into account both the synergistic effect of cross-channel electromyography signals and the temporal features of the signal itself, that is, it takes into account the temporal and spatial features of the signal, reduces data redundancy, and improves the accuracy of Tai Chi movement recognition.

[0128] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A method for recognizing Tai Chi movements based on multi-rate compressed convolution, the specific steps of which are as follows: S1. Collect electromyographic data of Tai Chi movements and perform data preprocessing; S2. Extract the time-domain features of the source signal and construct the feature matrix; S3. Extract cross-channel compressed feature sequences using a multi-rate compression convolution algorithm; Step S3 is as follows: The multi-rate compressed convolution (CConv) algorithm uses two convolutional layers to process the source signal S and the temporal feature matrix S extracted in step S2, respectively. MAV S RMS S WL S SD Compressed convolution is performed to obtain a one-dimensional vector; in, Perform multi-rate compression convolution on S, as follows: (1) Expand the source signal S with dimension C×T; Each channel sequence has a length of T, and C represents the total number of channels in the data sequence. Given the expansion rate d of the dilated convolution, the size k of the corresponding convolution kernel is obtained according to the formula C = (k-1)d + k. The result is rounded up, and k ≥ d must be satisfied. Then, the number of rows C′ of the expanded matrix is ​​obtained by inverse calculation based on the values ​​of d and k, denoted as C′ = (k-1)d + k. Fill the rows with each row of the signal sequence values ​​in sequence until there are C′ rows. That is, the first row filled is the value of the first row of the original sequence, the second row filled is the value of the second row of the original sequence, and so on; thus, an expanded matrix of size C′×T is obtained. By obtaining synergistic information of various muscle combinations through different expansion rates, a set of d values ​​(d1, d2, ..., d) is obtained. Z A set of k values ​​(k1, k2, ..., k Z ), and a set of C′(C′1,C′2,…,C′) Z ); Where Z represents the number of different expansion rates; (2) Extract compressed feature sequences through compressed convolution; Compressed convolutions consist of two layers of convolutional kernels. The first layer is a dilated convolution with a kernel size of k. z z∈(1,Z) has an expansion rate of d z The stride is 1, and the second layer is a regular convolution with a expansion rate of 1 and a kernel size of k. z The step size is 1; The single-step convolution formula for each layer is: ∑ i ∑ j k ij s′ ij k ij Let s' represent the value in the i-th row and j-th column of the convolution kernel. ij This represents the value in the i-th row and j-th column within the coverage area of ​​the convolution kernel in the expanded matrix; The formula for two-layer convolution is as follows: Where T represents the transpose of the matrix, i represents the i-th convolutional layer, i = 1, 2, S′ represents the padded data matrix, W represents the weight parameter matrix, b represents the offset parameter matrix, and O represents the final output matrix. The weight values ​​of the two weight matrices can be different; f(·) represents the activation function, using the Leaky ReLU function, whose formula is as follows: Where 'a' represents a hyperparameter; By compressing the convolution, the expansion rate d is obtained. z Compressed feature sequences in z∈(1,Z) The size is 1×M z ; Among them, M z =T-(k z -1)d z The feature sequence after compressed convolution contains compressed features of Tai Chi electromyography signals or time-domain feature sequences. Different expansion rates result in different compressed feature sequences, i.e., for d1, d2, ..., d Z and their corresponding k1,k2,…,k Z We can obtain lengths M1, M2, ..., M Z The compressed feature sequence, take Let M be the length of the longest compressed feature sequence; Using M as a reference, all compressed feature sequences with insufficient length are padded with zeros to obtain multiple compressed feature sequences of the same length. The compressed feature sequences obtained from compressed convolution at all expansion rates are then summed to obtain the final compressed feature sequence of the source signal S, denoted as […]. The dimensions are 1×M; In summary, similarly, we can obtain the following for the time-domain characteristic matrix S. MAV S RMS S WL S SD Perform multi-rate compressed convolution to obtain compressed feature sequences. S4. Based on the source data obtained in step S3 and the compressed feature sequences of four time-domain features, a multi-dimensional fusion feature matrix is ​​obtained by fusing them. Then, the temporal relationship features are obtained through a long short-term memory network. Finally, the action recognition result is obtained through a fully connected layer, thus completing the Tai Chi action recognition.

2. The Tai Chi Chuan movement recognition method based on multi-rate compressed convolution according to claim 1, characterized in that, The specific steps of S1 are as follows: The surface electromyography device, namely electrode patches, collects electromyographic signals from the upper limbs, lower limbs, and trunk of the instructor when performing various movements of Tai Chi. Each electrode patch is fixed at the corresponding position of the upper limb muscles, lower limb muscles, and trunk muscles, for a total of C electrode patches; After confirming that the device is securely fitted and connected to the data acquisition system, the instructor begins practicing each movement; each data point collected has C channels. The collected data is then preprocessed, as follows: First, perform a low-pass filter: A fourth-order Butterworth filter is used to perform low-pass filtering on the signal. The empirical value of the cutoff frequency is 5Hz, which should be adjusted appropriately for different devices and specific application scenarios. Then perform notch rectification: The signal is filtered by a notch filter. The notch frequency is set according to the electronic noise frequency of the acquisition device. The default setting is 50Hz and multiples thereof. The frequency is adjusted appropriately for different devices and specific application scenarios, and the notch nodes are smoothed. The preprocessed electromyographic data is referred to as the source signal, denoted as S, with dimensions C×T. The data sequence of the c-th channel of S is represented as Sc. c c∈(1,C), where the length of each channel sequence is T, and C represents the total number of channels in the data sequence.

3. The Tai Chi Chuan movement recognition method based on multi-rate compressed convolution according to claim 1, characterized in that, Step S2 is as follows: For each S c ,c∈(1,C), use a window of length L to generate the feature sequence; S21. Fill the source signal of length T with zero values ​​of length (L / 2)-1 before it and fill the source signal of length L / 2 after it to form an extended signal sequence of length T+L-1. S22. Start windowing from the beginning of the expanded sequence, with a length of L; S23. Calculate the characteristic values ​​of the signal within the window, expressed as: Where 'c' represents the channel number and 'type' represents the feature type, including four features: absolute mean (MAV), root mean square (RMS), waveform length (WL), and standard deviation (SD). The absolute mean (MAV) of the signal within the window of length L in channel c reflects the local intensity information of Tai Chi movements. The expression is calculated as follows: Among them, S c (t) represents the signal value of the t-th sampling point in the data sequence of the source signal channel c, and L represents the length of the window, that is, the number of sampling points in the sequence corresponding to the window; The root mean square (RMS) of the signal within the window of length L in channel c reflects the energy information contained in the electromyographic signal during Tai Chi movements. The expression is calculated as follows: The waveform length WL of the signal within the window of length L in channel c represents the cumulative length of the electromyographic signal waveform, reflecting the fluctuation and duration of Tai Chi movements. The expression is calculated as follows: The standard deviation SD of the signal within the window of length L in channel c represents the dynamic component of the signal energy, reflecting the energy information contained in Tai Chi movements. The expression is calculated as follows: in, This represents the average value of the electromyographic signal within the window; S24. Move the window backward with a step size of 1. Repeat steps S23 and S24 until the window contains all the values ​​of the sequence, obtaining S. c Feature sequences Where, type∈{MAV, RMS, WL, SD}; S25. Analyze the feature sequences of each channel. Combined into a characteristic matrix S type The dimension is C×T; In summary, each source signal S yields four characteristic matrices, which are represented by the absolute mean matrix S0. MAV Root mean square matrix S RMS Waveform length matrix S WL Standard deviation matrix S SD .

4. The Tai Chi Chuan movement recognition method based on multi-rate compressed convolution according to claim 1, characterized in that, Step S4 is as follows: The result obtained in step S3 and The integration yields a multi-dimensional fusion feature matrix. Its dimensions are 5×M, The LSTM model is input to extract temporal relationship features. A fully connected layer and softmax are added after the LSTM model to obtain the final prediction value of the model, thus obtaining the action recognition result and completing the Tai Chi action recognition.

Images