Upper limb surface electromyogram signal analysis method fusing depth and width learning

Abstract

The invention discloses an upper limb surface electromyogram signal analysis method fusing depth and width learning, and relates to the technical field of upper limb surface electromyogram signal analysis. Comprising the following steps: collecting multi-channel electromyographic signals to establish a complete sEMG data set, and designing convolutional neural network branches with the same number and residual structures according to the collected multi-channel sEMG signals; the residual convolutional neural network structure output features of each channel are input into the respective width learning sub-model, the weight is calculated through pseudo-inverse regression, the contribution degree index is obtained through absolute value normalization, the contribution degree is stabilized through cross validation of a mean value, the muscle activation weight corresponding to the upper limb action is analyzed, the muscle activation degree characterization is verified, and the muscle activation degree characterization is analyzed. Cooperative extraction of multi-channel muscle characteristics and quantification of muscle activation contribution degree are realized, the problem that the traditional muscle cooperative activation index is insufficient in representation capability of a real nerve-muscle control process is solved, and high-precision and interpretable quantification of a muscle cooperative relationship is realized.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of upper limb surface electromyogram analysis, in particular to an upper limb surface electromyogram analysis method fusing deep learning and width learning. BACKGROUND

[0002] With the rising incidence of neurodegenerative diseases, patients suffer from upper limb motor dysfunction (such as muscle weakness and decreased motor coordination). The upper limb is the most flexible organ for human interaction with the outside world. The current traditional evaluation method is an evaluation scale established by experience, and there is a lack of quantitative evaluation method of upper limb muscle activation state based on physiological signal analysis of patients. Surface electromyogram (sEMG) is an electrophysiological signal generated by skeletal muscle fibers during contraction, which can reflect the neural-muscular activity state and is widely used in the fields of rehabilitation training, prosthesis control and human-computer interaction. Therefore, the characteristics of surface electromyogram can be extracted through deep learning representation method by analyzing surface electromyogram, so as to quantitatively analyze the activation state of muscle coordination in the process of human movement, provide technical support for personalized rehabilitation and functional reconstruction of patients with upper limb dysfunction, and provide a quantifiable examination tool and intuitively display the cooperation state of each muscle.

[0003] For example, the announcement number is: CN114041808B Chinese invention patent discloses a transfer entropy coupling analysis method based on multi-channel surface electromyogram decomposition. Firstly, the multi-channel surface electromyogram is decomposed into motion units by using a convolution kernel compensation method, and the original electromyogram superimposed by multiple motion units is decomposed. After motion unit decomposition, a correlation matrix is established using transfer entropy, and a threshold method or a fixed weighted edge method is used to remove weak connection edges to construct an intermuscular network model related to motor function, draw an undirected graph of each frequency band of electromyogram, and calculate the intermuscular network indicators such as connectivity rate and small-world characteristics to establish a complex network. Through motion unit decomposition and calculation of transfer entropy, the coupling characteristics of electromyogram are fundamentally analyzed.

[0004] For example, the patent for number: CN116269450B Chinese invention patent disclosed based on the patient's limb rehabilitation state evaluation system and method of myoelectric signal: the central processing unit obtains the patient's limb action; the multi-channel signal collector collects the myoelectric signal corresponding to each myoelectric electrode in the set collection time with the set collection frequency; the multi-channel preamplifier carries out signal amplification processing to the myoelectric signal; the multi-channel filter carries out signal filtering processing to the myoelectric signal after signal amplification; the multi-channel analog-to-digital converter carries out analog-to-digital conversion to the myoelectric signal after filtering; the central processing unit is used for analyzing and correcting the myoelectric signal after analog-to-digital conversion, and the myoelectric signal of all myoelectric electrodes after correction is input into the trained convolutional neural network model corresponding to the limb rehabilitation state evaluation corresponding to the patient's limb action for model training and prediction, so as to evaluate the patient's limb rehabilitation state; the display outputs the patient's limb rehabilitation state.

[0005] In the prior art, human motion is instructed by the brain to activate specific muscle groups through neural pathways, and multiple electrodes simultaneously collect the electrical signals on the surface of the muscles. Each channel corresponds to the muscles of a specific part. In the multi-channel sEMG signal, different channels represent different activation states of different muscles in the same action. If the channel signals are directly extracted and fused to process the features, the unique information of individual muscle groups will be lost, the recognition ability of the activation degree of specific muscle groups will be reduced, some muscles or channels will be over-represented or underestimated, and the accuracy of the overall evaluation will be affected. In addition, the deep learning representation method easily ignores the essential differences between channels in the process of extracting the correlation and coupling features between channels. The difference between channels actually affects the analysis of muscle activation patterns and may suppress some important personalized features, thereby affecting the accurate representation of muscle activation state. Therefore, there is a technical problem that the accuracy of the muscle synergistic activation quantitative index is insufficient, and the real neural-muscle control process of the human body cannot be effectively reflected. For example, to make a gesture such as "raising hands", A muscle needs to contract with great force, B muscle needs to contract but with less force, and C muscle needs to relax with less force, and multiple muscles need to cooperate to complete an action. Some patients with limb dysfunction may not be able to make this action because of the weakness of A muscle, and some other patients may not be able to make this action because of the overuse of C muscle. If the specific muscles are not accurately quantified, it is difficult to accurately locate the problem muscle, and it is impossible to develop a personalized rehabilitation training program (such as "strengthening A muscle or relaxing C muscle"). SUMMARY

[0006] To solve the technical problem that the traditional muscle synergistic activation index in the prior art has insufficient representation ability for the real neural-muscle control process, the embodiment of the present application provides a surface electromyography signal analysis method for upper limbs fusing deep and wide learning. The technical scheme is as follows: The multi-channel electromyography signals of the local parts of the upper limbs are collected, a complete sEMG data set is established, and data preprocessing is performed, and the sEMG data set is used to comprehensively and objectively describe the relationship between the entire upper limb muscles and movement; for the collected multi-channel sEMG signals, the same number of convolutional neural network branches with residual structures are designed, effective upper limb action features are extracted from different channels, and the effective upper limb action features can reflect the activation mode of each muscle group in the upper limb action; the residual convolutional neural network structure of each channel outputs features to the respective width learning sub-model, calculates the weight through pseudo-inverse regression, obtains the contribution index through absolute value normalization, and analyzes the muscle activation weight corresponding to the upper limb action through the mean stable contribution degree of cross-validation, and verifies the muscle activation degree representation.

[0007] The technical scheme provided by the embodiment of the application has at least the following beneficial effects: 1. The network structure combining depth and width learning improves the analysis accuracy and efficiency of the upper limb action. Depth learning extracts local features from sEMG signals through a convolutional neural network, and width learning optimizes the contribution of channels, which can accurately analyze the activation weight of muscle groups. Assuming that a certain limb dysfunction patient cannot complete a certain action, the integrated network can extract relevant features from multi-channel signals and accurately analyze the insufficient activation of muscle A, and give corresponding rehabilitation suggestions (such as strengthening the training of muscle A).

[0008] 2. Since the electromyography signals need to cover the key parts of the upper limbs as much as possible to extract the most representative complete upper limb action, the noise ratio and action linkage coupling also increase. The existing sEMG data preprocessing mainly focuses on denoising and signal filtering. Common filtering methods include low-pass filtering or band-pass filtering. These methods can only remove noise in a specific frequency band, and the processing process is relatively simple, which may affect some key information of the signal. The Butterworth filter and sliding window segmentation processing are adopted. In the Butterworth band-pass filtering denoising process, time reversal is also used for phase compensation to ensure that the phase information of the signal is not disturbed and to avoid the phase distortion that may be introduced by traditional filtering methods.

[0009] 3、The convolutional neural network in the prior art usually uses different convolution kernels on each channel for feature extraction, or different network structures are used for each channel. Although the features of each channel can be extracted, there may be problems of channel information interference or excessive parameters. By constructing a residual convolutional neural network, the same structure of convolution block is adopted in the feature extraction of each channel, and the same type of features is ensured to be extracted by sharing the convolution kernel weight. This design avoids signal interference between channels and ensures decoupling between channels. For example, when analyzing the fist clenching action, A muscle and B muscle may be activated at the same time. Through the residual convolutional network and the weight sharing mechanism, the model can ensure that the activation features of each muscle group are independently and accurately extracted without being affected by other channel signals.

[0010] 4、By the width learning sub-model, the regression coefficient weights of each channel are calculated by pseudo-inverse regression, and the contribution index is obtained by absolute value normalization. These indexes can accurately reflect the contribution of each channel in different actions, thereby effectively quantifying the weight of muscle activation; this process enables the network to quantify the activation degree of each muscle group, helping the system better analyze the role of each muscle in upper limb action; for example, when analyzing the patient's hand lifting action, the model can quantify the activation weight of A muscle and B muscle through the contribution index, determine whether A muscle is too tired (insufficient activation), or whether B muscle is over-activated, and help develop a personalized rehabilitation plan. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0012] Figure 1 The flowchart of the upper limb surface electromyography signal analysis method provided by the embodiment of the present application fuses depth and width learning; Figure 2 The schematic diagram of six upper limb actions shown by the sEMG data set provided by the embodiment of the present application; Figure 3 The schematic diagram of the overall structure of CDWS-ResCNN and the local structure of residual block provided by the embodiment of the present application; Figure 4 The structure diagram of the BLS sub-model corresponding to the cth channel provided by the embodiment of the present application. DETAILED DESCRIPTION

[0013] The technical solutions in the present application will be described below with reference to the drawings.

[0014] In the embodiments of the present application, the words such as "exemplary", "for example", etc. are used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "exemplary" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two optionally.

[0015] In order to make the technical problems, technical solutions and advantages to be solved by the present application more clear, the following will be described in detail in combination with the drawings and specific embodiments.

[0016] As shown in Figure 1 The flow chart of the upper limb surface electromyography signal analysis method combining deep and wide learning provided by the embodiments of the present application is shown in the figure, and the method comprises the following steps: S1, multi-channel electromyography signals are collected for local parts of the upper limb, a complete sEMG data set is established, and data preprocessing is performed, and the sEMG data set is used to comprehensively and objectively describe the relationship between the entire upper limb muscle and movement.

[0017] S2, for the collected multi-channel sEMG signals, a convolutional neural network branch with the same number and residual structure is designed, effective upper limb action feature representation is extracted from different channels, and the effective upper limb action feature representation can reflect the muscle group activation mode in the upper limb action. Since different channels represent different activation states of different muscles in the same action, when mining the activation weight regularity of muscles at different positions, a channel-decoupled and weight-shared multi-channel residual convolutional neural network structure (Channel-Decoupled Weight-Shared Residual Convolutional Neural Network, CDWS-ResCNN) is constructed, which not only guarantees to extract high representation ability, independent channel decoupling features in each channel, but also guarantees to extract features with the same properties for all channels, thereby guaranteeing the fairness of evaluating the muscle activation state.

[0018] In each branch of CDWS-ResCNN, each branch consists of 6 residual blocks. Input is passed across layers via shortcut connections and added to the output of the convolutional layers (i.e., identity mapping). Specifically, each residual block consists of two layers of one-dimensional convolutions and ReLU activation functions. A 1×1 convolution is introduced to achieve channel dimension scaling, ensuring the effectiveness of feature fusion and mitigating gradient vanishing and model degradation issues in deep networks through cross-layer propagation. Furthermore, to enhance the network's efficient extraction of key features from sEMG signals, a SoftPool is introduced after each residual block for temporal (1D) downsampling, extracting highly discriminative deep features block by block.

[0019] S3, the residual convolutional neural network structure of each channel outputs features and inputs them into its respective width learning sub-model. The weights are calculated through pseudo-inverse regression, and the absolute value is normalized to obtain the contribution index. The contribution index is stabilized by cross-validation of the mean. The muscle activation weights corresponding to upper limb movements are analyzed, and the representation of the degree of muscle activation is verified.

[0020] In this embodiment, for the task of assessing human upper limb motor function, by deeply exploring the patterns of activation weights of upper limb muscle groups in different movements, a set of interpretable algorithms for measuring the contribution of human upper limb muscle activation weights is constructed, providing important quantitative indicators and theoretical support for upper limb motor function assessment.

[0021] An architecture integrating deep learning and breadth learning is constructed to deeply mine and quantify the muscle activation weights corresponding to upper limb movements, obtaining interpretable muscle activation contributions. Based on CDWS-ResCNN deep learning, deep features of sEMG signals are extracted from a "vertical" perspective through multi-layer nonlinear stacking, backpropagation, and repeated parameter tuning. Building upon this, a breadth learning system (BLS) is combined to further mine and quantify muscle activation weights from a "horizontal" perspective, making model parameters more transparent, the model more interpretable, and easier to tune and deploy.

[0022] like Figure 2 The diagram shown is a schematic of six upper limb movements displayed in the sEMG dataset provided in this embodiment of the invention. The main content is: six upper limb movements were designed based on human daily behavior, in which the movement involves grasping a thermos cup filled with water (weighing about 550g).

[0023] Further, the partial parts of the upper limbs are collected to collect multi-channel sEMG signals, and a complete sEMG data set is established, and the specific process is as follows: according to the analysis result of the upper limb movement process, the position of the electrode sheet of the electromyograph for collecting sEMG is determined; through the upper limb movement task, the sEMG signals of different muscle groups in different contraction states are collected through the electrode sheet of each channel, a multi-channel sEMG data set including several types of upper limb actions is constructed, and the muscle contraction of each action is analyzed; the relationship between the muscle contraction mode and the movement instruction is extracted, and the change of the electromyographic signal in different movement states is analyzed.

[0024] In the embodiment, the upper limb movement is a highly coordinated process, which is completed by the central nervous system (CNS) and the peripheral nervous system (PNS), involves the cerebral cortex, the spinal cord, the peripheral nerves and the skeletal muscles innervated by them, is a comprehensive effect produced by multiple muscles under the mutual coordination, and the activation characteristics of a single muscle are inconsistent when the upper limb performs different movement postures. After the human upper limb movement is deeply analyzed from the perspective of movement physiology, the muscle contraction state of the sEMG collection position is comprehensively considered, and the electromyographic signal data set of the human upper limb movement based on the whole upper limb is established.

[0025] In order to collect the sEMG signals of the key muscles or muscle groups in different contraction states in the upper limb movement process, 6 types of upper limb movements are designed in combination with human daily behaviors, the muscle contraction conditions corresponding to different movements are specifically shown in Table 1, the 6 types of movements uniformly mobilize the states of the corresponding muscle groups, so that most of the muscles can exhibit different contraction states in different movements, and thus the collected sEMG signals can more comprehensively and objectively describe the relationship between the muscles and the movement.

[0026] Table 1 Muscle contraction form of the upper limb under different movements

[0027] Among them, CA represents concentric contraction, when the muscle produces active force and the length is shortened at the same time, concentric contraction occurs, which shortens the distance between the proximal and distal attachment points of the muscle, EA represents eccentric contraction, when the muscle produces active force (tries to contract) but is pulled to a longer length by a more dominant external force at the same time, eccentric contraction occurs, and IA represents isometric contraction, when the muscle produces active force while maintaining a constant length, isometric contraction occurs.

[0028] Further, the determination of the position of the electrode sheet of the electromyography for collecting sEMG also includes full-arm coverage collection, and the specific process is as follows: collecting a plurality of collection points covering key muscle groups of the full arm, and according to the muscle contraction form, the upper limb movement quantitative index, and the distribution size of the muscle on the skin surface of the human upper limb, representative muscles affecting the shoulder, upper arm and forearm are selected.

[0029] In the embodiment, existing sEMG data sets are mostly concentrated in gesture action recognition, and few describe sEMG signals in the whole upper limb full-arm range. A complete upper limb action is the result of the cooperation of the shoulder, upper arm, forearm and hand (the result of multi-muscle group cooperation), so the electromyography signal collection needs to cover as many key parts of the upper limb as possible to extract the most representative complete upper limb action.

[0030] It should be noted that, considering that sEMG is collected during the contraction of muscle fibers under the skin during the contraction of superficial muscles, and it is difficult to collect stable electromyography signals of the hand muscles by using ordinary electrode sheets, therefore, the electromyography signals of the forearm muscles controlling the hand action are collected to reflect the hand posture in the upper limb action.

[0031] Further, the data preprocessing includes Butterworth filter denoising, and the specific process is as follows: obtaining the original electromyography signal and performing time reversal and filter processing to improve the signal processing accuracy. The original electromyography signal represents the original time domain electrical signal sequence recorded directly by the sensor without processing. The filter frequency range is set to the signal allowed range, and the frequency components exceeding the signal allowed range are filtered out.

[0032] In the embodiment, a Butterworth band-pass filter is applied to the original signal to obtain an intermediate signal after forward filtering; the intermediate signal sequence after forward filtering is sequentially reversed; the same Butterworth band-pass filter is applied to the time-reversed intermediate signal again for reverse filtering; and the signal sequence after reverse filtering is sequentially restored to obtain the filter result, so as to realize complete compensation of the phase and achieve the purpose of zero phase distortion while essentially offsetting the delay.

[0033] The electromyography signal collection needs to cover as many key parts of the upper limb as possible to extract the most representative complete upper limb action, and a forward-reverse (Forward-backward) infinite impulse response (Infinite Impulse Response, IIR) Butterworth band-pass filter (Butterworth Band-pass Filter, Butterworth BPF) is used for time domain filtering.

[0034] The band-pass filter can retain the part in a specific frequency range in the signal, and filter out the frequency components exceeding the signal allowed range. When the original signal is x[n], the normalized cutoff frequencies are as follows:

[0035] where ω low is the low frequency cutoff angular frequency, ω high is the high frequency cutoff angular frequency, f s is the sampling frequency, f N =f s / 2 is the Nyquist frequency, f low is the low cutoff frequency, f high is the high cutoff frequency.

[0036] IIR Butterworth filter can achieve faster roll-off with lower order, reducing the amount of calculation, its difference equation can be written as: , where N is the order of the filter, representing the maximum historical time point involved in the process of IIR filter; a k and b k are the gain coefficients of the filter, determined by the Butterworth design method, y[n-k] is the sample of the filter output sequence y[n] before k time, and x[n-k] is the sample of the filter input sequence x[n] before k time, n is the index of the current time, representing the time being calculated, k is the index of time delay, used to represent the values of different time in y[n] and x[n].

[0037] And the conventional IIR filter will introduce group delay, which needs to be compensated to align with the original signal, in order to solve this problem, the invention adopts forward-backward filtering strategy, that is, first forward filtering the signal, then backward filtering the result after time reversal, and finally reversing the result to the original sequence, so as to realize complete compensation of phase, which essentially offsets the delay and achieves the purpose of zero phase distortion, the process can be represented as: , where F -1 is the inverse Fourier transform, used to convert the frequency domain signal back to time domain, is the frequency spectrum of the original signal x[n], is the frequency response function of Butterworth band-pass filter, indicates the corresponding frequency response after time reversal. The offset phase response ensures that the filtered signal retains the original phase structure.

[0038] Further, the data preprocessing also includes sliding window segmentation processing, and the specific process is: according to the window length required for feature extraction and calculation, the denoised sEMG signal is segmented into a plurality of sliding windows, signal segments for subsequent feature extraction and analysis are generated, the signal after the sliding window can expand the training sample set scale without significantly increasing the model calculation complexity, the expression ability of the model to the feature is improved, and more useful features are extracted from the non-stationary sEMG signal.

[0039] When the processing of a window is completed, the sliding window continues to slide forward, new data points will be added to the end of the window, and old data points will be discarded until the entire signal is processed.

[0040] In the embodiment, in order to extract more useful features from the non-stationary sEMG signal, the filtered sEMG signal y=[y1, y2,..., yN] is segmented by using a sliding window, and the segmented signal is denoted as y=[y1, y2,..., yN-w+1], where w is the window length, and N is the length of the sEMG signal. n ] R T Segmentation processing is performed, the signal after the sliding window can expand the training sample set scale without significantly increasing the model calculation complexity, the expression ability of the model to the feature is improved, and the subsequence is continuously extracted from the original time sequence sEMG signal, and the signal of the i(i≥1)th window is: , Wherein, w is the set window length, st is the sliding step, represents the feature sequence extracted by the i-th sliding window.

[0041] As shown in Figure 3 , it is a schematic diagram of the overall structure of the CDWS-ResCNN and the local structure of the residual block provided by the embodiment of the application, and the main content is: a deep learning model based on the CDWS-ResCNN structure is displayed, which extracts the features of the sEMG signal through the convolutional neural network (CNN) and the residual connection. Through processing, fusion and classification of signals in multiple channels, the model can extract rich muscle activity information from the original signal for subsequent action recognition and muscle state analysis.

[0042] Further, the extraction of effective representation of upper limb action features from different channels also includes constructing a weight sharing mechanism in the network, and the specific process is: each branch uses a residual convolution block (Residual Block) with the same structure and parameters to extract local features, and all branches share the same set of convolution kernel weights.

[0043] The features extracted by each branch are merged at the back end of the network, the difference between the current network output and the target is calculated through a loss function, and a loss value is obtained.

[0044] The loss value is passed back to the network through the back propagation algorithm to adjust the weight parameters, and finally the network is continuously optimized to improve the accuracy of feature extraction.

[0045] In this embodiment, the loss value reflects the prediction error of the model. In the action classification task, the cross-entropy loss function is usually used to calculate the difference between the output class probability distribution and the true class label; in the muscle activation quantification task, the global optimality of the weight solution is guaranteed by solving the convex optimization problem, so as to measure the difference between the predicted value and the true value.

[0046] When processing different channel signals, although each channel has an independent input path, the same set of convolution kernel weights and network module structure need to be shared in the local feature extraction stage, so as to ensure that each channel extracts the same type of features. In the CDWS-ResCNN network, although each channel corresponds to different muscle group signals (such as A muscle, B muscle and C muscle), by sharing convolution kernels and network modules, the activation features of A, B and C muscles will be extracted as consistent feature types (such as mean, variance, etc.), ensuring that the network can fairly evaluate the activation state of each muscle group without being dominated by some muscle groups with stronger activation in feature extraction.

[0047] The shared weight mechanism ensures that the network processes different muscle group signals consistently and can effectively extract muscle coordination features. For example, A muscle and B muscle both play a role in the same action, but their activation levels are different. By sharing convolution kernels and residual blocks, the network can accurately evaluate the contribution of each muscle group to the entire action, ensuring the extraction of muscle coordination features.

[0048] Further, the residual convolutional neural network structure of each channel outputs features to the respective width learning sub-model, which further includes feature standardization processing of the input features. The specific process is as follows: different channel extraction of the feature data to be standardized, and calculation of the mean and standard deviation of each feature channel of the original feature value.

[0049] The standardized feature data will be input into the BLS model for learning and prediction, ensuring the matching degree of feature value scale and the balanced contribution of each feature to training.

[0050] Before standardization, the original feature data of each channel is the same type of deep feature extracted from the sEMG signal through the CDWS-ResCNN network, and after standardization, the feature values of each channel are converted into data with zero mean and unit variance, for example, after standardization, the muscle signal originally in the range of 0 to 1000 may become a feature value with a mean of 0 and a standard deviation of 1.

[0051] The purpose of feature standardization processing is to convert features of different scales into standard normal distribution data with uniform scale. Through this processing, the mean of the feature data becomes 0 and the standard deviation becomes 1, so that the features are in the same scale, avoiding the influence of different feature scale differences on model training, making the feature data suitable for machine learning algorithms such as BLS model, which helps to speed up the convergence of the model and improve the performance.

[0052] In this embodiment, before the features extracted by each channel are input into the BLS sub-model, it is necessary to ensure the matching degree of the feature value scale and ensure the balanced contribution of each feature to the training. For this purpose, the features are standardized by using the standard score (z-score) algorithm, and the specific formula is as follows: , , , wherein X C represents the feature value of the cth channel after standardization, x c is the original feature value of the cth channel, is the mean of the original feature, is the standard deviation of the original feature, M is the total number of features, and j is the feature value of the jth sample in the cth channel when calculating the mean and standard deviation. All M samples are traversed to calculate.

[0053] As shown in Figure 4 , it is the structure diagram of the BLS sub-model corresponding to the cth channel provided by the embodiment of the application, and the main content is: the relationship between the feature mapping node, the enhanced feature node and the output in the BLS sub-model is shown. Through the layer-by-layer processing of these modules, the model can extract useful features from the input sEMG signal and finally make a prediction.

[0054] Further, the residual convolutional neural network structure of each channel outputs features to the respective width learning sub-model, and the specific process is: the standardized features are input into the width learning sub-model, and in the input layer of the width learning sub-model, the standardized features are divided by channel and nonlinearly mapped by using the hyperbolic tangent activation function to obtain the feature mapping node.

[0055] The feature mapping nodes are re-mapped non-linearly to generate enhanced feature nodes, further improve the learning performance of the network, and can fit more complex data structures; the feature mapping nodes and the enhanced feature nodes are jointly used as hidden layers of the corresponding channel width learning sub-model, and are transmitted to an output layer to obtain output values of the width learning sub-model and connection weights from the hidden layer to the output layer.

[0056] In the embodiment, in order to ensure the interpretability of the muscle activation degree algorithm, the prior art usually adopts a structure-transparent "white box" algorithm, such as a traditional machine learning algorithm such as SVM, LDA, KNN, etc. The present application fully utilizes the advantages of width learning which also has the feature of structure transparency, and combines the powerful constraints on the CNN deep learning algorithm, so that the architecture integrating deep learning and width learning can establish an interpretable and quantifiable muscle activation degree analysis on the basis of extracting sEMG deep features, and is more interpretable and scientific.

[0057] In the input layer, the width learning does not directly input the features into the network structure, but performs feature mapping on the input data. The feature mapping can effectively improve the representation ability of the features in the high-dimensional space, and can also enhance the discrimination performance of the model.

[0058] The standardized features are divided by channels, the fixed order of the electrode / muscle group channels is determined (such as Ch1→Ch2→…→ChC), and the same type of deep features is extracted for each channel through the CDWS-ResCNN network; in the same batch, the timestamps of each row (sample / window) on all channels are consistent; the standardized features of each sample are block-wise spliced according to the predetermined channel order: Ch1 feature column first, then Ch2 feature column, and so on until ChC. A "total feature table" is obtained, which is block-wise arranged according to the channels, and records the column block range of each channel. When the input of a certain channel is needed, the entire block column of the channel is taken out from the total feature table according to the mapping.

[0059] The divided features are input into the width learning sub-model, and the Hyperbolic Tangent (Tanh) activation function is used to input the features for non-linear mapping. The function can output continuous values that are symmetric about the origin and have an interval of [-1, 1], which can effectively suppress extreme values of the features during the non-linear transformation of the input features, stabilize the feature distribution, and reduce the risk of numerical overflow. The input data of the model is the standardized features of each channel X C , the first p group of feature mapping nodes Z P is: , wherein, and are random weights and biases, respectively, and n 1 feature mapping nodes are combined to obtain In order to further improve the learning performance of the network and be able to fit more complex data structures, the feature mapping nodes are further subjected to hyperbolic tangent nonlinear mapping to generate enhanced feature nodes, and the first group of enhanced feature nodes q H q are: , Thus, n2 enhanced feature nodes are obtained, and after combination, the feature mapping nodes and the enhanced feature nodes are jointly used as the hidden layer of the corresponding channel BLS submodel and are transmitted to the output layer, and the output of the BLS submodel of the cth channel and the connection weight W c between the hidden layer and the output layer can be expressed as: .

[0060] Further, the weight is calculated by pseudo-inverse regression, and the contribution index is obtained by absolute value normalization. The specific process is as follows: the output value of the width learning submodel is subjected to pseudo-inverse operation, and the regression coefficient weight of each channel width learning branch is solved.

[0061] The absolute values of the regression coefficient weights on all mapping nodes are summed, and the sum of all channels in the same upper limb action is normalized to obtain the contribution index of each channel corresponding to each type of action. The contribution index is used to reflect the weight contribution of the channel feature in the model discrimination process and provides an effective quantitative basis for channel importance ranking and muscle function analysis.

[0062] The relative contribution index is calculated by cross-validation algorithm to obtain the normalized contribution of each channel to each category in each fold. The average value of the results after a certain number of folds is taken as the final contribution index. The final contribution index is used to reduce the accidental influence caused by sample division to improve the reliability of the activation weight.

[0063] In this embodiment, the BLS is subjected to pseudo-inverse operation, i.e., the formula is transformed to obtain: , wherein is the pseudo-inverse matrix obtained by ridge regression approximation, and the parameters , , and ​All of these are random variables that remain unchanged during network training, so the calculation process is a convex optimization problem. Through optimization... The value of determines the network output fitted value. With known upper limb movement target labels Closer. The optimization problem is modeled as follows: , in, It minimizes the loss function, thereby optimizing the weight matrix. , Let ||·|| be the regularization coefficient. F The Frobenius norm is used to control the minimization of training error; due to the... Adding a positive number to the diagonal of the original inverse pseudo-equation gives it exactly one solution. , Where I is the identity matrix, Then the analytical solution to this optimization problem is: ; The contribution index of channel C to the normalized processing of the k-th type of upper limb movement is defined as:

[0064] Where 'a' is the index of the sample, representing the feature contribution of the 'a'-th sample, and 'b' is the index of the feature dimension, representing the contribution of the 'b'-th feature dimension to the final result.

[0065] This contribution index quantifies the relative activation weight of each channel in different upper limb movement categories. The larger the contribution value for a certain category, the higher the contribution of that channel in representing and judging that category, and the stronger the activation degree.

[0066] To reduce the randomness of sample partitioning, this invention employs... Folded cross-validation improves the reliability of activation weights. Specifically, it calculates the normalized contribution of each channel to each category for each fold, and finally selects the optimal value. The average of the calculated results is used as the final contribution index of that channel to each category. This strategy effectively improves the stability and repeatability of the contribution evaluation, as shown in the following formula: , Where f represents the index of the f-th fold, which is the number used to mark the current fold in F-fold cross-validation.

[0067] The muscle activation weight quantification method based on the BLS sub-model ensures the rigor of measuring the activation degree of muscles or muscle groups in upper limb movements and has good interpretability. It is an effective proof of the differential role of each sEMG channel in multi-classification tasks.

[0068] Furthermore, the muscle activation weights corresponding to upper limb movements were analyzed, and the representation of muscle activation degree was verified. The specific process is as follows: the output values ​​of the corresponding channels of the learning sub-model of a set width are concatenated column by column to obtain the prediction matrix to be classified; the prediction matrix to be classified is input into a multilayer perceptron (MLP) to perform category classification prediction and output the action classification probability. The action classification probability is used to reflect the degree of activation of each action category, thereby indirectly representing the activation degree of the corresponding muscle group.

[0069] The quality of representation information in the output of each channel width learning sub-model is judged based on the action classification probability: If the movement classification probability is greater than the set movement classification probability, then the muscle activation weight of this group is determined to successfully represent the degree of muscle activation of the movement, and the greater the contribution value of the muscle group when performing the movement, the stronger the degree of muscle activation.

[0070] If the movement classification probability is not greater than the set movement classification probability, then the muscle activation weight of that group is determined to be unable to represent the degree of muscle activation for that movement, and retraining is required until the movement classification probability is greater than the set movement classification probability.

[0071] In this embodiment, to verify the action representation capability of the BLS sub-model weight matrix and thus demonstrate the effectiveness of the muscle activation weight contribution model, this invention establishes an MLP-based classification model for the output of each BLS sub-module. The model's classification performance is used to illustrate its representation capability. Specifically, the output matrices of the corresponding channels of each BLS sub-model are sorted column-wise. The matrix S to be classified is obtained by concatenating the matrix.

[0072] in, Indicates will Each BLS submodel corresponds to a channel output matrix. The columns are concatenated to obtain the prediction matrix S to be classified. S is then fed into the MLP for classification to output the final classification probability. The MLP makes full use of the complementarity and high-order synergy between channels, so that the channel contribution value analysis has the full-process explanatory power from linear discrimination to deep fusion, and can deeply explore the correlation between key channels and action recognition.

[0073] The above-described embodiments can be implemented in part or in whole through software, hardware (e.g., circuitry), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When loaded and executed by a computer, the computer instructions or computer programs can generate the flow or function according to the embodiments of the present application in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, such as from one website, computer, server, or data center to another website, computer, server, or data center through a wired (e.g., infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more collections of available media. The available media can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0074] It should be understood that the term "and / or" in this document is merely used to describe an associated relationship between associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the front and rear associated objects, but can also represent an "and / or" relationship. The specific meaning can be understood according to the context before and after.

[0075] In various embodiments of the present application, the size of the sequence number of each process described above does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0076] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0077] Those skilled in the art can clearly understand the specific working process of the device, the apparatus and the unit described above can refer to the corresponding process in the foregoing method embodiments for the convenience and brevity of description, which will not be repeated here.

[0078] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for analyzing surface electromyography signals of upper limbs by fusing deep learning and width learning, characterized in that, The method comprises the following steps: S1, collecting multi-channel electromyography signals of local parts of the upper limbs, establishing a complete sEMG data set, and performing data preprocessing, wherein the sEMG data set is used to comprehensively and objectively describe the relationship between the entire upper limb muscles and movements; S2, for the collected multi-channel sEMG signals, a convolutional neural network branch with the same number and residual structure is designed to extract effective upper limb action features from different channels, wherein the effective upper limb action features can reflect the activation mode of each muscle group in the upper limb action; S3, the residual convolutional neural network structure of each channel outputs features to a respective width learning sub-model, calculates the weight through pseudo-inverse regression, obtains the contribution index through absolute value normalization, and analyzes the muscle activation weight corresponding to the upper limb action through cross-validation mean stable contribution, and verifies the muscle activation degree representation.

2. The surface electromyography signal analysis method of claim 1, wherein the fusion of depth and width learning is performed by a convolutional neural network. The method for collecting multi-channel electromyography signals of local parts of the upper limbs and establishing a complete sEMG data set comprises the following steps: According to the analysis result of the upper limb movement process, the position of the electrode sheet of the electromyography instrument for collecting sEMG is determined; Through the upper limb movement task, the sEMG signals of different muscle groups in different contraction states are collected through the electrode sheet of each channel, a multi-channel sEMG data set including several types of upper limb actions is constructed, and the muscle contraction of each action is analyzed; The relationship between muscle contraction mode and movement instruction is extracted, and the change of electromyography signal in different movement states is analyzed.

3. The surface electromyography signal analysis method of claim 2, wherein the fusion of depth and width learning is performed by a convolutional neural network. The method for determining the position of the electrode sheet of the electromyography instrument for collecting sEMG further comprises collecting sEMG from the whole arm, and the specific process comprises the following steps: Collecting several collection points covering the key muscle groups of the whole arm; According to the muscle contraction form, the upper limb movement quantitative index, and the distribution size of the muscle on the surface of the human upper limb skin, the representative muscles affecting the shoulder, upper arm and forearm are selected.

4. The surface electromyography signal analysis method of claim 1, wherein the fusion of depth and width learning is performed by using a convolutional neural network (CNN) and a recurrent neural network (RNN). The data preprocessing comprises the following steps: The original electromyography signal is obtained, and time reversal and filtering are performed to improve the signal processing accuracy, wherein the original electromyography signal represents the original time domain electrical signal sequence recorded directly by the sensor without processing; The frequency range of the filter is set to the signal allowed range, and the frequency components exceeding the signal allowed range are filtered out.

5. The surface electromyography signal analysis method of claim 4, wherein the fusion of depth and width learning is performed by a convolutional neural network. The data preprocessing further comprises the following steps of sliding window segmentation processing: According to the window length required for feature extraction and calculation, the sEMG signal after denoising is segmented into a certain number of sliding windows to generate signal segments for subsequent feature extraction and analysis; When the processing of a window is completed, the sliding window continues to slide forward, and new data points will be added to the end of the window, and old data points will be discarded until the whole signal is processed.

6. The surface electromyography signal analysis method of claim 1, wherein the fusion of depth and width learning is performed by a convolutional neural network. The method for extracting effective upper limb action features from different channels further comprises the following steps of constructing a weight sharing mechanism in the network: Each branch uses a residual convolution block with the same structure and parameters to extract local features, and all branches share the same set of convolution kernel weights; The features extracted by each branch are combined at the back end of the network, and the difference between the current network output and the target is calculated through a loss function to obtain a loss value. The loss value is passed back to the network through the backpropagation algorithm to adjust the weight parameters.

7. The surface electromyography signal analysis method of claim 1, wherein the fusion of depth and width learning is performed by a convolutional neural network. The residual convolutional neural network structure for each channel outputs features, and its respective width learning sub-model also includes feature standardization processing of the input features. The specific process is as follows: Extract the feature data to be standardized from different channels, and calculate the mean and standard deviation of the original feature values ​​for each feature channel. The standard score algorithm is used for standardization. The standardized feature data will be used as input to the BLS model for learning and prediction.

8. The surface electromyography signal analysis method of claim 7, wherein the fusion of depth and width learning is performed by a neural network. The residual convolutional neural network structure of each channel outputs features that are input into their respective widths to learn a sub-model. The specific process is as follows: The standardized features are input into the width learning sub-model. In the input layer of the width learning sub-model, the standardized features are divided by channel and non-linearly mapped using the hyperbolic tangent activation function to obtain feature mapping nodes. The feature mapping nodes are re-mapped nonlinearly to generate enhanced feature nodes; The feature mapping nodes and enhanced feature nodes are used together as the hidden layer of the corresponding channel width learning sub-model and passed to the output layer to obtain the output value of the width learning sub-model and the connection weights from the hidden layer to the output layer.

9. The surface electromyography signal analysis method of claim 1, wherein the fusion of depth and width learning is performed by a convolutional neural network. The contribution index is obtained by calculating weights through pseudo-inverse regression and normalizing the absolute value. The specific process is as follows: The output value of the width learning sub-model is pseudo-inversely calculated to solve the regression coefficient weights of each channel width learning branch; The absolute values ​​of the regression coefficient weights on all mapping nodes are summed, and the sum of all channels in the same type of upper limb movement is normalized to obtain the contribution index of each channel for each type of movement. The contribution index is used to reflect the weight contribution of channel features in the model discrimination process. The relative contribution index is calculated using a cross-validation algorithm to obtain the normalized contribution of each channel to each category at each fold. The average value of the results after a set number of folds is taken as the final contribution index. The final contribution index is used to reduce the randomness of sample partitioning and improve the reliability of activation weights.

10. The surface electromyography signal analysis method of claim 1, wherein the fusion of depth and width learning is performed by a convolutional neural network. The analysis of upper limb movements corresponds to muscle activation weights, and the verification of muscle activation levels are performed. The specific process is as follows: The output values ​​of the corresponding channels of the learning sub-models with a set width are concatenated column by column to obtain the prediction matrix to be classified; The prediction matrix to be classified is input into a multilayer perceptron to perform category classification prediction and output the action classification probability. The action classification probability is used to reflect the degree of activation of each action category, thereby indirectly representing the degree of activation of the corresponding muscle group. The quality of representation information in the output of each channel width learning sub-model is judged based on the action classification probability: If the action classification probability is greater than the set action classification probability, then the muscle activation weight of this group is determined to successfully represent the degree of muscle activation of this action, and the greater the contribution value of the muscle group when performing this action, the stronger the degree of muscle activation. If the movement classification probability is not greater than the set movement classification probability, then the muscle activation weight of that group is determined to be unable to represent the degree of muscle activation for that movement, and retraining is required until the movement classification probability is greater than the set movement classification probability.

Images