A method and system for identifying the force-sensitive state of upper limb movements based on brain-myography fusion

Abstract

The present invention relates to the field of brain and myoelectric signal classification tasks, and in particular to a method and system for identifying the force-sensitive state of upper limb movements based on brain and myoelectric fusion. The original EEG signal and the original EMG signal are preprocessed, the preprocessed EEG signal is resampled, the mutual information and similarity between the preprocessed EMG signal and the resampled EEG signal are calculated, and the brain-muscle information is spliced ​​together. Based on the preprocessed EMG signal, the preprocessed EEG signal and the brain-muscle information, the EMG feature, the EEG feature and the brain-muscle feature are extracted respectively through a three-branch network structure, and flattened into a one-dimensional feature vector, and feature splicing is performed to obtain the overall feature. The force-sensitive state classification of the upper limb movement is completed by connecting two fully connected layers and a Softmax layer in series in sequence. The force-sensitive state classification is performed by extracting the EMG feature, the EEG feature and the brain-muscle feature through the designed three-branch network structure and fusing them to improve the accuracy and stability of the force-sensitive state classification of the upper limb movement.

Description

Technical Field

[0001] The present invention relates to the field of brain and myoelectric signal classification tasks, and in particular to a method and system for identifying the force-sensitive state of upper limb movements based on brain and myoelectric signal fusion. Background Art

[0002] Brain-computer interface (BCI) technology enables a direct connection between the brain and external upper limb devices. Through real-time control using EEG signals, users can perform a range of actions, such as grasping objects. This technology can restore or improve motor skills for those with limited upper limb function due to nerve damage or other neurological disorders. This technology can help amputees directly control their intelligent prostheses and other upper limb assistive devices, such as intelligent exoskeletons, using EEG signals, thereby restoring some of their daily functions. Electroencephalography (EEG), the primary method for implementing BCI technology, has become the preferred control system due to its high timeliness, ease of operation, and high cost-effectiveness. However, BCI systems still have some limitations. Over-reliance on a single EEG signal can compromise system stability. For example, changes in the force-sensitive state of the user's limb movements can alter the quality of the control signal.

[0003] With the rapid development of related fields, especially neuroscience and bioengineering in recent years, the application scope and potential of brain-computer interface technology are constantly expanding. Today, brain-computer interface technology can not only convert EEG signals into control signals, but also integrate EEG signals with other bioelectric signals, such as electromyography and heart rate, through information fusion technology to enhance the accuracy and stability of the system. These advances provide new directions and hope for the future application of brain-computer interface technology. Among them, surface electromyography is a technology that records the electrical signals generated by muscle activity by placing electrodes on the surface of the skin. This method can non-invasively monitor and analyze muscle electrical activity and is widely used in medical diagnosis, sports science, rehabilitation engineering, human-computer interface and other fields.

[0004] Brain-electromagnetic (EMG) information fusion is an advanced scientific research field that combines electrophysiological signals from the brain and muscles to provide more comprehensive neuromuscular activity information. EMG information fusion technology attempts to overcome the limitations of a single signal source by integrating these two signals and improve the comprehensive interpretation ability of the data. Therefore, combining EEG and EMG signals and fully extracting the features in EMG signals are particularly important for developing efficient brain-computer interface systems.

[0005] Although existing technologies can determine whether the brain intends to move the upper limbs and respond, it is difficult to distinguish the force-sensitive state of the upper limbs in different movement modes, which limits the ability to finely control external devices. Summary of the Invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that although the existing technology can determine whether the brain has the intention to move the upper limbs, it has limited recognition of the force-sensitive state of the upper limbs in different movement modes, which limits the ability to finely control external devices.

[0007] To solve the above technical problems, the present invention provides a method for identifying the force-sensitive state of upper limb movements based on brain and myoelectricity fusion, comprising the following steps:

[0008] A method for identifying upper limb force-sensitive states based on brain-myography fusion includes the following steps: obtaining original EEG signals and original EMG signals under different upper limb force-sensitive states, recording corresponding force-sensitive state data, constructing a data set, and dividing the data set into a training set and a test set;

[0009] Preprocessing the original EEG signal and the original EMG signal to obtain preprocessed EEG signal and EMG signal;

[0010] Resampling the preprocessed EEG signal, calculating the mutual information and similarity between the preprocessed EMG signal and the resampled EEG signal based on the preprocessed EMG signal and the resampled EEG signal, and concatenating the mutual information and similarity between the preprocessed EMG signal and the resampled EEG signal to form brain-muscle information;

[0011] Constructing a brain and muscle electrosensitivity state recognition model, the brain and muscle electrosensitivity state recognition model includes: a three-branch feature extraction module and a recognition module;

[0012] Inputting the preprocessed electromyographic signal, the preprocessed electroencephalographic signal and the brain-muscle information into the three-branch feature extraction module, and using the three-branch feature extraction module to extract electromyographic features, electroencephalographic features and brain-muscle features;

[0013] The myoelectric features, EEG features, and brain-muscle features are input into the recognition module to obtain the category of the upper limb movement force sensitivity state, including:

[0014] Flatten the EMG features, EEG features and brain-muscle features into one-dimensional feature vectors, and perform feature splicing on the one-dimensional feature vectors corresponding to the EMG features, EEG features and brain-muscle features to obtain the overall features;

[0015] The overall features are input into two serially connected fully connected layers and Softmax layers to classify the upper limb force sensitivity state and obtain the category to which the upper limb force sensitivity state belongs;

[0016] The preprocessed EMG signals, preprocessed EEG signals, and brain-muscle information of the training set were input into the brain-muscle electrosensitivity state recognition model to obtain the probability of the upper limb force sensitivity state category in each training task. During training, the cross-entropy loss function was used to minimize the difference between the classification results and the true labels. The Adam optimizer was used to optimize the cross-entropy loss function and update the learning rate.

[0017] The performance of the brain and muscle electrosensitivity state recognition model was evaluated using the classification accuracy on the test set.

[0018] Preferably, the original EEG signal is preprocessed to obtain a preprocessed EEG signal, including: performing 0.5Hz high-pass filtering on the original EEG signal, filtering out 50Hz power frequency interference and removing baseline interference, extracting the signal segment corresponding to the force-sensitive state of the upper limb movement, and normalizing the data of each channel to obtain the preprocessed EEG signal.

[0019] Preferably, the original electromyographic signal is preprocessed to obtain a preprocessed electromyographic signal, including: performing high-pass filtering on the original electromyographic signal, retaining signals above the 20 Hz frequency band, extracting signal segments corresponding to the force-sensitive state of the upper limb movement, and obtaining the preprocessed electromyographic signal.

[0020] Preferably, the mutual information and similarity between the preprocessed electromyographic signal and the resampled electroencephalographic signal are calculated as follows:

[0021] The mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity between the preprocessed EMG signals and the resampled EEG signals were calculated to obtain the mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity between the preprocessed EMG signals and the resampled EEG signals.

[0022] Preferably, the preprocessed electromyographic signal is input into the three-branch feature extraction module, and the electromyographic feature is extracted using the three-branch feature extraction module, including:

[0023] The preprocessed EMG signal is input into the 2D convolution module to extract features from the preprocessed EMG signal to obtain global channel features;

[0024] The preprocessed EMG signals are divided into four groups, each containing two adjacent channels. The four groups of EMG signals are input into the 2D convolution module respectively, and the temporal and spatial information of each group of EMG signals are extracted to obtain four groups of local channel features.

[0025] The four groups of local channel features are spliced ​​and fused with the global channel features to obtain the electromyographic features.

[0026] Preferably, the brain-muscle information is input into the three-branch feature extraction module, and the brain-muscle features are extracted using the three-branch feature extraction module, including:

[0027] The brain-muscle information is input into the 2D convolution module, and features of the brain-muscle information are extracted to obtain brain-muscle features.

[0028] Preferably, the preprocessed EEG signal is input into the three-branch feature extraction module, and the EEG features are extracted using the three-branch feature extraction module, including:

[0029] The preprocessed EEG signal passes through the 2D convolution module and the batch normalization layer in sequence to extract the time information of the EEG signal. The extracted time information of the EEG signal passes through the depthwise separable convolution module, the activation function layer and the batch normalization layer in sequence to obtain the fusion characteristics of the time information and spatial information of the EEG signal.

[0030] Preferably, the calculation formula of the cross entropy loss function is:

[0031]

[0032] Among them, H represents the number of samples, C represents the number of upper limb movement force sensitivity state categories;

[0033] If the classification result is the same as the true label, then f hc is 1, otherwise 0;

[0034] p hc It represents the probability that the hth sample belongs to the cth upper limb movement force-sensitive state category.

[0035] The present invention also provides a system for identifying the force-sensitive state of upper limb movements based on brain-myoelectric fusion, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the steps of the above-mentioned method for identifying the force-sensitive state of upper limb movements based on brain-myoelectric fusion are implemented.

[0036] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0037] The present invention describes a method and system for identifying the force-sensitive state of upper limb movements based on brain-electromagnetic fusion. The method and system resample the preprocessed EEG signal, calculate the mutual information and similarity of each channel of the resampled EEG signal and the preprocessed EMG signal, and concatenate the calculated mutual information and similarity to form brain-muscle information. Based on the preprocessed EMG signal, the preprocessed EEG signal and the brain-muscle information, the method extracts the EMG feature, the EEG feature and the brain-muscle feature, respectively, flattens the EMG feature, the EEG feature and the brain-muscle feature into one-dimensional feature vectors, and concatenates the one-dimensional feature vectors corresponding to the EMG feature, the EEG feature and the brain-muscle feature to obtain the overall feature. The overall feature is input into two fully connected layers and a Softmax layer connected in series in sequence to obtain the category to which the force-sensitive state of the upper limb movement belongs. The EEG information, the EMG information and the brain-muscle information are fully extracted and fused through the designed three-branch network structure to classify the force-sensitive state of the upper limb movement, thereby effectively improving the accuracy and stability of the recognition of the force-sensitive state of the upper limb movement. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0039] Figure 1 This is an overall framework diagram of the brain-myoelectricity sensitivity state recognition model of the upper limb movement force-sensitive state recognition method and system based on brain-myoelectricity fusion of the present invention.

[0040] Figure 2 This is a flowchart of the steps of a method for identifying the force-sensitive state of upper limb movements based on brain and myoelectricity fusion of the present invention.

[0041] Figure 3 This is a timing diagram of a single force-sensitive action experiment in a method for identifying the force-sensitive state of upper limb movements based on brain and myoelectricity fusion and a system of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0043] Reference Figure 2 As shown, the first embodiment of the present invention provides a method for identifying the force-sensitive state of upper limb movements based on brain and myoelectricity fusion, comprising the following steps:

[0044] Step S1: Obtaining original EEG signals and original EMG signals under different upper limb force-sensitive states, and recording corresponding force-sensitive state data, constructing a data set, and dividing the data set into a training set and a test set; Step S2: Preprocessing the original EEG signals and original EMG signals to obtain preprocessed EEG signals and EMG signals;

[0045] In this embodiment, preferably, the preprocessing of the original EEG signal and the original EMG signal to obtain the preprocessed EEG signal and EMG signal includes:

[0046] The original EEG signal was processed with a 0.5Hz high-pass filter to eliminate the low-frequency components and retain the high-frequency parts; the original EEG signal was filtered out of 50Hz power frequency interference to ensure that the signal quality is not affected by the grid frequency, and the baseline interference was removed to eliminate the influence of long-term trends, making the signal more stable and reliable. The signal segment corresponding to the force-sensitive state of the upper limb movement was extracted, and the data of each channel was normalized to obtain the preprocessed EEG signal.

[0047] The original electromyographic signal is processed by high-pass filtering, retaining the signal above the 20Hz frequency band, filtering out the 50Hz power frequency interference, removing the low-frequency noise and baseline drift, enhancing the high-frequency component of the signal, improving the information content of the original electromyographic signal, extracting the signal segment corresponding to the force-sensitive state of the upper limb movement, and obtaining the preprocessed electromyographic signal.

[0048] Step S3: resampling the preprocessed EEG signal, calculating the mutual information and similarity between the preprocessed EMG signal and the resampled EEG signal based on the preprocessed EMG signal and the resampled EEG signal, and concatenating the mutual information and similarity between the preprocessed EMG signal and the resampled EEG signal to form brain-muscle information;

[0049] In this embodiment, preferably, the mutual information and similarity between the preprocessed electromyographic signal and the resampled electroencephalogram signal are calculated. In this embodiment, the similarity between the preprocessed electromyographic signal and the resampled electroencephalogram signal is calculated by calculating one or more of the Pearson correlation coefficient, Euclidean distance, Manhattan distance, and cosine similarity.

[0050] Specifically:

[0051] The mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity between the preprocessed EMG signal and the resampled EEG signal were calculated to obtain the mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity between the preprocessed EMG signal and the resampled EEG signal;

[0052] In this embodiment, the resampled EEG signal has 32 channels, and the preprocessed EMG signal has 8 channels. The mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity between the preprocessed EMG signal and the resampled EEG signal are calculated to obtain an 8×32 mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity matrix. The five 8×32 matrices are spliced ​​to obtain brain-muscle information.

[0053] The calculation formula of mutual information is:

[0054]

[0055] Where MI(A;B) represents the mutual information between the preprocessed EMG signal and the resampled EEG signal; N is the total number of preprocessed EMG signal channels, M is the total number of resampled EEG signal channels, i∈(1,N), j∈(1,M);

[0056] x i represents the myoelectric sequence of the i-th channel of the preprocessed myoelectric signal, wherein each element in the myoelectric sequence represents the amplitude corresponding to different time points in the i-th channel of the preprocessed myoelectric signal;

[0057] y i represents the EEG sequence of the j-th channel of the resampled EEG signal, where each element in the EEG sequence represents the amplitude corresponding to different time points in the j-th channel of the resampled EEG signal;

[0058] p(x i ,y j ) represents the joint probability distribution of the myoelectric sequence of the i-th channel of the preprocessed EMG signal and the EEG sequence of the j-th channel of the resampled EEG signal;

[0059] p(x i ) represents the marginal probability distribution of the myoelectric sequence of the i-th channel of the preprocessed myoelectric signal, p(y j ) represents the marginal probability distribution of the EEG sequence of the jth channel of the resampled EEG signal.

[0060] The formula for calculating the Pearson correlation coefficient is:

[0061]

[0062] Where r represents the Pearson correlation coefficient between the i-th channel of the preprocessed EMG signal and the j-th channel of the resampled EEG signal;

[0063] n represents the total number of time points of the i-th channel of the preprocessed EMG signal or the j-th channel of the resampled EEG signal, and a represents the a-th time point;

[0064] X a Y represents the amplitude of the a-th time point in the i-th channel of the preprocessed electromyographic signal, a represents the amplitude corresponding to the ath time point in the jth channel of the resampled EEG signal;

[0065] It represents the average value of the amplitude of the i-th channel at each time point after preprocessing. Represents the average value of the amplitude corresponding to each time point of the jth channel of the resampled EEG signal.

[0066] The formula for calculating Euclidean distance is:

[0067]

[0068] Where d(i,j) represents the Euclidean distance between the i-th channel of the preprocessed EMG signal and the j-th channel of the resampled EEG signal;

[0069] The formula for calculating Manhattan distance is:

[0070]

[0071] Where t represents the Manhattan distance between the i-th channel of the preprocessed EMG signal and the j-th channel of the resampled EEG signal;

[0072] The formula for calculating cosine similarity is:

[0073]

[0074] Among them, Cosine(i,j) represents the cosine similarity between the i-th channel of the preprocessed EMG signal and the j-th channel of the resampled EEG signal;

[0075] A single similarity metric may be affected by specific data distribution or algorithms, leading to evaluation deviations. By combining multiple similarity metrics such as Pearson correlation coefficient, Euclidean distance, Manhattan distance, and cosine similarity, the error caused by a single metric can be reduced. It can also better capture the relationship between the resampled EEG signals and the preprocessed EMG signals, making the combined brain-muscle information more comprehensive.

[0076] Step S4: Construct a brain and muscle electrosensitivity state recognition model, such as Figure 1 As shown, Figure 1 This is the overall framework diagram of the brain and muscle electrosensitivity state recognition model, which includes: a three-branch feature extraction module and a recognition module;

[0077] Step S5: inputting the preprocessed electromyographic signal, the preprocessed electroencephalographic signal and the brain-muscle information into the three-branch feature extraction module, and using the three-branch feature extraction module to extract electromyographic features, electroencephalographic features and brain-muscle features;

[0078] In this embodiment, preferably, the preprocessed electromyographic signal is input into the three-branch feature extraction module, and the electromyographic feature is extracted using the three-branch feature extraction module, including:

[0079] The preprocessed EMG signal is input into the 2D convolution module to extract features from the preprocessed EMG signal to obtain global channel features;

[0080] The preprocessed EMG signals are divided into four groups, each containing two adjacent channels. The four groups of EMG signals are input into the 2D convolution module respectively, and the temporal and spatial information of each group of EMG signals are extracted to obtain four groups of local channel features.

[0081] The four groups of local channel features are spliced ​​and fused with the global channel features to obtain the electromyographic features.

[0082] Global channel features capture the overall signal characteristics, while local channel features can more finely express the local structure and dynamic changes corresponding to each channel. Through splicing and fusion, global and local information are comprehensively utilized to make the feature representation richer and more comprehensive.

[0083] In this embodiment, preferably, the brain-muscle information is input into the three-branch feature extraction module, and the brain-muscle features are extracted using the three-branch feature extraction module, including: inputting the brain-muscle information into the 2D convolution module, extracting features from the brain-muscle information, and obtaining brain-muscle features.

[0084] The 2D convolution module can effectively identify and capture features at different scales. It extracts features by sliding a convolution kernel across the entire input data. The parameters within the kernel are shared across different locations in the input data. This parameter sharing reduces the number of model parameters, helps mitigate overfitting, and more effectively extracts brain-muscle features.

[0085] In this embodiment, preferably, extracting EEG features based on the preprocessed EEG signal includes:

[0086] The preprocessed EEG signal passes through the 2D convolution module and the batch normalization layer in sequence to extract the time information of the EEG signal. The extracted time information of the EEG signal passes through the depthwise separable convolution module, the activation function layer and the batch normalization layer in sequence to obtain the fusion characteristics of the time information and spatial information of the EEG signal.

[0087] The 2D convolution module can effectively extract temporal information from the preprocessed EEG signals and can more fully capture the dynamic characteristics of the signal changing over time. The depthwise separable convolution structure allows channel and spatial information to be processed separately during the convolution process and then fused together. This method can effectively extract and fuse the temporal dynamics and spatial structures in the preprocessed EEG signals.

[0088] The batch normalization layer is also included after the 2D convolution module, which helps to maintain the dynamic range of the signal and helps the time series information to be better utilized and integrated;

[0089] The depth-wise separable convolution module also includes an activation function layer and a batch normalization layer. The activation function layer can further integrate the features of temporal information and spatial information after the depth-wise separable convolution, while batch normalization normalizes and accelerates convergence by standardizing the input of each feature channel.

[0090] Step S6: Input the myoelectric features, EEG features, and brain-muscle features into the recognition module to obtain the category of the upper limb movement force-sensitive state, including:

[0091] Flatten the EMG features, EEG features and brain-muscle features into one-dimensional feature vectors, and perform feature splicing on the one-dimensional feature vectors corresponding to the EMG features, EEG features and brain-muscle features to obtain the overall features;

[0092] The overall features are input into two serially connected fully connected layers and Softmax layers to classify the upper limb force sensitivity state and obtain the category to which the upper limb force sensitivity state belongs;

[0093] Step S7: Input the preprocessed EMG signals, preprocessed EEG signals, and brain-muscle information of the training set into the brain-muscle electrosensitivity state recognition model to obtain the probability of the upper limb movement force sensitivity state category in each training task. During training, the cross-entropy loss function is used to minimize the difference between the classification result and the true label. The Adam optimizer is used to optimize the cross-entropy loss function and update the learning rate.

[0094] In this embodiment, the calculation formula of the cross entropy loss function is preferably:

[0095]

[0096] Among them, H represents the number of samples, C represents the number of upper limb movement force sensitivity state categories;

[0097] If the classification result is the same as the true label, then f hc is 1, otherwise 0;

[0098] p hc It represents the probability that the hth sample belongs to the cth upper limb movement force-sensitive state category.

[0099] By using the cross-entropy loss function to minimize the difference between the classification results and the corresponding force-sensitivity state data, the brain-myography electrosensitivity state recognition model can learn parameters that make the predicted results as close as possible to the actual target, and can effectively reflect the accuracy of upper limb movement force-sensitivity state classification.

[0100] Step S8: Using the classification accuracy rate on the test set, the performance of the brain and muscle electrosensitivity state recognition model is evaluated.

[0101] The second embodiment is a process of training the brain and muscle electrosensitivity state recognition model, and comparing the training results with single EEG signals and EMG signals to further verify the beneficial effects of the present invention.

[0102] like Figure 3 As shown, Figure 3 This is a timing diagram of a single force-sensitive action experiment in a method for identifying the force-sensitive state of upper limb movements based on brain-myoelectric fusion and the system of the present invention. According to the designed experimental paradigm, the subject prepares within 0-2 seconds, and the screen will prompt "Relax, prepare for the next action." The screen will prompt the performed action within 2-3.5 seconds and then disappear. After seeing the prompt on the screen to perform the action, the subject quickly performs the action and reaches the corresponding force level before 5 seconds, maintains it for 3 seconds, and then rests for 5 seconds to prepare for the next experiment.

[0103] There were two subjects, aged 22-30 years old, and four types of upper limb motor force sensitivity states were selected, including: normal state, 10% of the maximum voluntary contraction force, 20% of the maximum voluntary contraction force, and 40% of the maximum voluntary contraction force. The maximum voluntary contraction force refers to the maximum contraction force that the muscle can produce under voluntary control. The normal state refers to the state of not actively exerting force and performing normal upper limb movements. The selected upper limb movement tasks include five upper limb movements: fist clenching, palm extension, pinching, two-finger press, and thumb press.

[0104] The original EEG signals are mainly recorded from 32 electrode channels and sampled at 128 Hz. The original EMG signals are mainly recorded from 8 electrode channels and sampled at 200 Hz. The EEG signals and EMG signals collected on site are used to form a data set to train the brain-myoelectric sensitivity state recognition model.

[0105] The data set is divided into a training set and a test set according to a preset ratio, and the EEG signal and EMG signal data of the training set are input into the brain and myoelectricity sensitivity state recognition model to obtain the probability of the category to which the action force sensitivity state belongs in each task;

[0106] When training the electromyography state recognition model, the cross-entropy loss function is used to minimize the difference between the classification result and the corresponding true label, the Adam optimizer is used to optimize the loss function and update the learning rate, and finally the classification accuracy is used on the test set to evaluate the performance of the electromyography state recognition model.

[0107] The Adam optimizer was used to minimize the categorical cross entropy loss function, with a learning rate of 0.001, and training for 150 epochs. After five epochs of training and no decrease in loss, the learning rate was reduced by 80%, the dropout value was set to 0.5, and early stopping was used during training.

[0108] The upper limb movement force-sensitive state classification results of the upper limb movement force-sensitive state recognition method based on brain and myoelectric fusion are compared with the accuracy of the upper limb movement force-sensitive state classification results of single EEG signals, EMG signals, and brain and myoelectric splicing fusion. The specific results are shown in Table 1:

[0109] Table 1

[0110]

[0111] It can be seen from Table 1 that the method for identifying the force-sensitive state of upper limb movements based on brain and myoelectricity fusion provided by the present invention has a more accurate classification performance, and achieves more precise identification of the force-sensitive state of upper limb movements.

[0112] The third embodiment of the present invention provides a system for identifying the force-sensitive state of upper limb movements based on brain-myoelectric fusion, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned method for identifying the force-sensitive state of upper limb movements based on brain-myoelectric fusion are implemented.

[0113] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for identifying the force-sensitive state of upper limb movements based on brain and myoelectricity fusion, characterized in that: The following steps are involved: Obtain the original EEG signals and original EMG signals under different upper limb force-sensitive states, record the corresponding force-sensitive state data, construct a data set, and divide the data set into a training set and a test set; Preprocessing the original EEG signal and the original EMG signal to obtain preprocessed EEG signal and EMG signal; Resampling the preprocessed EEG signal, calculating the mutual information and similarity between the preprocessed EMG signal and the resampled EEG signal based on the preprocessed EMG signal and the resampled EEG signal, and concatenating the mutual information and similarity between the preprocessed EMG signal and the resampled EEG signal to form brain-muscle information; Constructing a brain and muscle electrosensitivity state recognition model, the brain and muscle electrosensitivity state recognition model includes: a three-branch feature extraction module and a recognition module; Inputting the preprocessed electromyographic signal, the preprocessed electroencephalographic signal and the brain-muscle information into the three-branch feature extraction module, and using the three-branch feature extraction module to extract electromyographic features, electroencephalographic features and brain-muscle features; The myoelectric features, EEG features, and brain-muscle features are input into the recognition module to obtain the category of the upper limb movement force sensitivity state, including: Flatten the EMG features, EEG features and brain-muscle features into one-dimensional feature vectors, and perform feature splicing on the one-dimensional feature vectors corresponding to the EMG features, EEG features and brain-muscle features to obtain the overall features; The overall features are input into two serially connected fully connected layers and Softmax layers to classify the upper limb force sensitivity state and obtain the category to which the upper limb force sensitivity state belongs; The preprocessed EMG signals, preprocessed EEG signals, and brain-muscle information of the training set were input into the brain-muscle electrosensitivity state recognition model to obtain the probability of the upper limb force sensitivity state category in each training task. During training, the cross-entropy loss function was used to minimize the difference between the classification results and the true labels. The Adam optimizer was used to optimize the cross-entropy loss function and update the learning rate. The performance of the brain and muscle electrosensitivity state recognition model was evaluated using the classification accuracy on the test set.

2. The method for recognizing upper limb force-sensitive states based on brain-myoelectric fusion according to claim 1, characterized in that: The original EEG signal is preprocessed to obtain a preprocessed EEG signal, including: After the original EEG signal was processed by 0.5Hz high-pass filtering, 50Hz power frequency interference was filtered out, and baseline interference was removed, the signal segment corresponding to the force-sensitive state of the upper limb movement was extracted, and the data of each channel was normalized to obtain the preprocessed EEG signal.

3. The method for recognizing upper limb force-sensitive states based on brain-myoelectric fusion according to claim 1, characterized in that: The original electromyographic signal is preprocessed to obtain a preprocessed electromyographic signal, including: The original electromyographic signal was processed by high-pass filtering, the signal above the 20 Hz frequency band was retained, the 50 Hz power frequency interference was filtered out, and the signal segment corresponding to the force-sensitive state of the upper limb movement was extracted to obtain the preprocessed electromyographic signal.

4. The method for recognizing upper limb force-sensitive states based on brain-myoelectric fusion according to claim 1, characterized in that: The mutual information and similarity between the preprocessed electromyographic signal and the resampled electroencephalographic signal are calculated as follows: The mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity between the preprocessed EMG signals and the resampled EEG signals were calculated to obtain the mutual information, Pearson correlation coefficient, Euclidean distance, Manhattan distance and cosine similarity between the preprocessed EMG signals and the resampled EEG signals.

5. The method for recognizing upper limb force-sensitive states based on brain-myoelectric fusion according to claim 1, characterized in that: Inputting the preprocessed electromyographic signal into the three-branch feature extraction module, and extracting electromyographic features using the three-branch feature extraction module, including: The preprocessed EMG signal is input into the 2D convolution module to extract features from the preprocessed EMG signal to obtain global channel features; The preprocessed EMG signals are divided into four groups, each containing two adjacent channels. The four groups of EMG signals are input into the 2D convolution module respectively, and the temporal and spatial information of each group of EMG signals are extracted to obtain four groups of local channel features. The four groups of local channel features are spliced ​​and fused with the global channel features to obtain the electromyographic features.

6. The method for recognizing upper limb force-sensitive states based on brain-myoelectric fusion according to claim 1, characterized in that: Inputting brain-muscle information into the three-branch feature extraction module, and using the three-branch feature extraction module to extract brain-muscle features, including: inputting brain-muscle information into a 2D convolution module, extracting features from the brain-muscle information, and obtaining brain-muscle features.

7. The method for recognizing upper limb force-sensitive states based on brain-myoelectric fusion according to claim 1, characterized in that: Inputting the preprocessed EEG signal into the three-branch feature extraction module, and extracting EEG features using the three-branch feature extraction module, including: The preprocessed EEG signal passes through the 2D convolution module and the batch normalization layer in sequence to extract the time information of the EEG signal. The extracted time information of the EEG signal passes through the depthwise separable convolution module, the activation function layer and the batch normalization layer in sequence to obtain the fusion characteristics of the time information and spatial information of the EEG signal.

8. The method for recognizing upper limb force-sensitive states based on brain-myoelectric fusion according to claim 1, characterized in that: The calculation formula of the cross entropy loss function is: Among them, H represents the number of samples, C represents the number of upper limb movement force sensitivity state categories; If the classification result is the same as the true label, then f hc is 1, otherwise 0; p hc It represents the probability that the hth sample belongs to the cth upper limb movement force-sensitive state category.

9. A system for recognizing the force-sensitive state of upper limb movements based on brain and myoelectricity fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for identifying the force-sensitive state of upper limb movements based on brain-myoelectric fusion as described in any one of claims 1 to 8 are implemented.

Images