A motor intention decoding method based on multi-dimensional electroencephalogram feature fusion

Abstract

The application relates to a motor intention decoding method based on multi-dimensional electroencephalogram feature fusion. The dependence on a single motor preparation potential feature has seriously restricted the performance improvement of the motor intention decoding algorithm. In view of the motor intention decoding problem under the condition of disappearance of the motor preparation potential, the application provides a decoding method based on multi-dimensional electroencephalogram feature fusion. The time-frequency, brain network and cross-frequency coupling features of the motor preparation process under the conditions of normal and abnormal motor preparation potential are compared and analyzed, and the electroencephalogram features that can be used for motor intention decoding are determined. The motor intention is calculated in combination with the above features. The method can improve the accuracy of motor intention decoding under the condition of disappearance of the motor preparation potential, and provides a new method for motor intention decoding research.

Description

Technical Field

[0001] This invention relates to a method for decoding motor intentions based on multidimensional EEG feature fusion, belonging to the field of information processing technology; in particular, it relates to a method for decoding motor intentions that can improve decoding accuracy when motor preparation potentials disappear. Background Technology

[0002] Readiness potentials (RP) are slowly rising negative electrical potentials that appear in the central frontal lobe of the scalp before motor execution. They are a type of neural information that can be detected by electrodes, and researchers believe this information reflects the brain's motor planning and preparation processes before motor execution. Motor intention is the cognitive activity of the brain before motor execution; the process of extracting motor intention from neural information is called motor intention decoding.

[0003] Motor readiness potentials reflect the psychological and physiological state of a subject and are currently the primary neural information used for motor intention decoding. Motor intention decoding plays a crucial role in brain-computer interfaces and the diagnosis of neurological diseases. However, motor readiness potentials are weak and difficult to extract, and are significantly influenced by psychological and physiological factors. Previous studies have found that some subjects exhibited the disappearance of motor readiness potentials, which reduces the accuracy of motor intention decoding. This low accuracy limits its application in clinical medicine and significantly impacts the practical application of motor intention analysis.

[0004] With the rapid development and widespread application of deep learning technology, Convolutional Neural Networks (CNN) have also been used for decoding motion intentions. Mammone et al. used CNN to realize the recognition of motion intentions based on the time-frequency map of the posterior cortical EEG signal, achieving an average recognition accuracy of 90.3% for two classifications

[56] . Although the motion intention decoding algorithm based on CNN has achieved the highest accuracy, there are still problems such as slow model training speed and difficulty in interpreting the extracted features. Summary of the Invention

[0005] The purpose of this invention is to provide a motion intention decoding method based on multidimensional EEG feature fusion, in order to solve the problems of low accuracy in motion intention decoding when the motion preparation potential disappears, or slow training speed in deep learning-based decoding.

[0006] To achieve the above objectives, the present invention includes:

[0007] The present invention provides a technical solution for a motion intention decoding method based on multidimensional EEG feature fusion, comprising the following steps:

[0008] 1) Extract low-frequency features (0.1–4 Hz), high-frequency features (10–30 Hz), brain network features, and cross-frequency coupling features from EEG data; the brain network features reflect the functional connectivity strength between brain regions in the subject's resting state or during the exercise preparation phase, and the cross-frequency coupling features reflect the coupling value between low-frequency phase and high-frequency energy.

[0009] 2) The low-frequency features, high-frequency features, brain network features, and cross-frequency coupling features are fused to obtain the fused features;

[0010] 3) Input the fused features into the trained classifier to obtain the motion intent decoding result.

[0011] This invention attempts to provide a solution to improve the accuracy of motor intention decoding when the motor preparation potential is absent. Addressing the issue of decreased accuracy in motor intention decoding when the motor preparation potential is absent, this invention proposes a decoding algorithm based on multi-dimensional EEG feature fusion. Comparative analysis of EEG features under conditions of absent and normal motor preparation potentials shows that the absence of the motor preparation potential has little impact on the time-frequency, brain network, and cross-frequency coupling features of the motor preparation process. Therefore, a method based on time-frequency-cross-frequency-network feature extraction and fusion of the motor preparation process is proposed, effectively improving the decoding accuracy when the motor preparation potential is absent, with an average decoding accuracy exceeding 85%, which is superior to current decoding algorithms based on motor preparation potential features. This research provides a new method for decoding motor intention under abnormal motor preparation potential conditions.

[0012] Furthermore, the brain network features are extracted as follows: First, the average value of the functional connectivity matrix of all trials in the resting state and the exercise preparation phase of the corresponding subject is calculated to obtain the template of brain network connectivity in the two cases; during feature extraction in step 1), the correlation coefficient between the connectivity matrix with a set center frequency and its corresponding template is calculated for each trial to obtain the brain network features.

[0013] Furthermore, the templates for the exercise preparation phase and the resting state can be defined as follows:

[0014]

[0015] Where N represents the number of subjects, Let f be the average adjacency matrix of the k-th subject during the exercise preparation phase. Let f be the average adjacency matrix of the k-th subject in the resting state. and These represent the adjacency matrix templates for the motion preparation phase and the idle state, respectively, with a center frequency of f.

[0016] Furthermore, the correlation coefficient is the Pearson correlation coefficient.

[0017] Furthermore, for a trial, the adjacency matrix of the center frequency f is A, and its Pearson correlation coefficient with the corresponding template is:

[0018]

[0019] Where n represents the number of elements in the adjacency matrix, a i This represents the elements in the adjacency matrix A. Let b represent the mean of the adjacency matrix A. i This refers to elements within a template. This represents the mean of the elements in the template.

[0020] Furthermore, the set value is 8 to 30 Hz.

[0021] Furthermore, the low-frequency characteristics include four types: minimum peak value, slope, standard deviation, and average amplitude of the slow potential before motion.

[0022] Furthermore, filtered EEG signals are obtained from the EEG data of the corresponding trials using a finite unit impulse response filter, and then the average energy of each channel is calculated as the high-frequency feature.

[0023] Furthermore, the cross-frequency coupling characteristic is the cross-frequency coupling value of 2Hz phase and 10Hz energy for each channel signal.

[0024] Furthermore, the classifier is a support vector machine classifier, the kernel function is a linear kernel function, and the training and testing processes use five-fold cross-validation.

[0025] This chapter addresses the problem of low accuracy in decoding motor intentions due to abnormal motor preparation potential characteristics. It analyzes multi-dimensional EEG features during the motor preparation process, including time-frequency, brain network, and cross-frequency coupling, and proposes a motor intention decoding algorithm based on multi-dimensional EEG feature fusion. This algorithm achieves average recognition rates of 88.9% and 85.5% under normal and abnormal motor preparation potential conditions, respectively. Compared with a motor intention decoding algorithm based on motor preparation potential characteristics, the average accuracy rates for both tasks are improved by 7.8% and 8.8%, respectively, and the difference in recognition rates between the two tasks is reduced from 4.4% to 3.4%. This research provides a new method for decoding motor intentions under abnormal motor preparation potential conditions. Attached Figure Description

[0026] Figure 1 It is a preparatory potential for movement and its scalp topography;

[0027] Figure 2(a) shows the time-frequency characteristics analysis results of the Alpha band during the motion preparation process of Task 1;

[0028] Figure 2(b) shows the time-frequency characteristics analysis results of the Beta band during the motion preparation process of Task 1;

[0029] Figure 2(c) shows the time-frequency characteristics analysis results of the Alpha band during the motion preparation process of Task 2;

[0030] Figure 2(d) shows the time-frequency characteristics analysis results of the Beta band during the motion preparation process of Task 2;

[0031] Figure 3 This is a schematic diagram of the FIR filter structure used in the embodiment;

[0032] Figure 4 It is the COH functional connection matrix for the exercise preparation process;

[0033] Figure 5 This is a schematic diagram of brain network connections during the exercise preparation process;

[0034] Figure 6 This is a schematic diagram of brain network feature extraction;

[0035] Figure 7(a) Cross-frequency coupling characteristics of the motor preparation process in FCz channels of motor-related brain regions;

[0036] Figure 7(b) Cross-frequency coupling characteristics of the motor preparation process in the FC3 channel of motor-related brain regions;

[0037] Figure 7(c) Cross-frequency coupling characteristics of the motor preparation process in the FC4 channel of motor-related brain regions;

[0038] Figure 8 This is a schematic diagram of the motion intent decoding process based on motion preparation potential characteristics;

[0039] Figure 9 A flowchart illustrating a multidimensional EEG feature fusion and motion intent decoding method based on feature-stage fusion.

[0040] Figure 10 A flowchart illustrating a multidimensional EEG feature fusion and motion intent decoding method for decision-making stage;

[0041] Figure 11 Visualization results of motion preparation potential characteristics and multidimensional fusion characteristics;

[0042] Figure 12(a) Comparison of accuracy results between the motion intention decoding algorithm based on multidimensional EEG feature fusion and the decoding based on motion preparation potential features;

[0043] Figure 12(b) Comparison of true positive rate between the motor intention decoding algorithm based on multidimensional EEG feature fusion and the decoding based on motor preparation potential features;

[0044] Figure 12(c) Comparison of false positive rate between the motion intention decoding algorithm based on multidimensional EEG feature fusion and the decoding based on motion preparation potential features;

[0045] Figure 12(d) Comparison of the results of the motion intention decoding algorithm based on multidimensional EEG feature fusion and the decoding based on motion preparation potential features regarding the Kappa coefficient. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0047] The motor preparation potential is the pre-motor execution portion of the motor-related cortical potential (MRCP). Motor preparation potentials appear before voluntary arm movements, squatting, standing, finger movements, motor imagery, and voluntary walking, and their waveforms are as follows: Figure 1 As shown. The motion preparation potential is a response lock-in potential, which can be further divided into early motion preparation potential and late motion preparation potential. The classic interpretation of these two phases is as follows: the early motion preparation potential reflects the planning for the upcoming movement, while the late motion preparation potential reflects the preparation for movement after the movement decision. The motion preparation potential begins to appear about 1.5 seconds before movement, manifesting as a slowly rising negative potential. Its amplitude begins to increase sharply about 0.5 seconds before movement, reaching its peak during movement execution, and then gradually returns to near zero potential. The motion preparation potential is mainly distributed in the central region of the scalp, specifically manifesting as a negatively deflected potential.

[0048] This embodiment first introduces the extraction of several EEG features during the exercise preparation process.

[0049] Part 1: Extraction of EEG features during the exercise preparation process.

[0050] 1. Time-frequency characteristics of the exercise preparation process.

[0051] (1) Wavelet time-frequency analysis method.

[0052] The wavelet transform was originally proposed by the French engineer Morlet. This method boasts excellent time-frequency resolution and is widely used in the analysis of non-stationary signals. Compared to the short-time Fourier transform, the wavelet function has two variables: scaling and translation. It can perform scaling and translation transformations, thus simultaneously achieving analytical results in both the time and frequency domains. The formula for implementing the wavelet transform is as follows:

[0053] Assume the time series y(t)∈L 2 (R), whose wavelet transform is:

[0054]

[0055] In the formula, a is the scaling factor, representing the scale of the function; b is the time shift factor, reflecting the translation position of the wavelet function; W(a,b) are the coefficients of the wavelet transform, and its inverse transform is:

[0056]

[0057] (2) Results of time-frequency characteristic analysis.

[0058] Event-related desynchronization (ADR) in the contralateral sensorimotor cortex is considered a time-frequency EEG characteristic during motor imagery or execution. Current brain-computer interface (BCI) technologies based on motor imagery utilize this characteristic and have achieved good recognition results. Related research indicates that motor preparation and motor imagery processes share similar neural activity patterns and also exhibit ADR in the contralateral sensorimotor cortex. However, the ADR characteristic during motor preparation is weaker compared to that during motor imagery.

[0059] To investigate the time-frequency characteristics of EEG signals during the exercise preparation process, this section extracts EEG data from -3 to 1 second of the exercise process for wavelet time-frequency analysis. The results of the time-frequency analysis of the movement-related brain regions are as follows: Figures 2(a) to 2(d) As shown.

[0060] As shown in Figures 2(a) and 2(b), the energy of the Alpha and Beta bands in Task 1 decreased before the motion was executed. Similarly, similar phenomena can be observed in Figures 2(c) and 2(d). This result indicates that although the motion preparation potential features are missing during the motion preparation process, the energy features of the Alpha and Beta bands are only slightly affected. Therefore, the energy features of these two bands can be extracted to supplement the motion preparation potential features, which can help solve the problem of a significant decrease in the accuracy of motion intent decoding when the motion preparation potential is missing.

[0061] (3) Time-frequency feature extraction.

[0062] Wavelet time-frequency analysis results show that the energy of the pre-motor EEG signal decreases in the Alpha and Beta frequency bands. To extract the features of this frequency band, a bandpass filter needs to be designed for each trial of EEG data to obtain the filtered EEG signal. Currently, digital filters mainly include Infinite Impulse Response (IIR) filters and Finite Impulse Response (FIR) filters. IIR filters only consider amplitude characteristics and do not consider phase characteristics, while FIR filters can guarantee both amplitude and phase characteristics. Therefore, FIR filters are chosen for bandpass filtering of EEG signals. The design process of the FIR filter is as follows:

[0063] Let h(n) be the unit impulse response of the FIR filter, 0 ≤ n ≤ N-1, then the system function is:

[0064]

[0065] The structure of the filter is as follows: Figure 3 As shown. The input signal is x(n), and the filtered output signal is y(n). The difference form of the input and output signals is:

[0066]

[0067] The FIR filter achieves its function by designing a unit impulse response h(n). The linear phase condition of this filter is:

[0068]

[0069] For h(n) of length N, its transfer function is:

[0070] H(e jω )=H g (ω)e -jθ(ω) (6)

[0071] Among them, the amplitude characteristic is H g (ω), with phase characteristic θ(ω).

[0072] Let the FIR-filtered EEG signal be... BP =[e BP1 ,e BP2 ,e BP3 ,···,e BPm ]∈R m×n Here, m represents the number of channels, which is 62; n represents the number of sampling points. This chapter extracts data from the first 1.5 seconds of motion execution, where n is 375.

[0073] Then, the energy characteristics of each channel signal are calculated using the following formula:

[0074]

[0075] In the formula, var represents the variance. The extracted feature vector is then:

[0076]

[0077] 2. Brain network characteristics during the exercise preparation process.

[0078] (1) Weighted phase lag index method.

[0079] The Weighted Phase-lag Index (wPLI), proposed by Vinck et al. in 2011, is an index for assessing the phase synchronization between neural signals, reflecting the information exchange ability between neural clusters. It is developed based on the PLI method and overcomes the shortcomings of PLI, such as low sensitivity to phase outliers and the degree of distribution concentration. Furthermore, this method avoids the influence of volume conduction effects present when neural electrical signals are transmitted to the scalp. Its calculation formula is as follows:

[0080]

[0081] In the formula, C abt Let imag represent the cross spectrum of the signals from electrodes a and b at time t, imag represent the imaginary part of the cross spectrum, and sgn represent the sign (positive value is 1, negative value is -1).

[0082] (2) Results of weighted phase lag network analysis.

[0083] The wPLI functional connectivity matrices for the Delta, Theta, Alpha, Beta, and Gamma bands in the three cases of Task 1, Task 2, and resting state are as follows: Figure 4 As shown. Overall, across the five frequency bands mentioned above, the functional connectivity matrix structures of Task 1 and Task 2 are similar, but the structures of these two tasks' functional connectivity matrices differ significantly from those of the resting-state functional connectivity matrix. Specifically, in the Delta band, the connectivity strength of the resting-state functional connectivity matrix is ​​higher than that of Task 1 and Task 2; in the Theta and Gamma bands, the connectivity strength of the upper-left corner functional connectivity matrix of Task 1 and Task 2 is significantly higher than that of the resting-state result, while the connectivity strength of the lower-right corner functional connectivity matrix of the resting-state is significantly higher than that of Task 1 and Task 2; in the Alpha and Beta bands, the connectivity strength of the functional connectivity matrices of Task 1 and Task 2 is significantly higher than that of the resting-state task.

[0084] To further analyze the changes in scalp-brain network connectivity under the three tasks, edges with the highest average connectivity strength in each group under the three conditions were retained. This process effectively reduces the number of spurious connections. A scalp topology map of the brain network during the motor preparation process was also constructed. Figure 5 As shown. In the Alpha and Beta bands, the number of connections is similar across the three scenarios. However, for Task 1 and Task 2, the connections are mainly distributed in the left sensorimotor area and the central frontal lobe, while the resting-state connections are located in the parietal and occipital regions. In the Delta band, the resting-state connections are significantly more numerous than those for Task 1 and Task 2, mainly distributed in the parietal and occipital regions of the scalp. In the Theta and Gamma bands, the resting-state connections are significantly fewer than those for Task 1 and Task 2, mainly distributed in the peripheral regions of the scalp. The brain network features of the motor preparation process in the Alpha and Beta bands differ most from those of the resting-state brain network; therefore, the brain network features of these two bands were selected as features for motor intention decoding.

[0085] Network features can characterize the information interaction capabilities between nodes. Since the introduction of network analysis methods into the field of EEG analysis, they have been widely used. Researchers typically use brain network features to analyze the working mechanisms of the brain, but rarely apply them to the decoding of EEG signals. In 2019, Professor Xu Peng's team in China used the discriminative spatial network pattern method to achieve classification of decision-making tasks in a single trial using brain network features in the decision-making process, achieving an average classification accuracy of 90%. This research result indicates that the information contained in brain network features can also be used to decode EEG features. Although there is currently limited research on neural decoding based on EEG features, brain network features still have the potential to become effective neural decoding features.

[0086] (3) Brain network feature extraction.

[0087] The analysis results in the previous section showed significant differences in wPLI brain network features during the alpha and beta bands of motor preparation and the resting state, which could potentially serve as EEG features for motor intention decoding and improve decoding performance. Since the stability of brain network attribute features in a single trial is poor, a template matching method was considered to extract brain network features. The basic idea of ​​this method is: because there are significant differences between the brain network patterns during motor preparation and those in the resting state, for trial k, the brain network pattern of that trial can be compared with a template to obtain the Pearson correlation coefficient. This correlation coefficient can then be used as the EEG feature for motor intention decoding. The feature extraction process is as follows: Figure 6 As shown.

[0088] The specific brain network feature extraction process is as follows:

[0089] For each trial, 23 adjacency matrices were calculated for the center frequencies f = 8, 9, 10, ..., 30 Hz. Then, for each task class, the average adjacency matrix of the adjacency matrices for each frequency across all trials for all subjects was calculated and used as a template for that task class. The templates for the exercise preparation task and the resting state can be defined as follows:

[0090]

[0091] In the formula, N represents the number of subjects. Let f be the average adjacency matrix of the k-th subject's exercise preparation task center frequency. Let f be the average adjacency matrix of the k-th subject in the resting state. and These represent the adjacency matrix templates for the ready state and the idle state, respectively, with a center frequency of f.

[0092] For a single trial, assuming the adjacency matrix of the center frequency f is A, its Pearson correlation coefficient with the template is:

[0093]

[0094] In the formula, n represents the number of elements in the adjacency matrix, and a i This represents the elements in the adjacency matrix A. Let b represent the mean of the adjacency matrix A. i This refers to elements within a template. This represents the mean of the elements in the template.

[0095] Finally, for each trial, the correlation coefficient between that trial and the motor task and the resting-state template at each center frequency point was calculated. The final extracted brain network features are as follows:

[0096]

[0097] 3. Cross-frequency coupling characteristics of the motion preparation process.

[0098] (1) Cross-frequency coupling analysis method.

[0099] Cross-frequency coupling is a newly proposed method for analyzing electroencephalogram (EEG) signals in recent years. Low-frequency brain activity usually reflects the dynamic communication process between brain regions, while high-frequency activity reflects the information processing capacity of local areas of the cerebral cortex. Therefore, cross-frequency coupling, as a mechanism, transfers information from large-scale brain networks operating on a time scale to rapid, local cortical processing to achieve efficient computation. Cross-frequency coupling characterizes the modulation relationship between the phase and amplitude of EEG signals in different frequency bands. It can usually be subdivided into phase-phase coupling, phase-amplitude coupling, and amplitude-amplitude coupling. Currently, neuroinformatics research based on EEG data mainly uses phase-amplitude coupling, specifically the coupling of low-frequency phase with high-frequency amplitude.

[0100] The Modulation Index (MI) is a cross-frequency coupling calculation index proposed by Canolty et al. in 2006, and it is currently the most widely used index. Its calculation formula is as follows:

[0101]

[0102] In the formula, n represents the total number of time points in the EEG data, t represents the time point, and a t φ represents the energy of high-frequency brainwaves at time t. t This represents the phase of low-frequency EEG at time t.

[0103] (2) Results of cross-frequency coupling characteristic analysis.

[0104] The cross-frequency coupling characteristics of the motor preparation process are shown in Figures 7(a), 7(b), and 7(c). The results for the FCz, FC3, and FC4 channels in motor-related brain regions are plotted. The figures show that, compared to the resting state, the cross-frequency coupling values ​​for the 2Hz phase and 10Hz amplitude are lower during the motor preparation process. This feature is mainly distributed in motor-related brain regions, and the cross-frequency coupling results for Task 1 and Task 2 are similar in this frequency band. These results indicate that the cross-frequency coupling characteristics of the motor preparation process can be used as a feature for decoding motor intent. Similarly, this feature has a relatively small impact on the disappearance of the motor preparation potential.

[0105] (3) Cross-frequency coupling feature extraction.

[0106] Analysis of cross-frequency coupling revealed lower cross-frequency coupling values ​​for 2Hz phase and 10Hz amplitude during motion preparation compared to the resting state. Therefore, the modulation index for 2Hz phase and 10Hz amplitude of each channel was calculated and used as a feature for motion intent decoding. The feature extraction process is as follows:

[0107] For a single trial of EEG data, the FIR filter used in section "(3) Time-Frequency Feature Extraction" is first applied to the EEG data of each channel using bandpass filtering at 1 to 3 Hz and 9 to 11 Hz. It is assumed that the EEG data of one channel after filtering are E... P (t) and E A (t).

[0108] E can be extracted using the Hilbert transform. P The phase sequence of (t) is shown in the following equation:

[0109]

[0110] Similarly, E can be extracted based on the Hilbert transform. A The magnitude envelope sequence of (t) is shown in the following equation:

[0111]

[0112] Next, the phase sequence φ P (t) is divided into 18 phase segments of equal size, each phase segment being 20°. The average amplitude A distributed on each phase segment is calculated. A The average amplitude distributed over the j-th phase segment is defined as:

[0113] The modulation index is calculated using the standardized entropy H, where H is defined as follows:

[0114]

[0115] In the formula, N represents the number of phase segments, p j Defined as:

[0116]

[0117] Finally, for each trial, the extracted cross-frequency coupling features are as follows:

[0118] F CFC =[MI1,MI2,···,MI ch (18)

[0119] In the formula, ch represents the number of channels for EEG data.

[0120] The second part is the decoding based on the characteristics of the motion preparation potential.

[0121] Before introducing the motion intention decoding method based on multidimensional EEG feature fusion of the present invention, the decoding method based on motion preparation potential features will be introduced first.

[0122] Currently, commonly used motion preparation potential features for motion intent decoding include minimum peak value, slope, standard deviation, and average amplitude. In this embodiment, the classifier is a support vector machine.

[0123] 1. Support vector machine classifier for decoding motion preparation potential features.

[0124] The principle of Support Vector Machine (SVM) is to map input features to a high-dimensional feature space through a pre-selected non-linear relationship, and then find the optimal classification hyperplane in this space. This hyperplane maximizes the margin between classes, and the classifier has good generalization ability and significant advantages in few-shot learning. The specific calculation process is as follows:

[0125] (1) The feature set is linearly separable

[0126] Suppose x = {x1, x2, ..., x...} N Let} be the feature set, and the corresponding category labels be y = {y1, y2, ..., y3}. N}, x1∈R l y1∈{-1,1}. Since this feature set is linearly separable, a hyperplane can be found that separates the feature set, as shown in the following equation:

[0127] x T w+b=0 (19)

[0128] However, there is more than one hyperplane that can satisfy the above requirements. In order to find the optimal hyperplane, which maximizes the Euclidean distance between the feature point closest to the hyperplane and the hyperplane, the hyperplane that satisfies the above requirements should satisfy the following constraints:

[0129]

[0130] Solving this constraint is a quadratic programming problem, which can be solved using the Lagrange multiplier method. The corresponding Lagrange function is:

[0131]

[0132] Taking the derivatives with respect to w and b, we get:

[0133]

[0134] Substituting it into the Lagrange function, we get:

[0135]

[0136] Thus, there exists a unique optimal solution, let a i * Let the optimal solution be:

[0137]

[0138] According to constraint a i [y i (w T x i +b)-1]=0, we can calculate b * The final optimal classification function is:

[0139]

[0140] (2) The feature set is linearly inseparable

[0141] If the feature set is linearly inseparable, there is no hyperplane that can separate the feature set; therefore, a non-negative slack variable ξ needs to be introduced. i (ξ i For any ≥0, i=1,…,n), the classification hyperplane satisfies:

[0142] y i (w T x i +b)≥1-ξ i (26)

[0143] Introducing the objective function:

[0144]

[0145] In the formula, C is the penalty factor, and is a positive constant. The constraint condition is:

[0146]

[0147] For this quadratic programming problem, the constructed Lagrange function is:

[0148]

[0149] Let w, b and ξ i By taking the derivative and substituting it into the Lagrange function, we can obtain:

[0150]

[0151] For the linearly inseparable case, a nonlinear mapping is needed to map the inseparable samples to a high-dimensional space to transform it into a linear case. This inner product operation is the kernel function.

[0152] (3) Commonly used kernel functions

[0153] The following are some commonly used kernel functions for support vector machines:

[0154] (a) Linear kernel function: K(x,x′)=(x·x′)

[0155] (b) Polynomial kernel function: K(x,x′)=(γx·x′+c) d

[0156] (c) Gaussian radial basis kernel function: K(x,x′)=exp(-||xx′|| 2 / σ 2 )

[0157] (d) Sigmoid kernel function: K(x,x) i )=tanh((x·x i )+c),c>0.

[0158] 2. Evaluation indicators for classification effectiveness.

[0159] The evaluation of motion intent decoding results requires some objective indicators. Currently, there are many available evaluation indicators. This embodiment mainly selects four indicators: accuracy (Accuracy, Acc), true positive rate (TPR), false positive rate (FPR), and Kappa coefficient. These four indicators can comprehensively evaluate the decoding effect and are also commonly used indicators for evaluating motion intent decoding effect. Each indicator will be introduced separately below.

[0160] First, let's introduce the confusion matrix. For a binary classification result, the possible outcomes are shown in Table 1.

[0161] Table 1. Confusion matrix of binary classification results

[0162]

[0163] The formula for calculating the accuracy rate is as follows:

[0164]

[0165] The formula for calculating the true positive rate is:

[0166]

[0167] The formula for calculating the false positive rate is:

[0168]

[0169] The Kappa coefficient can be used to measure the consistency between the model's predictions and the actual classification results, and can solve the problem of imbalanced samples. Its calculation formula is as follows:

[0170]

[0171] In the formula, p0 is the accuracy rate, p e The sum of the products of the actual and predicted quantities for each category is divided by the square of the total number of samples.

[0172] 3. A motion intent decoding method based on motion preparation potential characteristics.

[0173] The algorithm based on motion preparation potential characteristics in this embodiment has the following method flow: Figure 8 As shown, it mainly includes four parts: filtering, feature extraction, feature transformation, and classification.

[0174] A 0.1–4 Hz bandpass filter was selected, which can filter out high-frequency signals while retaining slow-wave characteristics. Feature extraction utilized traditional motion intent decoding slow-potential features, specifically the minimum peak value, slope, standard deviation, and average amplitude of the slow potential before motion. Therefore, each channel yielded four features, resulting in 248 features across 62 channels. Feature transformation converted the 62*4 feature matrix into a 248*1 feature matrix for easier use by the subsequent classifier. A commonly used support vector machine (SVM) classifier was chosen, with the simplest linear kernel function selected. Five-fold cross-validation was used for both training and testing.

[0175] 4. Decoding results based on the characteristics of motion preparation potential.

[0176] Based on the above motion intent decoding algorithm based on motion preparation potential characteristics, the performance is analyzed in two cases: when the motion preparation potential is normal (task 1) and when the motion preparation potential disappears (task 2).

[0177] The following results are based on the designed experimental tasks of autonomous movement (Task 1) and movement after time estimation (Task 2). In Task 1, the subjects did not need to estimate the time; after the experimental prompts ended, a rotating clock face would appear, and the subjects only needed to press a button after the clock had completed one revolution. In Task 2, after the experimental prompts ended, the clock hands remained stationary, and the subjects were required to estimate the time and press a button as accurately as possible.

[0178] Studies have shown that motor readiness potentials disappear during motor tasks after time estimation. Therefore, the design of the experimental data above can be considered as follows: the EEG data collected in Task 1 of voluntary movement is EEG data with normal motor readiness potentials, and the EEG data collected in Task 2 of movement after time estimation is EEG data with the disappearance of motor readiness potentials.

[0179] The following data are from two tasks: voluntary movement (Task 1) and movement after time estimation (Task 2) of 16 subjects. Data from 1.5 seconds before the start of movement was taken as the movement preparation task data, and data from 1.5 seconds before the appearance of visual cues was taken as the resting state data.

[0180] The decoding results of motion intent based on motion preparation potential characteristics are shown in Tables 2 and 3:

[0181] Table 2 shows the recognition results of the motion intent decoding algorithm based on motion readiness potential characteristics in Task 1.

[0182]

[0183] Table 3 shows the recognition results of the motion intent decoding algorithm based on motion readiness potential characteristics in Task 2.

[0184]

[0185] Task 1 achieved an average accuracy rate of 81.1%, while Task 2 achieved an average accuracy rate of 76.7%, with Task 2's average accuracy rate significantly lower than Task 1 by 4.4% (p<0.01). Task 1 achieved an average true positive rate of 81.7%, while Task 2 achieved an average true positive rate of 75.8%, with Task 2's average true positive rate significantly lower than Task 1 by 5.9% (p<0.05). Task 1 achieved an average false positive rate of 20.5%, while Task 2 achieved an average false positive rate of 24.0%, with Task 2's average false positive rate significantly higher than Task 1 by 3.5% (p=0.08). The average Kappa coefficient for Task 1 was 0.62, while the average Kappa coefficient for Task 2 was 0.53, with Task 2's average Kappa coefficient significantly lower than Task 1 by 0.09 (p<0.01). These results are largely consistent with the expected results described in the background technique, namely, that the decoding performance of the task with the disappearance of the motion preparation potential is worse than that with the normal motion preparation potential.

[0186] Part Three: A method for decoding motor intent based on multidimensional EEG feature fusion.

[0187] 1. A motion intent decoding algorithm based on multidimensional EEG feature fusion.

[0188] This invention proposes a motion intention decoding algorithm based on multidimensional EEG feature fusion, combining frequency band energy features (time-frequency features), brain network features, and cross-frequency coupling features. Depending on the feature fusion method, this embodiment proposes two feature fusion approaches: a feature-stage approach and a decision-stage approach. Figure 9 and Figure 10As shown, the decision-stage fusion combines the class probabilities output by the support vector machine, while the feature-stage fusion combines all features before performing support vector machine classification. The low-frequency features in the figure are the motion preparation potential features (0.1–4 Hz), and the high-frequency features are the low-frequency features relative to 0.1–4 Hz, with a frequency range of 10–30 Hz. The specific feature extraction and fusion process is as follows:

[0189] Low-frequency time-domain feature extraction: Referring to Part 2, "3. Motion Intent Decoding Method Based on Motion Preparation Potential Features," 248 features are obtained for each trial, namely:

[0190] F LP =[f LP1 ,f LP2 ,f LP3 ,···,f LP248 (35)

[0191] High-frequency energy feature extraction: Referring to the content of "1. (3) Time-frequency feature extraction" in Part 1, the preprocessed data is subjected to a 10-30Hz bandpass filter, and then the average energy of each channel is calculated. For each trial, 62 features are obtained, namely:

[0192]

[0193] Brain network feature extraction: Referring to Part 1, “2. (3) Brain network feature extraction”, firstly, by calculating the average value of the functional connectivity matrix of all subjects in the resting state and the exercise preparation phase, templates for brain network connections in the two situations are obtained; then, for each trial, the correlation coefficient between the connectivity matrix with a center frequency of f Hz and its corresponding template is calculated, where f ranges from 8 to 30. For each trial, 46 features can be obtained, namely:

[0194]

[0195] Cross-frequency coupling feature extraction: Referring to Part 1, “3. (3) Cross-frequency coupling feature extraction”, the coupling value of the 2Hz phase and 10Hz energy of each channel signal was calculated and used as a feature. For each trial, 62 features were obtained, namely:

[0196] F CFC =[MI1,MI2,···,MI 62 (38)

[0197] Finally, the above features are fused to obtain the fused features:

[0198] F All =[F LP ,F BP,F wPLI ,···,F CFC (39)

[0199] To avoid the influence of classifier selection on the comparison of decoding results, the classifier selection and settings are consistent with the content of Part 2, "3. Decoding method of motion intent based on motion preparation potential characteristics".

[0200] 2. Results of motion intent decoding algorithm based on multidimensional EEG feature fusion.

[0201] (1) Results of multidimensional EEG feature fusion.

[0202] To visually compare the performance of the fused features and the original motion preparation potential features proposed in this embodiment, the feature data was visualized using the t-SNE (t-Distributed Stochastic Neighbor Embedding) method. This method is a nonlinear dimensionality reduction and visualization technique that can preserve the local and global information of the original data. The visualization result for one subject is shown below. Figure 11 As shown in the figure, the discriminative power of motor preparation potential features in Task 1 is better than that in Task 2. However, after multidimensional feature fusion, the discriminative power of both Task 1 and Task 2 is significantly improved, and the improvement in the discriminative power of Task 2 is more significant. This result indicates that multidimensional EEG feature fusion can effectively improve the performance of motor intention decoding features.

[0203] (2) Results of motion intent decoding.

[0204] Table 4 shows the decoding accuracy of motion intent under different feature conditions. For high-frequency energy features, the average decoding accuracy was 74.9% and 74.1% in Task 1 and Task 2, respectively; for brain network features, the average decoding accuracy was 67.2% and 66.5% in Task 1 and Task 2, respectively; for cross-frequency coupling features, the average decoding accuracy was 75.5% and 73.2% in Task 1 and Task 2, respectively. For decision fusion, the average decoding accuracy was 72.5% and 71.1% in Task 1 and Task 2, respectively; for feature fusion, the average decoding accuracy was 88.9% and 85.5% in Task 1 and Task 2, respectively. The motion intent decoding accuracy of feature-stage fusion was significantly better than that of decision-stage fusion (p<0.01).

[0205] Table 4. Motion intent decoding accuracy (%) under different feature conditions

[0206]

[0207] exist Figures 12(a) to 12(d)The results of motion intent decoding based on the fusion of motor readiness potential features and multidimensional EEG features were compared. (In each figure, the light-colored bars on the left represent the results of motor readiness potential features, and the dark-colored bars on the right represent the results of multidimensional EEG fusion features proposed in this invention. * indicates p (p value) < 0.05, ** indicates p < 0.01. Figures 12(a), 12(b), 12(c), and 12(d) reflect the average decoding accuracy, true positive rate, false positive rate, and Kappa coefficient of the two algorithms, respectively. The specific decoding results for each subject are shown in Tables 2, 3, 5, and 6. In Task 1, compared with motor readiness potential features, the motion intent decoding platform proposed in this invention...) The average accuracy improved significantly by 7.8% (p<0.01), the true positive rate improved significantly by 4.5% (p<0.05), the false positive rate decreased significantly by 12.1% (p<0.01), and the Kappa coefficient increased significantly by 0.16 (p<0.01). In Task 2, the average decoding accuracy improved significantly by 8.8% (p<0.01), the true positive rate improved by 3.5% (p=0.07), the false positive rate decreased significantly by 14.9% (p<0.01), and the Kappa coefficient increased significantly by 0.18 (p<0.01). Finally, the difference in decoding accuracy between Task 2 and Task 1 narrowed from 4.4% to 3.4%.

[0208] Table 5. Decoding results of motor intention based on multidimensional EEG feature fusion in Task 1.

[0209]

[0210] Table 6. Decoding results of motor intention based on multidimensional EEG feature fusion in Task 2.

[0211]

[0212] In this embodiment, the motion intent decoding results were first calculated based on traditional motion readiness potential features. Task 1 achieved an average decoding accuracy of 81.1%, and Task 2 achieved an average decoding accuracy of 76.7%. Compared with Task 1, the motion intent decoding accuracy of Task 2 decreased significantly. This result indicates that time estimation factors lead to a decrease in the accuracy of motion intent decoding algorithms based on motion readiness potential features, the direct cause of which is the abnormality of motion readiness potential features in the central scalp region. Therefore, relying solely on motion readiness potential features to decode motion intent suffers from feature instability and large fluctuations in recognition rate.

[0213] Time-frequency analysis of EEG signals during the motor preparation phase revealed that both Task 1 and Task 2 exhibited event-related desynchronization during this phase. Although widely used in motor imagery EEG signal decoding research, this feature is relatively weak during the motor preparation phase and is rarely used in current motor intention decoding algorithms. Furthermore, this study found that abnormalities in the motor preparation potential have a relatively small impact on this feature. Classification results based on this feature further demonstrate that similar accuracy rates can be achieved in both task scenarios. Therefore, applying this feature to motor intention decoding can not only improve the accuracy of motor intention decoding but also address the low recognition rate under abnormal motor preparation potential conditions.

[0214] Similar results were obtained based on cross-frequency coupling features, namely, a significant decrease in the modulation coefficients of low-frequency phase and high-frequency amplitude features during the motor preparation process. No significant difference was observed between Task 1 and Task 2, which may be related to the energy decrease in the Alpha band during the motor preparation process. The final classification results based on this feature showed better recognition performance than event-related desynchronization features, with Task 1 outperforming Task 2, but the difference was not significant. This result indicates that cross-frequency coupling features can serve as stable characteristics of pre-motor EEG activity. Brain network analysis results showed that brain network connectivity in the Alpha and Beta bands of motor-related brain regions was significantly enhanced during the motor preparation phase. Although the motor intention decoding result based on these features was only about 66%, it can still be used as a feature to increase the dimensionality of EEG features during the motor preparation process.

[0215] This embodiment explored two feature fusion strategies: the feature-stage strategy and the decision-stage strategy. The results showed that the feature-stage strategy significantly outperformed the decision-stage strategy, achieving average recognition rates of 88.9% and 85.5% in Task 1 and Task 2, respectively. Compared to the motion intention decoding algorithm based on motor preparation potential features, the average accuracy for both tasks improved by 7.8% and 8.8%, respectively, with a significant improvement in recognition rates for both tasks. Furthermore, the difference in recognition rates between the two tasks narrowed from 4.4% to 3.4%, indicating a smaller discrepancy in classification results. These results demonstrate that the motion intention decoding algorithm based on multidimensional EEG feature fusion proposed in this invention can overcome the instability of low-frequency features during the motor preparation process. By combining multiple dimensions of EEG features, it can effectively address the decrease in motion intention decoding accuracy under abnormal motor preparation potential conditions and improve the recognition accuracy of motion intention decoding algorithms based on motor preparation potential features.

[0216] The motion intent decoding algorithm based on multidimensional EEG feature fusion proposed in this invention is superior to current motion intent decoding algorithms based on motor preparation potential features, which achieve a maximum accuracy of around 80%. This result is close to the 90.3% recognition rate of convolutional neural networks, but the algorithm proposed in this invention has the advantages of smaller training samples and faster online recognition speed compared to convolutional neural networks.

Claims

1. A method for decoding motor intent based on multidimensional EEG feature fusion, characterized in that, Includes the following steps: 1) Extract the minimum peak value, slope, standard deviation and average amplitude of each channel from 0.1 to 4 Hz from the EEG data to form a low-frequency time domain feature matrix, extract the average energy feature of each channel from 10 to 30 Hz to form a high-frequency energy feature matrix, extract the brain network feature matrix and calculate the regulation index of 2 Hz phase and 10 Hz amplitude of each channel as features to obtain the cross-frequency coupling feature matrix. The brain network feature matrix is ​​extracted in the following way: First, the average value of the functional connectivity matrix of all trials in the resting state and the exercise preparation phase of the corresponding subject is calculated to obtain the template of brain network connectivity in the resting state and the exercise preparation phase. The correlation coefficients between the connection matrices with center frequencies of Alpha and Beta and their corresponding templates are calculated respectively. The correlation coefficients are used as EEG features for decoding motion intentions to obtain the brain network feature matrix. 2) The four row matrices, namely the low-frequency time-domain feature matrix, the high-frequency energy feature matrix, the brain network feature matrix, and the cross-frequency coupling feature matrix, are sequentially concatenated to obtain a fused feature matrix that is also a row matrix; 3) Input the fused feature matrix into the trained support vector machine to obtain the motion intent decoding result.

2. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, There are 62 EEG acquisition channels, and the low-frequency time-domain feature matrix is ​​mentioned. The feature matrix is ​​248×1; the high-frequency energy feature matrix The feature matrix is ​​62×1; the brain network feature matrix The feature matrix is ​​46×1; the cross-frequency coupling feature matrix It is a 62×1 characteristic matrix.

3. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, The templates for the exercise preparation phase and the resting state can be defined as follows: ， Where N represents the number of subjects, Let f be the average adjacency matrix of the k-th subject during the exercise preparation phase. Let f be the average adjacency matrix of the k-th subject in the resting state. and These represent the adjacency matrix templates for the motion preparation phase and the idle state, respectively, with a center frequency of f.

4. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, The correlation coefficient used is the Pearson correlation coefficient.

5. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, For a trial, the adjacency matrix of the center frequency f is A, and its Pearson correlation coefficient with the corresponding template is: Where n represents the number of elements in the adjacency matrix. This represents the elements in the adjacency matrix A. Let A represent the mean of the adjacency matrix A. This refers to elements within a template. This represents the mean of the elements in the template.

6. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 2, characterized in that, The fused feature matrix .

7. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, The FIR filters used were bandpass filtered at 1 to 3 Hz and 9 to 11 Hz for each EEG channel to obtain a 2 Hz phase and a 10 Hz amplitude for each EEG acquisition channel.

8. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, The filtered EEG signal is obtained from the corresponding test EEG data using a finite unit impulse response filter, and then the average energy of each channel is calculated as a high-frequency feature.

9. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, The EEG data was collected 1.5 seconds before the movement was performed, with 375 sampling points.

10. The motion intent decoding method based on multidimensional EEG feature fusion according to claim 1, characterized in that, The kernel function of the support vector machine is a linear kernel function, and five-fold cross-validation is used in the training and testing process.

Images