Fusion type multi-dimensional motor imagery electroencephalogram classification method, system and device, medium and product

Abstract

The invention discloses a fusion type multi-dimensional motor imagery electroencephalogram classification method, system and device, a medium and a product, and relates to the field of electroencephalogram signal processing.The method comprises the steps that electroencephalogram signals are preprocessed, and preprocessed signals are obtained; electroencephalogram features are extracted from the preprocessed signals through a feature extraction method; the electroencephalogram features are fused from the angles of the time domain, the time-frequency domain and the space domain, and fused features are obtained; constructing an extreme tree model, training the extreme tree model by adopting the training sample, and taking the trained extreme tree model as an electroencephalogram feature selection model until a set condition is met; inputting the fusion features into an electroencephalogram feature selection model to obtain electroencephalogram signal features; adopting a support vector machine to classify the electroencephalogram signal features to obtain a classification result; and generating a control instruction based on the classification result. The electroencephalogram classification accuracy can be further improved.

Description

Technical Field

[0001] The present application relates to the field of electroencephalogram (EEG) signal processing, and in particular to a fused multi-dimensional motor imagery EEG classification method, system, device, medium and product. Background Art

[0002] Wang Jinjia et al. used a hierarchical vector autoregression model for EEG signal extraction, and Liu Linlin et al. proposed an EEG feature extraction method based on phase synchronization combined with AR model coefficients. These two feature extraction methods mainly focus on feature extraction in the time domain and obtain satisfactory classification accuracy in the studied motor imagery tasks. The typical feature extraction method in the time-frequency domain is wavelet transform. Wavelet transform can characterize EEG signals in the time and frequency domains and decompose these EEG signals into multiple frequency bands to reveal different time-frequency features. On the other hand, co-spatial patterns extract features reflecting different power distributions in the spatial domain for different motor imagery tasks. Later, researchers combined co-spatial patterns with other optimization algorithms or classification methods to improve their classification accuracy. For example, Feng Jiankui et al. proposed a new correlation-based time window selection algorithm. The algorithm uses correlation analysis to select the best reference signal and the starting point of the time window for each category. Therefore, combined with the traditional co-spatial pattern, the average classification accuracy of the brain-computer interface based on motor imagery has been greatly improved. Gu Xuejing et al. combined the co-spatial pattern with the convolutional neural network (CNN) to extract high-level features of the original data.

[0003] When the above methods are applied to further improve the classification accuracy of motor imagery tasks, they have the following limitations: These methods only consider some local aspects of the basic characteristics of motor imagery EEG signals and cannot fully reflect the data characteristics and network information flow. These methods do not recognize that EEG signals are complex and rich signals that contain complex information, and lack feature selection methods to screen features to improve the algorithm running speed and obtain higher classification accuracy. Summary of the invention

[0004] The purpose of this application is to provide a fused multi-dimensional motor imagery EEG classification method, system, equipment, medium and product, which can further improve the accuracy of EEG classification.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a fusion-type multi-dimensional motor imagery EEG classification method, comprising:

[0007] Preprocessing the EEG signal to obtain a preprocessed signal;

[0008] Extracting EEG features from the preprocessed signal using a feature extraction method; the feature extraction method is one or more of an autoregressive model, a discrete wavelet transform, and a co-spatial pattern algorithm;

[0009] Fusing the EEG features from the perspectives of time domain, time-frequency domain and space domain to obtain fused features;

[0010] Constructing an extreme tree model, and using training samples to train the extreme tree model until a set condition is met, and then using the trained extreme tree model as an EEG feature selection model;

[0011] Inputting the fusion feature into the EEG feature selection model to obtain EEG signal features;

[0012] Using a support vector machine to classify the features of the EEG signal to obtain a classification result;

[0013] A control instruction is generated based on the classification result.

[0014] Optionally, the method for preprocessing the EEG signal is one or more of principal component analysis, independent component analysis, bandpass filtering, canonical correlation analysis and regression.

[0015] Optionally, a feature extraction method is used to extract EEG features from the preprocessed signal, specifically including:

[0016] Extracting time domain features of EEG features from the preprocessed signal using an autoregressive model;

[0017] Extracting spatial domain features of EEG features from the preprocessed signals using a common spatial pattern algorithm;

[0018] The time-frequency characteristics of the electroencephalogram characteristics are extracted from the preprocessed signal using discrete wavelet transform.

[0019] Optionally, the EEG features are fused from the perspectives of time domain, time-frequency domain and space domain to obtain fused features, specifically including:

[0020] From the perspectives of time domain, time-frequency domain and space domain, the time domain features of the EEG features, the time-frequency features of the EEG features and the space domain features of the EEG features are fused to obtain the fused features.

[0021] In a second aspect, the present application provides a fusion-type multi-dimensional motor imagery EEG classification system, comprising:

[0022] A signal acquisition module, used for collecting EEG signals;

[0023] A signal processing module, connected to the signal acquisition module, for implementing the fused multi-dimensional motor imagery EEG classification method according to any one of claims 1 to 4, so as to obtain a classification result based on the EEG signal;

[0024] A signal conversion module, connected to the signal processing module, for generating a control instruction based on the classification result;

[0025] An external device is connected to the signal conversion module and is used to execute the control instruction.

[0026] Optionally, the signal processing module includes:

[0027] A preprocessing unit, connected to the signal acquisition module, for preprocessing the EEG signal to obtain a preprocessed signal;

[0028] A feature extraction unit connected to the preprocessing unit, for extracting EEG features from the preprocessing signal using a feature extraction method; the feature extraction method is one or more of an autoregressive model, a discrete wavelet transform, and a co-spatial pattern algorithm;

[0029] A feature selection unit is connected to the feature extraction unit and is used to fuse the EEG features from the perspectives of time domain, time-frequency domain and space domain to obtain fused features; construct an extreme tree model and train the extreme tree model with training samples until the set conditions are met, and then use the trained extreme tree model as the EEG feature selection model; input the fused features into the EEG feature selection model to obtain EEG signal features;

[0030] The feature classification unit is connected to the feature selection unit and is used to classify the features of the EEG signal using a support vector machine to obtain a classification result.

[0031] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned fused multi-dimensional motor imagery EEG classification method.

[0032] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned fused multi-dimensional motor imagery EEG classification method.

[0033] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned fused multi-dimensional motor imagery EEG classification method.

[0034] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0035] The present application provides a fused multi-dimensional motor imagery EEG classification method, system, device, medium and product, which fuses the EEG features from the perspectives of time domain, time-frequency domain and space domain, and can obtain new three-dimensional features that can represent motor imagery EEG signals. The fused features usually have some redundant information, which makes the classification model too complicated and has the disadvantage of overfitting, resulting in a decrease in prediction accuracy. In order to solve this problem, the present application uses an extreme tree model to perform feature selection for the fused features, so as to reduce the feature dimension while ensuring the necessary information content of the features. The present application uses a support vector machine to classify the EEG signal features, which can obtain more accurate classification results, thereby improving the accuracy of EEG classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 This is an application environment diagram of a fusion-type multi-dimensional motor imagery EEG classification method in one embodiment of the present application;

[0038] Figure 2 A schematic diagram of a flow chart of a fusion-type multi-dimensional motor imagery EEG classification method provided in one embodiment of the present application;

[0039] Figure 3 An experimental timing diagram of an execution action provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of a flow chart of a fusion-type multi-dimensional motor imagery EEG classification method provided in another embodiment of the present application;

[0041] Figure 5 A schematic diagram of the functional modules of a fusion-type multi-dimensional motor imagery EEG classification system provided in one embodiment of the present application;

[0042] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0045] The fusion type multi-dimensional motor imagery EEG classification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the collected EEG signal to the server 104. After the server 104 receives the collected EEG signal, the server 104 preprocesses the collected EEG signal to obtain a preprocessed signal. The EEG feature is extracted from the preprocessed signal using a feature extraction method. The EEG features are fused from the time domain, time-frequency domain and space domain to obtain a fusion feature. An extreme tree model is constructed, and the extreme tree model is trained with training samples until the set conditions are met, and the trained extreme tree model is used as the EEG feature selection model. The fusion feature is input into the EEG feature selection model to obtain the EEG signal feature. The support vector machine is used to classify the EEG signal features to obtain the classification result. The control instruction is generated based on the classification result. The server 104 can feedback the visual control instruction obtained for the EEG signal to the terminal 102. In addition, in some embodiments, the fused multi-dimensional motor imagery EEG classification method can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly classify and process the collected EEG signals, or the server 104 can obtain the collected EEG signals from the data storage system and classify and process the collected EEG signals.

[0046] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0047] In an exemplary embodiment, Figure 2 As shown, a fusion type multi-dimensional motor imagery EEG classification method is provided, which is executed by a computer device, specifically, it can be executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 201 to 206. Among them:

[0048] Step 200: Preprocess the EEG signal to obtain a preprocessed signal.

[0049] Step 201: extracting EEG features from the preprocessed signal using a feature extraction method, which is one or more of an autoregressive model, a discrete wavelet transform, and a co-spatial pattern algorithm.

[0050] Step 202: Fusing EEG features from the perspectives of time domain, time-frequency domain, and space domain to obtain fused features.

[0051] Step 203: construct an extreme tree model, and use training samples to train the extreme tree model until the set conditions are met, and then use the trained extreme tree model as the EEG feature selection model.

[0052] Step 204: Input the fusion features into the EEG feature selection model to obtain EEG signal features.

[0053] Step 205: Use a support vector machine to classify the EEG signal features to obtain a classification result.

[0054] Step 206: Generate a control instruction based on the classification result.

[0055] By implementing the above steps 201 to 206, a more accurate classification result can be obtained, thereby improving the accuracy of EEG classification. In addition, the present application uses an extreme tree model to perform feature selection for fused features, so as to reduce the feature dimension while ensuring the necessary information content of the features, thereby solving the problem that the fused features usually have some redundant information, making the classification model too complex and having the disadvantage of overfitting, which leads to a decrease in prediction accuracy.

[0056] In another exemplary embodiment of the present application, a visual interface is used to assist the subject to complete the imagined motor behavior, and a 32-channel wired EEG acquisition device is used to collect the EEG signals of the subject's cerebral cortex organization motor imagination. As a type of brain-computer interface, motor imagination refers to the behavior of imagining a specific action but not actually performing the action. The experimental sequence is as follows: Figure 3As shown. The brain generates various forms of signals when performing motor imagery activities, which can be measured by different means. And the EEG signals of different motor imagery tasks are usually accompanied by different sensorimotor rhythms, that is, the increase and decrease of the power of specific frequency bands in each brain region. This embodiment mainly processes this sensorimotor rhythm. For example, at the beginning of the experiment (t=0s), the screen is black, and the subject remains in a resting state. After 2s, a "+" cross appears on the screen, accompanied by a sound prompting the subject to get ready. After 1s, a left and right arrow (representing the left hand and right hand respectively) appears on the screen for 6s to prompt the subject to start motor imagery. At this time, the EEG signals of the subject during the imagination process are obtained.

[0057] In another exemplary embodiment of the present application, the EEG signal is a bioelectric signal with a low signal-to-noise ratio, and the signal amplitude is very weak. The EEG signal contains a large amount of noise and artifacts, and the noise comes from many sources, including non-neural noise and neural noise. In order to improve the signal-to-noise ratio for different noises and artifacts, commonly used preprocessing methods include principal component analysis, independent component analysis, bandpass filtering, canonical correlation analysis and regression.

[0058] For example, in the present embodiment, the pre-processing method of step 200 mainly uses Butterworth filter and independent component analysis method to remove noise and artifacts. Among them, the characteristic of Butterworth filter is that the frequency response curve in the passband is flat to the maximum extent, without fluctuations, and gradually decreases to zero in the stopband. On the Bode diagram of the logarithm of the amplitude to the diagonal frequency, starting from a certain boundary angular frequency, the amplitude gradually decreases with the increase of the angular frequency and tends to negative infinity. The idea of ​​the independent component analysis method comes from the central limit theorem, that is, there is: the result of the joint action of a group of random variables with the same order of magnitude of mean and variance must be close to the Gaussian distribution, so the separation result of a group of mixed signals generated by linear combination of mutually statistically independent sources is measured by non-Gaussianity, and when its non-Gaussianity reaches the maximum, it can be considered that the mixed signal has been optimally separated. Therefore, the independent component analysis method can help step 201 to extract useful EEG signals, i.e. EEG features.

[0059] In another exemplary embodiment of the present application, feature extraction is the core part of the brain-computer interface system, and its purpose is to extract useful information from the EEG signal so as to determine the classification result of step 205 through this information. There are four main features of EEG data: time domain features, frequency domain features, time-frequency features, and spatial domain features. Different features require different feature extraction methods. For the extraction of time domain features, commonly used methods include peak detection, waveform area calculation, and autoregression. For the extraction of frequency domain features, commonly used methods include spectral analysis. For the extraction of time-frequency features, wavelet transform is generally used. For the extraction of spatial domain features, spatial domain filters are generally used. Feature extraction involves how to extract a small amount of useful information from EEG signals, and use this information for classification to distinguish different motor imagination states.

[0060] In this embodiment, the implementation of step 201 of the present application can respectively use an autoregressive model, a co-spatial pattern algorithm and a discrete wavelet transform to extract three types of features, namely, time domain features, spatial domain features and time-frequency features. Among them, the time domain features of the EEG features are extracted from the preprocessed signal using an autoregressive model. The spatial domain features of the EEG features are extracted from the preprocessed signal using a co-spatial pattern algorithm. The time-frequency features of the EEG features are extracted from the preprocessed signal using a discrete wavelet transform.

[0061] For example, the autoregressive model is defined as: if there is an equal period, n sample points y(t) form a time series {y(t); t = 1, 2, ..., n}, the values ​​of the points at multiple previous moments of this time series itself can be regressed, the formula is:

[0062]

[0063] Where e(t) is a white noise sequence with a mean of 0 and a variance of 2, and p is the order of the autoregressive model. i is the coefficient of the i-th order autoregressive model, and y(ti) is the sample point at ti.

[0064] The common spatial pattern algorithm is a spatial filtering feature extraction algorithm for two-classification tasks, which can extract the spatial distribution components of each category from the multi-channel preprocessed signal. The basic principle of the common spatial pattern algorithm is to use the diagonalization of the matrix to find a set of optimal spatial filters for projection, so as to maximize the difference in the variance values ​​of the two types of signals, thereby obtaining a feature vector with high discrimination.

[0065] The preprocessed signal is represented as an N×T matrix X, where N is the number of channels and T is the number of sampling points for each channel. R1 and R2 are used to represent the normalized covariance matrix of left-hand motor imagery and right-hand motor imagery. Then, the normalized covariance matrix of the preprocessed signal is expressed as:

[0066]

[0067] Where X1 and X2 are the preprocessing signals of left-hand motor imagery and right-hand motor imagery.

[0068] Then the mixed space covariance matrix is ​​expressed as R m :

[0069]

[0070] In the formula, To take the average value.

[0071] The eigenvalue decomposition is performed according to the mixed space covariance matrix. The specific method is described as:

[0072] R m =UλU T (4)

[0073] Where U is the eigenvector matrix of matrix λ, and λ is the diagonal matrix of the corresponding eigenvalues. Then the whitening value matrix is ​​P:

[0074]

[0075] Transform R1 and R2 as shown in formula (6) to obtain matrix S1 and matrix S2.

[0076] S1=PR1P T , S2=PR2P T (6)

[0077] After principal component decomposition of matrix S1 and matrix S2, it can be proved that the eigenvector matrices of matrix S1 and matrix S2 are equal, and the sum of the diagonal matrices λ1 and λ2 of the two eigenvalues ​​is the unit matrix. Therefore, the spatial filter is constructed as:

[0078] W=B T P (7)

[0079] Where B is the eigenvector matrix of matrix S1 and matrix S2, and W is the spatial filtering result, that is, the spatial domain features of EEG features.

[0080] Wavelet transform is a transform analysis method that inherits and develops the positioning idea in short-time Fourier transform. As an ideal tool for signal time-frequency analysis, wavelet transform overcomes the disadvantage that the time-frequency window is fixed and cannot be adjusted with frequency. Among them, continuous wavelet transform is defined as:

[0081]

[0082] Where W x(a, b) is the wavelet change value, that is, the time-frequency characteristics of the EEG characteristics, x(t) is the preprocessed signal, ψ(t) is the mother wavelet, and a series of sub-wavelets ψ will be generated under the action of parameters a and b. a,b (t), a is the scaling factor of the wavelet function, and b is the translation parameter of the wavelet function. These two parameters adjust the frequency scale and time scale respectively. * (*) is the conjugate complex number of ψ(*). The mother wavelet may be a Daubechies-type db4 wavelet.

[0083] For different mother wavelets, the continuous wavelet transform of the same signal is different. Discrete wavelet transform requires discretization of the continuous wavelet transform, which is:

[0084] W x (j,r)=2 -j / 2 ∫x(t)ψ * (2 -j tr)dt= <x(t),ψ j,r (t)> (9)

[0085] In the formula, It is called a two-dimensional wavelet, W x (j,r) is the discretization result.

[0086] In another exemplary embodiment of the present application, the motor imagery (MI-EEG) signal always contains complex and diverse information from different brain regions. However, extracting features from only one dimension cannot reflect these comprehensive information. Therefore, step 202 of the present application adopts a feature fusion algorithm to obtain fusion features in the time domain, time-frequency domain and space domain to represent the three-dimensional features of the brain network. First, the coefficients of the autoregressive model (ARmodel, AR model) are extracted using the AR algorithm to construct a feature vector, thereby preliminarily obtaining the time domain features of the EEG features. Secondly, the spatial domain features of the EEG features are obtained by the Common Spatial Pattern (CSP) algorithm, and the variance of each channel is used as a feature vector. Finally, the time-frequency domain features of the EEG features are extracted by discrete wavelet transform (Discrete Wavelet Transformation, DWT), and the Daubechies-like db4 wavelet is used for 3-layer decomposition, and the average value of the square of the third layer coefficients of each channel is used as the feature vector.

[0087] In another exemplary embodiment of the present application, in order to retain and select the most important features, the tree model algorithm as an embedded feature filtering algorithm is based on machine learning theory to analyze the importance of features. Random forest and extreme tree are two typical tree model algorithms.

[0088] Random forest is a comprehensive learning method based on the bagging method. The advantage of this comprehensive algorithm is that each decision tree is built by random variables. The randomness of random forest lies in sample randomization, feature randomization, parameter randomization and model randomization. Extreme tree is a variant of the random forest algorithm with different random processes. Extreme tree uses all training samples as training samples to build each decision tree, and randomly extracts segmentation rules at each node, and selects the optimal segmentation rule according to the score. Therefore, another feature of randomness is introduced, namely split randomness. This split randomness greatly enhances the independence between each decision tree, so it improves the training speed and generalization ability of the classifier. Based on this, in the above step 203 of the present application, the construction process of the extreme tree model can be described as:

[0089] 1. All original training samples D are selected as input data.

[0090] 2. Select m features {f1,f2,...,f m}Build a decision tree. Generally, m is much smaller than M.

[0091] 3. Randomly select partition values ​​{d1, d2, ..., d k}, the optimal partition value d is selected by the specific normalization of information gain. The formula for the specific normalization of information gain is as follows:

[0092]

[0093] Among them, Score C (d,D) is the specific normalization result of information gain, H C (D) is the classification (logarithmic) entropy of training sample D, H d (D) is the splitting entropy, is the mutual information between the two.

[0094] 4. Sort by feature importance scores, select features with higher scores as important features, obtain the optimal feature subset, and repeat steps 2 and 3 until multiple decision trees are built to obtain an extreme tree model.

[0095] In another exemplary embodiment of the present application, brain-computer interaction is an important part of determining the performance of the brain-computer interface system. In order to convert brain activity into computer commands and realize brain-computer interaction, the most widely used classifiers in the EEG signal pattern classification process are linear discriminant analysis, support vector machine, deep learning and Riemann geometry classifier. In this embodiment, the support vector machine is mainly used to classify the EEG signal features, and the implementation process of step 205 can be:

[0096] S1: The optimal feature subset and classification label together constitute the data G.

[0097] S2: Divide the data G into training data and test data.

[0098] S3: Select a linear kernel function and use the training data to train a support vector machine (SVM).

[0099] S4: Use grid search method to determine the penalty factor c.

[0100] S5: Test the performance of the trained SVM using the test data.

[0101] S6: Use the trained SVM to classify and predict the EEG signal features to obtain the classification results.

[0102] In another exemplary embodiment of the present application, in order to convert the processed EEG signals into control instructions, for example, in a motor imagery brain-computer interface, imagining left hand movement or right hand movement represents different control commands. The subject generates corresponding EEG signals by imagining different movements, which are identified by corresponding algorithms to control the lower computer or external device.

[0103] In summary, EEG signals always include complex and diverse information from different brain regions. However, extracting features from only one dimension cannot reflect this comprehensive information. Figure 4 As shown, the present application adopts a feature fusion algorithm to fuse the time domain features extracted by the autoregressive model, the spatial domain features extracted by the common spatial pattern algorithm, and the time-frequency domain features extracted by the discrete wavelet transform to obtain a new three-dimensional feature that can represent the motor imagery EEG signal. The fused features usually have some redundant information, which makes the classification model too complex and overfitted, resulting in a decrease in prediction accuracy. In order to solve this problem, the present application adopts an extreme tree model feature selection algorithm to screen the fused features, reduce the dimension of the features, and ensure the necessary information content of the features. Finally, the selected features are placed in the classifier to obtain a higher target recognition accuracy.

[0104] Target recognition accuracy is a key part of studying brain-computer interfaces. With the improvement of target recognition accuracy, the accuracy of online brain-computer interfaces can be greatly increased. As the pace of life continues to accelerate, more and more people suffer from motor dysfunction every year due to various reasons. For the above patients, traditional treatment methods are to restore motor skills through high-intensity rehabilitation training, and auxiliary exercises such as wheelchairs are required outside of treatment, which is time-consuming and laborious. And with the improvement of living standards, the aging population is also increasing. The main purpose of online brain-computer interface devices is to repair or replace the lost motor skills of the above-mentioned people and improve their quality of life. In summary, this application has very important research significance for brain-computer interfaces.

[0105] Based on the same inventive concept, the embodiment of the present application also provides a fused multi-dimensional motor imagery EEG classification system for implementing the fused multi-dimensional motor imagery EEG classification method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more fused multi-dimensional motor imagery EEG classification system embodiments provided below can refer to the limitations of the fused multi-dimensional motor imagery EEG classification method above, and will not be repeated here.

[0106] In an exemplary embodiment, Figure 5 As shown, a fusion-type multi-dimensional motor imagery EEG classification system is provided, including: a signal acquisition module, a signal processing module, a signal conversion module and external equipment.

[0107] The signal acquisition module is used to collect EEG signals. The signal acquisition module can be a product of BP Company of Germany, which includes an amplifier, a 32-lead EEG cap, a laptop computer, an EEG analyzer and other components. The collected EEG signals are converted into a processable format through the device and processed using Matlab software.

[0108] The signal processing module is connected to the signal acquisition module, and is mainly used to implement the fused multi-dimensional motor imagery EEG classification method as claimed in any one of claims 1 to 4, so as to obtain classification results based on EEG signals.

[0109] The signal conversion module is connected to the signal processing module, and is mainly used to generate control instructions based on the classification results.

[0110] The external device is connected to the signal conversion module, which is mainly used to execute the control command. The external device can be a wheelchair, a mechanical arm, etc.

[0111] As an optional implementation, Figure 5 As shown, the signal processing module provided by the present application includes: a preprocessing unit, a feature extraction unit, a feature selection unit and a feature classification unit.

[0112] The preprocessing unit is connected to the signal acquisition module, and is mainly used to preprocess the EEG signal to obtain a preprocessed signal.

[0113] The feature extraction unit is connected to the preprocessing unit and is mainly used to extract EEG features from the preprocessing signal using a feature extraction method, which is one or more of an autoregressive model, a discrete wavelet transform, and a common space pattern algorithm.

[0114] The feature selection unit is connected to the feature extraction unit, and is mainly used to fuse EEG features from the perspectives of time domain, time-frequency domain, and space domain to obtain fused features. An extreme tree model is constructed, and the extreme tree model is trained with training samples until the set conditions are met, and the trained extreme tree model is used as the EEG feature selection model. The fused features are input into the EEG feature selection model to obtain EEG signal features.

[0115] The feature classification unit is connected to the feature selection unit, and is mainly used to classify the features of the EEG signal using a support vector machine to obtain a classification result.

[0116] Among them, the data processing procedures in the preprocessing unit mainly include data processing, feature extraction, feature selection and pattern classification. For data preprocessing, the collected EEG signals are band-pass filtered using a Butterworth filter, because most of the response frequency bands of motor imagery EEG signals are 8-30Hz. Therefore, the passband frequency of the filter is set to 8-30Hz. At the same time, the order of the filter is set to 4. Three methods, namely, autoregressive model, co-spatial pattern and discrete wavelet transform, are used to obtain the characteristics of EEG features in the time-frequency domain and spatial domain. Then, all the extracted features are fused to obtain a complete feature vector (i.e., fused features). In the subsequent feature extraction, the fused features are used to obtain the best feature subset through the extreme tree model, and irrelevant and redundant features are eliminated. Finally, the screened features are classified by the support vector machine to obtain the classification accuracy.

[0117] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store EEG classification data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a fusion-type multi-dimensional motor imagery EEG classification method is implemented.

[0118] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0119] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0120] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0122] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0123] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0124] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A fusion-type multi-dimensional motor imagery EEG classification method, characterized in that: The fusion type multi-dimensional motor imagery EEG classification method comprises: Preprocessing the EEG signal to obtain a preprocessed signal; Extracting EEG features from the preprocessed signal using a feature extraction method; the feature extraction method is one or more of an autoregressive model, a discrete wavelet transform, and a co-spatial pattern algorithm; Fusing the EEG features from the perspectives of time domain, time-frequency domain and space domain to obtain fused features; Constructing an extreme tree model, and using training samples to train the extreme tree model until a set condition is met, and then using the trained extreme tree model as an EEG feature selection model; Inputting the fusion feature into the EEG feature selection model to obtain EEG signal features; Using a support vector machine to classify the features of the EEG signal to obtain a classification result; A control instruction is generated based on the classification result.

2. The fusion-type multi-dimensional motor imagery EEG classification method according to claim 1 is characterized in that: The method for preprocessing the EEG signal is one or more of principal component analysis, independent component analysis, bandpass filtering, canonical correlation analysis and regression.

3. The fusion-type multi-dimensional motor imagery EEG classification method according to claim 1 is characterized in that: The EEG features are extracted from the preprocessed signal using a feature extraction method, specifically including: Extracting time domain features of EEG features from the preprocessed signal using an autoregressive model; Extracting spatial domain features of EEG features from the preprocessed signals using a common spatial pattern algorithm; The time-frequency characteristics of the electroencephalogram characteristics are extracted from the preprocessed signal using discrete wavelet transform.

4. The fusion-type multi-dimensional motor imagery EEG classification method according to claim 3 is characterized in that: The EEG features are fused from the perspectives of time domain, time-frequency domain and space domain to obtain fused features, specifically including: From the perspectives of time domain, time-frequency domain and space domain, the time domain features of the EEG features, the time-frequency features of the EEG features and the space domain features of the EEG features are fused to obtain the fused features.

5. A fusion-type multi-dimensional motor imagery EEG classification system, characterized in that: The fusion type multi-dimensional motor imagery EEG classification system comprises: A signal acquisition module, used for collecting EEG signals; A signal processing module, connected to the signal acquisition module, for implementing the fused multi-dimensional motor imagery EEG classification method according to any one of claims 1 to 4, so as to obtain a classification result based on the EEG signal; A signal conversion module, connected to the signal processing module, for generating a control instruction based on the classification result; An external device is connected to the signal conversion module and is used to execute the control instruction.

6. The fusion-type multi-dimensional motor imagery EEG classification system according to claim 5, characterized in that: The signal processing module comprises: A preprocessing unit, connected to the signal acquisition module, for preprocessing the EEG signal to obtain a preprocessed signal; A feature extraction unit connected to the preprocessing unit, for extracting EEG features from the preprocessing signal using a feature extraction method; the feature extraction method is one or more of an autoregressive model, a discrete wavelet transform, and a co-spatial pattern algorithm; A feature selection unit is connected to the feature extraction unit and is used to fuse the EEG features from the perspectives of time domain, time-frequency domain and space domain to obtain fused features; construct an extreme tree model and train the extreme tree model with training samples until the set conditions are met, and then use the trained extreme tree model as the EEG feature selection model; input the fused features into the EEG feature selection model to obtain EEG signal features; The feature classification unit is connected to the feature selection unit and is used to classify the features of the EEG signal using a support vector machine to obtain a classification result.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the fused multi-dimensional motor imagery EEG classification method according to any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fused multi-dimensional motor imagery EEG classification method described in any one of claims 1 to 4 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the fused multi-dimensional motor imagery EEG classification method described in any one of claims 1 to 4 is implemented.

Images