Motor imagery brain-computer interface communication method, device, system, medium and equipment

Abstract

The application provides a motor imagery brain-computer interface communication method, device, system, medium and equipment. The motor imagery brain-computer interface communication method comprises the following steps: acquiring labeled electroencephalogram data, and dividing the electroencephalogram data into a training set and a verification set; randomly selecting a plurality of source domain subjects as a mixed domain, obtaining a mixed source domain model based on the electroencephalogram data of the training set corresponding to the mixed domain; and training the mixed source domain model based on the electroencephalogram data of each source domain subject in the training set, to obtain a target domain model corresponding to each source domain subject. The application can improve the classification performance of the motor imagery brain-computer interface, and realize the deployment of the motor imagery brain-computer interface in reality.

Description

Technical Field

[0001] This invention belongs to the field of brain-computer interface communication and computing technology, and relates to detection methods and feedback systems, particularly to a motor imagery brain-computer interface communication method, device, system, medium and equipment. Background Technology

[0002] Brain-computer interfaces (BCIs) are a novel technology that enables information interaction between the human brain and external devices by collecting electroencephalogram (EEG) signals from the surface of the human skull. Several main implementation methods of BCIs can be categorized according to the type of EEG signals extracted, including P300 BCIs, motor imagery BCIs, and SSVEP (Steady-state visual evoked potential) BCIs. Among these, motor imagery BCIs have vast application potential in both medical rehabilitation and non-medical fields, thus attracting considerable attention from researchers. Motor imagery tasks are accomplished by imagining a specific task without actually performing it. Currently, widely used motor imagery tasks involve imagining body parts such as the right hand, left hand, right foot, left foot, both feet, and tongue; many other tasks are also under investigation, such as movements related to the elbow, fist, and fingers. During motor imagery, the cerebral cortex generates two distinct rhythmic signals: a μ rhythm of 8-15 Hz and a β rhythm of 18-24 Hz. During motor imagery, neurons are activated, metabolism accelerates, and the energy of the EEG rhythm in the contralateral motor sensory cortex of the cerebral cortex decreases significantly, while the energy of the EEG rhythm in the ipsilateral motor sensory cortex increases. This phenomenon is called Event Related Desynchronization (ERD) / Event Related Synchronization (ERS). Based on this relationship, various control commands can be generated by actively controlling the left and right hemispheres and adjusting the amplitude of the rhythms. The brain control process produces the ERD / ERS phenomenon of left and right hand motor imagery. The ERD / ERS phenomenon is strongly correlated with the brain's actual motor intention, and it is generated by the subject through voluntary motor imagery, without the need for actual limb movement or external stimulation. Therefore, decoding the motor intention in the EEG signals during the formation of the ERD / ERS phenomenon is very suitable as a basis for brain-computer interface control of external devices. Especially for patients with paraplegia or limb defects, motor imagery can be used to control external devices such as exoskeletons, wheelchairs, and robotic arms for rehabilitation training and assisted walking.

[0003] A complete motor imagery brain-computer interface (BCI) system comprises three key technical steps: signal acquisition, signal processing, and external device control and feedback. Specifically, in the signal acquisition phase, the subject needs to visualize movement in a specific body part. For example, hand movements might involve gripping a ball or shifting gears in a car, while lower limb movements might involve gripping the ground with the toes. This visualization needs to be sustained for a certain time (typically 2–4 seconds), generating ERD / ERS signals in the corresponding area of ​​the subject's cerebral cortex. Simultaneously, by placing an electrode cap on the subject, electrodes can non-invasively acquire EEG signals from the scalp surface. These signals are then transmitted via cables to an analog-to-digital conversion module and a data forwarding module, where standardized digital EEG signals are obtained on a host computer. By processing and analyzing this digital signal using specific algorithms, it becomes clear which body part the subject is visually moving and what specific control command they are attempting to output. This control command is then transmitted to an external device to achieve the desired control effect. The control effect of the external device serves as feedback, adjusting the subject's motor imagery and thus completing the entire workflow of the motor imagery BCI.

[0004] After preprocessing such as downsampling, filtering, and artifact removal, the signal-to-noise ratio (SNR) of EEG signals in BCI (Body-Computer Interface) is enhanced, but the data volume is still large, requiring further extraction of feature information related to motor imagery. EEG signal feature extraction algorithms in BCI systems often analyze from different perspectives: time domain, frequency domain, and spatial domain. Common time-domain feature extraction algorithms include Hjorth parameters, Adaptive Autoregressive (AAR) models, and particle filtering. Common frequency-domain feature extraction algorithms include Fast Fourier Transform (FFT), spectral features, and Wavelet Transform (WT). A common spatial-domain feature extraction algorithm is the Common Spatial Pattern (CSP). This algorithm aims to find a spatial filter bank that maximizes the variance ratio of different signal classes. In addition to the traditional feature extraction algorithms mentioned above, many more feature extraction algorithms combine time-domain, frequency-domain, and spatial-domain analysis methods. For example, wavelet packet transform (WPT) can extract information from multiple time windows and frequency bands, making it more suitable for non-stationary EEG signals than FFT.

[0005] After feature extraction from EEG signals, the feature values ​​are input to a classifier for further processing. The classifier selects a suitable label from two or more categories for the input feature values ​​to generate discrete output control signals (e.g., controlling a robotic arm to move up or down). Classification learning algorithms can be categorized into supervised, unsupervised, and semi-supervised algorithms based on whether the samples used to train the classifier are labeled. As the names suggest, supervised classification algorithms use labeled samples; unsupervised classification algorithms use unlabeled samples; and semi-supervised classification algorithms use only partially labeled samples. Currently, the best-performing classifiers for motor imagery classification are all supervised algorithms, such as Kalman Adaptive Linear Discriminant Analysis (KALDA) based on Linear Discriminant Analysis (LDA), Bayesian classifiers, and Support Vector Machines (SVM). Neural networks (NNs), mimicking biological neural networks, are characterized by strong adaptability and high fitting ability, exhibiting a natural affinity for non-stationary and highly complex electroencephalogram (EEG) signals. Neural network-based algorithms such as DeepConvNet, ShallowCSPNet, EEGNet, and FBCNet have achieved good results in motor imagery brain-computer interface (BCI) classification problems. However, algorithms with higher classification accuracy typically require more training data. The preparation time for motor imagery BCI experiments is long, and the experimental process is lengthy, making it difficult to obtain sufficient training data for each subject. Furthermore, due to the significant differences in EEG signals among different subjects, training data collected from other subjects cannot be directly used for training new subjects. This is the main factor restricting the practical application of motor imagery BCIs.

[0006] Improving the performance of motor imagery brain-computer interface systems is a challenging problem when training samples for subjects are limited. In recent years, transfer learning (TL) has emerged as a novel approach to address this issue. TL uses data from different feature spaces or feature distributions to compensate for the lack of labeled training data, thereby constructing more accurate classification models. The goal of TL is to extract useful knowledge from one or more source tasks and apply it to another target task. Unlike traditional machine learning, it does not require data distributions with identical feature distributions; only relevance is needed. Traditional machine learning and transfer learning differ in having different visual objectives, while in the combined SSVEP paradigm, N distinct frequencies can encode N... 2 There are 10 different visual targets, and this number is even higher when more than 2 visual stimuli are combined into one target.

[0007] A typical approach to improving the classification performance of motor imagery brain-computer interface (BCI) systems using transfer learning involves training a classification model using experimental samples from other subjects, then retaining some of the model's structure and continuing training on top of that model using new subject data. However, traditional deep transfer learning methods still have some shortcomings. Firstly, current transfer learning-based BCI algorithms typically use either a single, manually selected existing subject or all existing subjects as the source domain. The former fails to fully utilize all existing subject data and cannot meet the training data requirements of more complex models; the latter leads to interference between training data from different feature spaces (subjects), making model convergence difficult. Secondly, current transfer learning-based BCI algorithms are one-off and cannot be updated. When new subjects are introduced into the dataset, all trained subjects need to be retrained to utilize their data. If new subjects are introduced repeatedly with an irregular number of participants, the transfer algorithm needs to be modified accordingly, which is impractical. Therefore, current transfer learning-based BCI algorithms can only work with existing datasets and cannot function as a dynamically updated system that adapts to an increasing number of subjects. Summary of the Invention

[0008] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a motor imagination brain-computer interface communication method, device, system, medium and equipment to improve the classification performance of the motor imagination brain-computer interface and realize the deployment of the motor imagination brain-computer interface in reality.

[0009] To achieve the above and other related objectives, the present invention provides a motor imagery brain-computer interface communication method, the method comprising: acquiring labeled electroencephalogram (EEG) data and dividing the EEG data into a training set and a validation set; randomly selecting several source domain subjects as a hybrid domain, obtaining a hybrid source domain model based on the EEG data of the training set corresponding to the hybrid domain; and training the hybrid source domain model based on the EEG data of each source domain subject in the training set to obtain a target domain model corresponding to each source domain subject.

[0010] In one embodiment of the present invention, the step of randomly selecting several source domain subjects as a hybrid domain and obtaining a hybrid source domain model based on the EEG data of the training set corresponding to the hybrid domain includes: randomly selecting several source domain subjects as an initial hybrid domain, and obtaining a first hybrid source domain model based on the EEG data of the training set corresponding to the initial hybrid domain; extracting the feature extractor parameters of the first hybrid source domain model as initialization parameters for the model feature extractor of the next hybrid domain model; randomly selecting several source domain subjects as a second hybrid domain, and obtaining a second hybrid source domain model based on the EEG data of the training set corresponding to the second hybrid domain and the feature extractor parameters of the first hybrid source domain model; and so on, to obtain an Nth hybrid domain and an Nth hybrid source domain model corresponding to the Nth hybrid domain.

[0011] In one embodiment of the present invention, the parameters of the classifier in the first hybrid source domain model, the second hybrid source domain model, ..., the Nth hybrid source domain model are randomly initialized.

[0012] In one embodiment of the present invention, the method further includes: verifying the first mixed source domain model, the second mixed source domain model, ..., the Nth mixed source domain model based on the EEG data corresponding to the subjects in each source domain of the verification set, and outputting the verification results.

[0013] In one embodiment of the present invention, the method further includes: comparing the verification result with the historical mixed source domain model of the corresponding source domain subject to obtain the optimal mixed source domain model corresponding to the source domain subject.

[0014] In one embodiment of the present invention, training the hybrid source domain model based on the EEG data of subjects in each source domain of the training set includes: training the optimal hybrid source domain model corresponding to the subject in each source domain based on the EEG data of subjects in each source domain of the training set.

[0015] This invention also discloses a motor imagery brain-computer interface communication device, which includes: a dataset module for acquiring labeled EEG data and dividing the EEG data into a training set and a validation set; a hybrid domain model module for randomly selecting several source domain subjects as hybrid domains and obtaining a hybrid source domain model based on the EEG data of the training set corresponding to the hybrid domain; and a target domain model module for training the hybrid source domain model based on the EEG data of each source domain subject in the training set to obtain a target domain model corresponding to each source domain subject.

[0016] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the motor imagery brain-computer interface communication method described above.

[0017] The present invention also discloses an electronic device, the electronic device comprising: a memory storing a computer program; and a processor communicatively connected to the memory, which, when the computer program is invoked, implements the steps of the motor imagery brain-computer interface communication method described above.

[0018] This invention also discloses a motor imagery brain-computer interface communication system, which includes the electronic device described above, at least one EEG acquisition device, at least one computing device, and at least one external device; wherein: the electronic device is a cloud server; the EEG acquisition device is connected to the computing device and is used to acquire EEG data of source domain subjects and transmit the acquired EEG data of source domain subjects to the corresponding computing device; each computing device is connected to the cloud server and is used to classify and label the EEG data received from the EEG acquisition device to form labeled EEG data, transmit the labeled EEG data to the cloud server, and receive a target domain model corresponding to each source domain subject from the cloud server, and generate control commands to control the external device based on the target domain model; each external device is connected to the corresponding computing device and operates based on the control commands received from the computing device.

[0019] As described above, the motor imagery brain-computer interface communication method, device, system, medium, and equipment of the present invention have the following beneficial effects:

[0020] 1. In this invention, the source domain subjects are no longer manually selected, but rather the subjects automatically selected by the system from among all existing subjects who have the highest matching degree with the target subjects. This results in a larger amount of source domain data, with higher similarity to the target subjects' EEG data, leading to a greater improvement in classification performance and enhancing the classification performance of the motor imagery brain-computer interface.

[0021] 2. In this invention, whenever a new subject is introduced into the dataset, the model of all the subjects that have been trained can be updated when training the new subject. The classification accuracy of all subjects in the entire system will increase as the number of subjects collected increases, realizing the deployment of motor imagery brain-computer interface in reality. Therefore, it also solves the problem that the above-mentioned traditional methods are difficult to apply in reality. Attached Figure Description

[0022] Figure 1 The diagram shown is a flowchart illustrating the motor imagery brain-computer interface communication method according to an embodiment of the present invention.

[0023] Figure 2 The diagram shows the workflow of multi-source domain transfer learning in the motor imagery brain-computer interface communication method described in this embodiment of the invention.

[0024] Figure 3 The diagram shown is a schematic diagram of the principle structure of the motor imagination brain-computer interface communication device according to an embodiment of the present invention.

[0025] Figure 4 The diagram shown is a schematic representation of one implementation structure of the electronic device described in an embodiment of the present invention.

[0026] Figure 5 The diagram shows the workflow of the motor imagination brain-computer interface communication method and the principle structure of the motor imagination brain-computer interface communication system according to an embodiment of the present invention.

[0027] Component designation explanation

[0028] 100 Motor Imagination Brain-Computer Interface Communication Device

[0029] 110 Dataset Module

[0030] 120 Hybrid Domain Model Module

[0031] 130 Target Domain Model Module

[0032] 1. Motor Imagery Brain-Computer Interface Communication System

[0033] 10 Cloud Servers

[0034] 20 EEG acquisition devices

[0035] 30 Computing devices

[0036] 40 External devices

[0037] 101 Electronic Devices

[0038] 1001 Memory

[0039] 1002 processor

[0040] S100~S300 Steps Detailed Implementation

[0041] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0042] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0043] The purpose of this invention is to provide a motor imagination brain-computer interface communication method, device, system, medium and equipment to improve the classification performance of the motor imagination brain-computer interface and realize the deployment of the motor imagination brain-computer interface in reality.

[0044] This embodiment provides a method, device, system, medium, and equipment for motor imagery brain-computer interface communication. In comparison, in traditional motor imagery brain-computer interface systems, the training data and test data in this embodiment belong to the same subjects and are mutually corresponding. Training data from different subjects cannot be mixed, otherwise, classification accuracy will be severely affected. Traditional transfer learning motor imagery algorithms proposed to address this situation require manual selection of specific source domain subjects. During the transfer process, only the data from the selected source domain subjects is effective, while the data from other existing subjects is wasted, leaving significant room for improvement in classification performance. Furthermore, current transfer learning-based motor imagery brain-computer interface algorithms are only applicable to specific datasets and models, lacking updability and scalability.

[0045] The following will describe in detail the principles and implementation methods of the motor imagery brain-computer interface communication method, device, system, medium and equipment of this embodiment, so that those skilled in the art can understand the motor imagery brain-computer interface communication method, device, system, medium and equipment of this embodiment without creative effort.

[0046] Please see Figure 1 This embodiment provides a motor imagination brain-computer interface communication method, which includes the following steps S100 to S300.

[0047] S100, acquire labeled EEG data and divide the EEG data into a training set and a validation set;

[0048] S200, randomly select several source domain subjects as the mixed domain, and obtain the mixed source domain model based on the EEG data of the training set corresponding to the mixed domain;

[0049] S300, the hybrid source domain model is trained based on the EEG data of each source domain subject in the training set to obtain target domain models corresponding to each source domain subject.

[0050] The following provides a detailed description of steps S100 to S300 in the motor imagery brain-computer interface communication method of this embodiment.

[0051] S100: Acquire labeled EEG data and divide the EEG data into a training set and a validation set.

[0052] The training set and validation set are pre-defined in a specific ratio. Preferably, the training set is larger than the validation set, meaning the ratio of the training set to the validation set is greater than 1. For example, 80% of the labeled EEG data is used as the training set, and 20% is used as the validation set.

[0053] S200, randomly select several source domain subjects as the mixed domain, and obtain the mixed source domain model based on the EEG data of the training set corresponding to the mixed domain.

[0054] Figure 2 This diagram illustrates the workflow of multi-source domain transfer learning in the motor imagery brain-computer interface communication method described in this embodiment. In this embodiment, the step of randomly selecting several source domain subjects as a hybrid domain, and obtaining a hybrid source domain model based on the EEG data of the training set corresponding to the hybrid domain, includes:

[0055] 1) Randomly select several source domain subjects as the initial mixed domain, and obtain the first mixed source domain model based on the EEG data of the training set corresponding to the initial mixed domain;

[0056] 2) Extract the feature extractor parameters of the first hybrid source domain model as the initialization parameters of the feature extractor of the hybrid domain model in the next round; randomly select several source domain subjects as the second hybrid domain, and obtain the second hybrid source domain model based on the EEG data of the training set corresponding to the second hybrid domain and the feature extractor parameters of the first hybrid source domain model;

[0057] 3) By analogy, the Nth mixing domain and the Nth mixing source domain model corresponding to the Nth mixing domain are obtained.

[0058] S300, the hybrid source domain model is trained based on the EEG data of each source domain subject in the training set to obtain target domain models corresponding to each source domain subject.

[0059] In this embodiment, in each round of training, several source domain subjects are randomly selected from all source domain subjects as a mixed source domain, and the mixed source domain model is trained using their training data. Then, the feature extractor parameters of the model are sent to the next round as the initialization parameters of the feature extractor of the next round model.

[0060] In this embodiment, the parameters of the classifiers in the first hybrid source domain model, the second hybrid source domain model, ..., the Nth hybrid source domain model are randomly initialized. That is, the classifier is randomly initialized in each round.

[0061] In this embodiment, the method further includes: verifying the first mixed source domain model, the second mixed source domain model, ..., the Nth mixed source domain model based on the EEG data corresponding to the subjects in each source domain of the verification set, and outputting the verification results.

[0062] In this embodiment, the method further includes: comparing the verification result with the historical mixed source domain model of the corresponding source domain subject to obtain the optimal mixed source domain model corresponding to the source domain subject.

[0063] In other words, the source domain model obtained in each round is validated on the corresponding subject's validation set, and the parameters of the model with better performance are retained. After all mixed source domain training is completed, all source domain subjects receive the updated new model.

[0064] In this embodiment, training the hybrid source domain model based on the EEG data of subjects in each source domain of the training set includes: training the optimal hybrid source domain model corresponding to the subject in each source domain based on the EEG data of subjects in each source domain of the training set.

[0065] In traditional motor imagery brain-computer interface systems, a large amount of training data needs to be collected for each subject before use to train the feature extractor and classifier. The amount of training data required for model fitting usually takes several hours or even days to collect, which is unacceptable in practical applications. In contrast, this embodiment is based on a multi-source transfer learning, dynamically updatable motor imagery brain-computer interface method. It can utilize the EEG data of all existing source domain subjects for model training, achieving better classification performance while significantly reducing the amount of training data for new source domain subjects.

[0066] Meanwhile, this embodiment overcomes the shortcomings of past motor imagery brain-computer interface systems that were only applicable to specific subjects and specific model structures. Whenever new subject data is obtained, the models of all existing subjects can be updated. This embodiment does not have requirements for the EEG feature extraction and classification algorithms used; appropriate EEG feature extraction and classification algorithms can be replaced according to actual needs. Furthermore, all traditional algorithms can achieve significant improvements in classification performance when combined with this embodiment.

[0067] The scope of protection of the motor imagery brain-computer interface communication method described in this invention is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0068] like Figure 3 As shown, this embodiment also discloses a motor imagination brain-computer interface communication device 100, which includes: a dataset module 110, a hybrid domain model module 120, and a target domain model module 130.

[0069] The dataset module 110 is used to acquire labeled EEG data and divide the EEG data into a training set and a validation set; the hybrid domain model module 120 is used to randomly select several source domain subjects as hybrid domains and obtain hybrid source domain models based on the EEG data of the training set corresponding to the hybrid domains; the target domain model module 130 is used to train the hybrid source domain models based on the EEG data of each source domain subject in the training set to obtain target domain models corresponding to each source domain subject.

[0070] In this embodiment, the motor imagery brain-computer interface communication device 100 can implement the motor imagery brain-computer interface communication method of the present invention. However, the implementation device of the motor imagery brain-computer interface communication method of the present invention includes, but is not limited to, the structure of the motor imagery brain-computer interface communication device 100 listed in this embodiment. All structural modifications and substitutions of the prior art made according to the principles of the present invention are included within the protection scope of the present invention.

[0071] In this application, the motor imagination brain-computer interface communication device 100 can implement the motor imagination brain-computer interface communication method described in this embodiment. Therefore, the specific implementation functions of each module of the motor imagination brain-computer interface communication device 100 are described in detail in the motor imagination brain-computer interface communication method, and will not be repeated here.

[0072] However, the implementation device of the motor imagination brain-computer interface communication method described in this invention includes, but is not limited to, the motor imagination brain-computer interface communication device 100 listed in this embodiment. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the above-mentioned division of functional units and modules is only used as an example. In actual applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system / device is divided into different functional units or modules to complete all or part of the functions described above.

[0073] See Figure 4As shown, this embodiment of the invention also provides an electronic device 101, which is preferably a cloud server. The electronic device 101 includes a memory 1001 and a processor 1002. The memory 1001 stores a computer program; the processor 1002 is communicatively connected to the memory 1001, and when the computer program is invoked, it implements the steps of the motor imagery brain-computer interface communication method described in this invention.

[0074] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the motor imagery brain-computer interface communication method described in this invention.

[0075] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0076] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0077] like Figure 5 As shown, this embodiment of the invention also provides a motor imagination brain-computer interface communication system 1, which includes an electronic device 101 (i.e., cloud server 10) as described above, at least one EEG acquisition device 20, at least one computing device 30, and at least one external device 40.

[0078] Wherein: the EEG acquisition device 20 is connected to the computing device 30, and is used to acquire EEG data of the source domain subjects and transmit the acquired EEG data of the source domain subjects to the corresponding computing device 30; each computing device 30 is connected to the electronic device 101, and is used to classify the EEG data received from the EEG acquisition device 20 into labeled EEG data, transmit the labeled EEG data to the electronic device 101, and receive the target domain model corresponding to each source domain subject from the electronic device 101, and generate control instructions to control the external device 40 based on the target domain model; each external device 40 is connected to the corresponding computing device 30 and operates based on the control instructions received from the computing device 30.

[0079] In this embodiment, the motor imagery brain-computer interface communication system 1 includes an EEG acquisition device 20, a host computer computing device 30 containing corresponding signal processing and motor imagery recognition algorithms, a cloud server 10, and an external device 40 that needs to transmit instructions through the brain-computer interface to work.

[0080] This embodiment is the first to deploy transfer learning in a real-world motor imagery brain-computer interface system, which can be applied in fields such as medical rehabilitation and exoskeletons. Training data from all source domain subjects can be stored on cloud server 10. Whenever training data from a new user is acquired, cloud server 10 automatically updates the models of all source domain subjects, allowing all users to benefit from the performance improvements brought about by increased data, thus accelerating the practical application of motor imagery brain-computer interfaces in key areas such as medical rehabilitation and exoskeletons.

[0081] For example, the motor imagery brain-computer interface communication system 1 in this embodiment needs to be used to control an exoskeleton device. Both the EEG signal acquisition device and the exoskeleton device are connected to the computing device 30, which communicates with the cloud server 10 via a network.

[0082] Paralyzed patients control the exoskeleton's left turn, right turn, forward movement, and stop by imagining movements of their left hand, right hand, and feet, as well as a resting state. Before use, patients imagine holding a ball in their left hand, holding a ball in their right hand, gripping the ground with their feet, and remaining at rest, following on-screen prompts. Each state is repeated 10 times, for a total of 40 repetitions. This step typically takes only 3-5 minutes. Afterward, the host computer packages the collected tagged EEG data in MAT format and sends it to a cloud server.

[0083] On cloud server 10, new user data and existing data jointly perform the model update process, with 80% of the labeled data used as the training set and 20% as the validation set. In each training round, several users are randomly selected from all users as the mixed source domain, and their training data is used to train a mixed source domain model. The feature extractor parameters of this model are then fed into the next round as initialization parameters for the next round's model's feature extractor. Simultaneously, the classifier is randomly initialized in each round. The source domain model obtained in each round is validated on the corresponding subject's validation set, retaining the parameters of the model with better performance. After all mixed source domain training is completed, all subjects receive the updated new model.

[0084] The new model is then fine-tuned using the training set data from each subject (with a smaller learning rate). Because the amount of data collected from each subject is small, the entire transfer learning process takes about the same amount of time as the traditional method of training a single subject with hundreds of experiments. However, the same amount of time can be used to update the models of all existing subjects, and the classification accuracy of each model is much higher than that of the traditional method.

[0085] In this embodiment, the model structure can typically adopt relatively mature models, such as DeepConvNet and EEGNet. These models have been tested by many researchers in laboratory environments and have good results in classifying and recognizing motor imagery brain-computer interface signals. At the same time, transfer learning can improve the classification accuracy of these models by 20% (from about 60% to about 80% for four-class classification), which meets the accuracy requirements of real-world applications.

[0086] In this embodiment, the training time of the model is related to the configuration of the cloud server 10. Using the latest commercial server CPUs and GPUs, the update of all subject models can be completed within 10 minutes. Therefore, the entire preparation process before each patient's first use takes only 15 minutes. The cloud server 10 sends the updated model to all patients' devices. Patients who have already used the device can use the latest updated model without waiting, effectively improving the update time of the subject models. Furthermore, the source domain subjects are no longer manually selected, but rather automatically selected by the system from among all existing subjects, choosing those with the highest match to the target subjects. This results in a larger volume of source domain data with higher similarity to the target subject's EEG data, effectively improving the accuracy of each model.

[0087] In this embodiment, after the user's host computer receives the model sent by the cloud server 10, it can begin using the exoskeleton. During use, the EEG acquisition device 20 remains operational, preprocessing the acquired signals as a sample every 2 seconds and sending them to the model for recognition. The patient can imagine movements of different body parts according to their needs. For example, when the patient needs to move forward, they imagine their legs moving. The current movement state recognized by the EEG acquisition device 20 will change from a resting state to forward movement. Upon receiving the forward movement signal, the exoskeleton device will maintain forward movement until the patient stops imagining leg movements and returns to a resting state, at which point the exoskeleton device will stop moving forward.

[0088] In summary, in this invention, the source domain subjects are no longer manually selected, but rather the subjects automatically chosen by the system from among all existing subjects who have the highest matching degree with the target subjects. This results in a larger volume of source domain data with higher similarity to the target subjects' EEG data, leading to a greater improvement in classification performance and enhancing the classification performance of the motor imagery brain-computer interface. Furthermore, whenever new subjects are introduced into the dataset, the models of all already trained subjects are updated during training for these new subjects. The classification accuracy of all subjects in the entire system improves as the number of subjects collected increases, enabling the deployment of the motor imagery brain-computer interface in reality. This also solves the problem of the aforementioned traditional methods being difficult to apply in practice. This invention effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0089] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A motor imagery brain-computer interface communication method, characterized in that, The motor imagery brain-computer interface communication method includes: Acquire labeled EEG data and divide the EEG data into a training set and a validation set; Several source domain subjects were randomly selected as the hybrid domain, and a hybrid source domain model was obtained based on the EEG data of the training set corresponding to the hybrid domain. The hybrid source domain model is trained based on the EEG data of subjects in each source domain in the training set to obtain target domain models corresponding to subjects in each source domain. The step of randomly selecting several source domain subjects as the mixed domain, and obtaining the mixed source domain model based on the EEG data of the training set corresponding to the mixed domain includes: Several source domain subjects were randomly selected as the initial mixed domain, and the first mixed source domain model was obtained based on the EEG data of the training set corresponding to the initial mixed domain. Extract the feature extractor parameters of the first hybrid source domain model and use them as the initialization parameters of the hybrid domain model feature extractor in the next round. Several source domain subjects were randomly selected as the second mixed domain, and the second mixed source domain model was obtained based on the EEG data of the training set corresponding to the second mixed domain and the feature extractor parameters of the first mixed source domain model. By analogy, the Nth mixing domain and the Nth mixing source domain model corresponding to the Nth mixing domain are obtained; Also includes: Based on the EEG data of subjects in each source domain in the validation set, the first mixed source domain model, the second mixed source domain model, ..., the Nth mixed source domain model are validated, and the validation results are output. The verification results are compared with the historical mixed source domain models of the corresponding source domain subjects to obtain the optimal mixed source domain model corresponding to the source domain subjects. The step of training the hybrid source domain model based on the EEG data of subjects in each source domain of the training set includes: training the optimal hybrid source domain model corresponding to the subject in each source domain based on the EEG data of subjects in each source domain of the training set.

2. The motor imagery brain-computer interface communication method of claim 1, wherein, The parameters of the classifier in the first mixed source domain model, the second mixed source domain model, ..., the Nth mixed source domain model are randomly initialized.

3. A motor imagery brain-computer interface communication device, characterized by, The motor imagery brain-computer interface communication device includes: The dataset module is used to acquire labeled EEG data and divide the EEG data into a training set and a validation set. The hybrid domain model module randomly selects several source domain subjects as hybrid domains, and obtains a hybrid source domain model based on the EEG data of the training set corresponding to the hybrid domains. The random selection of several source domain subjects as hybrid domains and obtaining the hybrid source domain model based on the EEG data of the training set corresponding to the hybrid domains includes: randomly selecting several source domain subjects as initial hybrid domains, and obtaining a first hybrid source domain model based on the EEG data of the training set corresponding to the initial hybrid domains; extracting the feature extractor parameters of the first hybrid source domain model as initialization parameters for the feature extractor of the next hybrid domain model. Several source domain subjects are randomly selected as the second mixed domain, and a second mixed source domain model is obtained based on the EEG data of the training set corresponding to the second mixed domain and the feature extractor parameters of the first mixed source domain model; this process is repeated to obtain the Nth mixed domain and the Nth mixed source domain model corresponding to the Nth mixed domain; the first mixed source domain model, the second mixed source domain model, ..., the Nth mixed source domain model are validated based on the EEG data of each source domain subject in the validation set, and the validation results are output; the validation results are compared with the historical mixed source domain models of the corresponding source domain subjects to obtain the optimal mixed source domain model corresponding to the source domain subject; the step of training the mixed source domain model based on the EEG data of each source domain subject in the training set includes: training the optimal mixed source domain model corresponding to the source domain subject based on the EEG data of each source domain subject in the training set; The target domain model module trains the hybrid source domain model based on the EEG data of subjects from each source domain in the training set, thereby obtaining target domain models corresponding to subjects from each source domain.

4. A computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the program implements the steps of the motor imagery brain-computer interface communication method according to any one of claims 1 to 2.

5. An electronic device, comprising: The electronic device includes: A memory that stores a computer program; The processor, which is communicatively connected to the memory, implements the steps of the motor imagery brain-computer interface communication method according to any one of claims 1 to 2 when the computer program is invoked.

6. A motor imagery brain-computer interface communication system characterized by, The motor imagery brain-computer interface communication system includes the electronic device as described in claim 5, at least one EEG acquisition device, at least one computing device, and at least one external device; wherein: The electronic device is a cloud server; The EEG acquisition device is connected to the computing device and is used to acquire EEG data of the source domain subject and transmit the acquired EEG data of the source domain subject to the corresponding computing device. Each of the aforementioned computing devices is connected to the cloud server and is used to classify and label the EEG data received from the EEG acquisition device to form labeled EEG data, transmit the labeled EEG data to the cloud server, receive the target domain model corresponding to each source domain subject from the cloud server, and generate control commands to control the external device based on the target domain model. Each of the external devices is connected to the corresponding computing device and operates based on control commands received from the computing device.

Images