Fusion enhanced online motor imagery intention recognition system and training method thereof

Abstract

The invention discloses a fusion enhancement online motor imagery intention recognition system and a training method thereof.The method comprises the steps that a first training data set and a second training data set of motor imagery of a user are collected, the first training data set is subjected to a fusion enhancement method containing at least one of noise enhancement, phase enhancement, frequency band enhancement and channel enhancement, and the fusion enhancement method can be implemented independently or in series; generating a first enhancement data set based on the initial fusion enhancement parameters, and training an identification model in a first training stage; in the evaluation stage, identifying a second training data set by using the model, and generating a second enhanced data set for correctly identified data by using the same fusion enhancement method and parameters; in the second training stage, the two types of enhanced data sets are combined to re-train the model for optimization; and meanwhile, traversing the fusion enhancement parameter space, and determining optimal parameters containing various enhancement parameters. Through multiple selectable data enhancement, online loop optimization and automatic parameter optimization, the problems that motor imagery electroencephalogram data is small in sample size, uneven in distribution and high in timeliness are effectively solved.

Description

Technical Field

[0001] The present application belongs to the field of brain-computer interface technology, and specifically relates to a fusion-enhanced online motor imagery intention recognition system and a training method thereof. Background Art

[0002] Brain-computer interface technology is a control method that does not rely on peripheral or muscular control. Users can send control commands to the outside world directly through brain activity. Motor imagery is a common brain-computer interface paradigm. The user only needs to attempt to perform an action, and the system will recognize the user's intention. Due to the time-varying nature of brain activity during motor imagery, different users will have different EEG activity characteristics when performing the same motor imagery task at different times. Users usually need to complete a training phase, and the system can only train the classifier after collecting sufficient data to recognize motor intention under the current conditions.

[0003] Motor imagery-based brain-computer interface systems can be used in stroke rehabilitation. However, in practical applications, considering the patient experience, the amount of training data is limited and unevenly distributed. Previous approaches have included optimizing classifiers and using public data to train large, cross-subject universal models. None of these approaches can fundamentally address the issue of limited single-session training data. Summary of the Invention

[0004] The technical purpose of this application is to address the problem that the current brain-computer interface system is unable to fundamentally solve the problem of insufficient single training data, and to develop a data-enhanced online motor imagery intention recognition system and its training method that integrates multiple methods, and to improve model accuracy by utilizing data enhancement and cyclic online training solutions.

[0005] In order to achieve the above technical objectives, this application adopts the following technical solutions.

[0006] In a first aspect, an embodiment of the present application provides a training method for a fusion-enhanced online motor imagery intention recognition system, comprising:

[0007] Collecting a first training data set and a second training data set of a user performing motor imagery;

[0008] For the first training data set, based on the initial fusion enhancement parameters, a fusion enhancement method is used to generate a first enhanced training data set;

[0009] In a first training phase, an initial intent recognition model is trained based on the first enhanced training data set; the fusion enhancement method includes at least one of noise enhancement, phase enhancement, frequency band enhancement, and channel enhancement, and each enhancement method is implemented separately or in series;

[0010] In the evaluation phase, the trained initial intent recognition model is used to identify the second training data set, and for data with correct recognition results, the fusion enhancement method is used based on the initial fusion enhancement parameters to generate a second enhanced training data set;

[0011] In a second training phase, the initial intent recognition model is retrained based on the first enhanced training data set and the second enhanced training data set to obtain an optimized intent recognition model;

[0012] In the process of retraining the initial intent recognition model based on the first enhanced training data set and the second enhanced training data set, the fusion enhancement parameter space is traversed to determine the current optimal fusion enhancement parameters, and the fusion enhancement parameters include noise enhancement parameters, phase enhancement parameters, frequency band enhancement parameters, channel enhancement parameters and / or series enhancement parameters.

[0013] In a second aspect, the present application provides a fusion-enhanced online motor imagery intention recognition system, comprising:

[0014] A data collection module is used to collect a first training data set and a second training data set of a user performing motor imagery;

[0015] A data enhancement module, configured to generate a first enhanced training data set using a fusion enhancement method based on initial fusion enhancement parameters for the first training data set;

[0016] A model training module is configured to, in a first training phase, train an initial intent recognition model based on the first enhanced training data set; the fusion enhancement method includes at least one of noise enhancement, phase enhancement, frequency band enhancement, and channel enhancement, each enhancement method being implemented individually or in series; and, in an evaluation phase, use the trained initial intent recognition model to identify the second training data set;

[0017] The data enhancement module is further configured to generate a second enhanced training data set using the fusion enhancement method based on the initial fusion enhancement parameters for the data with correct recognition results during the evaluation phase;

[0018] The model training module is further configured to, in a second training phase, retrain the initial intent recognition model based on the first enhanced training data set and the second enhanced training data set to obtain an optimized intent recognition model;

[0019] An enhancement parameter optimization module is used to traverse the fusion enhancement parameter space to determine the current optimal fusion enhancement parameters during the process of retraining the initial intent recognition model based on the first enhancement training data set and the second enhancement training data set. The fusion enhancement parameters include noise enhancement parameters, phase enhancement parameters, frequency band enhancement parameters, channel enhancement parameters and / or series enhancement parameters.

[0020] Compared with the existing technology, the fusion-enhanced online motor imagery intention recognition system and its training method provided in the embodiment of the present application have the following beneficial technical effects: through a variety of optional data enhancement, online loop optimization and automatic parameter optimization, it effectively solves the problems of small sample size, uneven distribution and strong time variability of motor imagery EEG data. The data volume can be expanded while retaining the key features of the original signal, and the recognition model's generalization ability and recognition accuracy for EEG differences of different users can be improved; in the online stage, dynamic fusion and retraining of correct recognition data are achieved to achieve continuous iterative optimization of model accuracy; the parameter optimization mechanism takes into account both recognition accuracy and computational efficiency through global traversal and statistical screening, and finally forms a brain-computer interface recognition solution that takes into account data utilization efficiency, model adaptability and clinical practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present application in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only schematic and are used to help understand the present application. They do not specifically limit the shapes and proportional dimensions of the components of the present application. Those skilled in the art can select various possible shapes and proportional dimensions to implement the present application according to the specific circumstances under the guidance of the present application. In the drawings:

[0022] Figure 1 A schematic diagram of the principle framework of the training method for the fusion-enhanced online motor imagery intention recognition system provided in the embodiment;

[0023] Figure 2 A schematic diagram of a fusion enhancement parameter optimization process in a training method for a fusion enhancement online motor imagery intention recognition system provided in an embodiment;

[0024] Figure 3 A structural diagram of the fusion-enhanced online motor imagery intention recognition system provided in the embodiment. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0026] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more features.

[0027] In order to solve the problem that the current motor imagery recognition method cannot fundamentally solve the problem of insufficient single training data, the embodiment of the present application proposes a multi-method fusion data-enhanced online motor imagery intention recognition system and its training method, which uses data enhancement and cyclic online training solutions to improve model accuracy.

[0028] Embodiment 1: This embodiment provides a training method for a fusion-enhanced online motor imagery intention recognition system, comprising the following steps:

[0029] Collecting a first training data set and a second training data set of a user performing motor imagery;

[0030] For the first training data set, based on the initial fusion enhancement parameters, a fusion enhancement method is used to generate a first enhanced training data set; the fusion enhancement method includes at least one of noise enhancement, phase enhancement, frequency band enhancement and channel enhancement methods, and each enhancement method is implemented separately or in series;

[0031] In the first training phase, an initial intent recognition model is trained based on the first enhanced training data set. In the evaluation phase, the trained initial intent recognition model is used to recognize the second training data set. For data with correct recognition results, a fusion enhancement method is used based on the initial fusion enhancement parameters to generate a second enhanced training data set.

[0032] In the second training phase, the initial intent recognition model is retrained based on the first enhanced training data set and the second enhanced training data set to obtain an optimized intent recognition model;

[0033] In the process of retraining the initial intent recognition model based on the first enhanced training data set and the second enhanced training data set, the fusion enhancement parameter space is traversed to determine the current optimal fusion enhancement parameters, which include noise enhancement parameters, phase enhancement parameters, frequency band enhancement parameters, channel enhancement parameters and / or series enhancement parameters.

[0034] In some embodiments, the first training data set is offline data of a user performing a motor imagery task, and the second training data set is online data of a user performing a motor imagery task once. Figure 1 The offline and online phases shown in the figure below: The offline phase trains the initial recognition model (the current recognition model). The online phase uses this initial recognition model to identify the user's motion intention and optimizes the model in real time using new data. Furthermore, the online phase includes a fusion-enhanced parameter optimization phase, which iteratively traverses the entire parameter space. A specific method is then used to find the optimal parameters.

[0035] Combine Figure 1 As shown, in some embodiments, the offline phase includes the following steps:

[0036] 1. Collect offline data: The system prompts the user to complete a prescribed motor imagery task and collects data. This data can be scalp electroencephalogram (EEG) data, covering at least the motor areas on both sides of the head. After the motor imagery training is completed, a raw offline dataset is generated.

[0037] For example, EEG data is commonly used, consisting of 16 channels and meeting the 10-20 position standard. The motor imagery task can be a binary classification task involving imagining the left hand and right hand clenched, or a three-class task involving imagining the left hand, right hand clenched, and resting state. Tasks should be evenly distributed during the offline phase. The total length of the EEG data can be determined based on system performance and actual conditions. If the system performance is sufficiently good and the recognition accuracy is high enough, the number of offline phases can be reduced, shortening training time and improving the user experience.

[0038] As an example, single offline data is two-dimensional, the first dimension is the lead, and the second dimension is the data point. When the sampling rate is 250 Hz, 4 seconds, and the number of leads is 16, the dimension is 16×1000.

[0039] Optionally, the offline phase also includes:

[0040] 2. Perform data preprocessing on the offline data set: Preprocessing includes steps such as bandpass filtering, notch filtering, and outlier processing to generate a preprocessed offline data set.

[0041] In this application, the EEG signal is divided into three frequency bands, namely the first frequency band 0Hz-8Hz, the second frequency band 8Hz-25Hz, and the third frequency band 25Hz-35Hz.

[0042] In a specific embodiment, data preprocessing may specifically include: using a bandpass filter with a frequency band of 3Hz-35Hz and a notch filter at 50Hz, and the filtering method may be an 8th-order Butterworth method. Outlier identification is then performed. If an outlier is identified, the single offline data is discarded. The outlier identification method may be any of the following:

[0043] (1) Calculate the standard deviation of each lead in the single offline data. If the standard deviation is greater than 100 μV, it is identified as an outlier.

[0044] (2) Calculate the absolute value of the data of each lead in a single offline data, and calculate the number of data with an absolute value exceeding 200μV. If the data exceeding 200μV in a single lead exceeds a preset proportion of the total amount (for example, the preset proportion is set to 30%), it is identified as an abnormal value.

[0045] (3) In a single offline data, processing is performed according to the frequency band:

[0046] ①The average energy of a single lead in the first, second and third frequency bands.

[0047] ② Calculate the total mean and standard deviation of the energy mean of all leads in a single frequency band, and calculate the single energy deviation value of each lead.

[0048] ;

[0049] in, is the energy deviation value of lead i in frequency band j, The average energy of lead i in frequency band j, is the number of leads, is the sum of the average energy of all leads in frequency band j, and abs is the absolute value calculation mark.

[0050] If the single energy deviation value of each lead is greater than the preset multiple of the standard deviation (the preset multiple of the standard deviation can be set to 6 times), it is identified as an abnormal value.

[0051] In an embodiment, the data fusion enhancement method may include four data enhancement methods: noise enhancement, phase enhancement, frequency band enhancement, and channel enhancement, which can effectively increase the amount of data, generate an enhanced offline data set, and effectively improve the recognition accuracy of the classifier.

[0052] In a specific embodiment, any one of the four data enhancement methods of noise enhancement, phase enhancement, frequency band enhancement and channel enhancement can be used for data enhancement, or all fusion enhancement methods can be used. Further, cascade enhancement can be performed, in which channel enhancement acts on the data after at least two fusion enhancement methods of noise enhancement, phase enhancement or frequency band enhancement are used to further balance the sample size.

[0053] In this embodiment, a recognition model is trained using a first enhanced training dataset (or an enhanced offline dataset) to generate an initial intention recognition model (i.e., an initial motor imagery intention recognition model, referred to as the "initial recognition model"). After offline model training is complete, an online optimization phase (or simply the "online phase") is performed. During this phase, the fused enhanced online motor imagery intention recognition system identifies EEG data after each user performs a motor imagery task and outputs a recognition result.

[0054] In an embodiment, the online phase may include the following steps:

[0055] 1. Collecting Single-Session Raw Online Data: Collect data from a single motor imagery task. This single task can be left-hand motor imagery. The task category must be consistent with the offline phase. For example, if performing a binary classification task of left / right hand, the single task would be left or right hand imagery.

[0056] 2. Data Preprocessing: Preprocessing includes steps such as bandpass filtering, notch filtering, and outlier processing to generate single-shot raw filtered online data. In this embodiment, the preprocessing method in the online optimization phase is the same as that in the offline model phase. If the data is marked as anomaly, no further processing is required.

[0057] 3. Identify Single Movement Intention: Utilizing the initial intention recognition model, the user's movement intention is identified from the EEG data, resulting in a single recognition result. In some embodiments, for a binary classification task of left / right hand, the recognition result is either left or right. Online tasks typically have a target, which the user must complete. If the recognition result matches the target, the online task is considered correct; otherwise, it is considered an error.

[0058] 4. Data Fusion Enhancement: Using the same fusion enhancement method and fusion enhancement parameters as in the offline training phase, a single raw filtered online data is generated. In this embodiment, the single online data can be enhanced using the same method and fusion enhancement parameter set as in the offline model phase to generate multiple sets of data.

[0059] 5. Optimize the recognition model: Retrain the recognition model using the enhanced offline dataset and all the single-shot raw filtered online data generated online. In some cases, the correct single-shot result from the online phase can be obtained. If the single-shot recognition result is the same as the correct single-shot result, the recognition model optimization step is performed; if not, the step is skipped.

[0060] In some embodiments, the enhanced single-shot online data is combined with other previously generated enhanced single-shot online data to form an enhanced online dataset, which is then combined with the enhanced offline dataset to form an enhanced optimized dataset (i.e., a second enhanced training dataset). This dataset is used to retrain the initial recognition model, further improving model recognition accuracy by increasing the amount of training data.

[0061] 6. Continue the above steps of the online optimization phase until the online phase is completed.

[0062] like Figure 2 As shown, in some embodiments, the fusion enhancement parameter optimization stage includes the following steps:

[0063] 1. Data preparation: Prepare data for fusion enhancement parameter optimization. The data should be clinical data, with gender and age distribution similar to the clinical situation, and form a certain scale. The data quality can be checked and only the data with good quality is retained. A set of clinical data is that the user completes the offline stage first and then immediately completes the online stage to form a complete set of data.

[0064] 2. A set of clinical data completes the following processing steps:

[0065] 1. Generate initial fusion enhancement parameters: traverse all possibilities of fusion enhancement parameters, and generate a single fusion enhancement parameter group each time. During the entire fusion enhancement parameter optimization stage, the single fusion enhancement parameter group generated each time is not repeated.

[0066] 2. For the offline data in a group of data, generate enhanced data according to the single fusion enhancement parameter group to perform the offline model training phase: use the generated single fusion enhancement parameter group to complete the offline phase and obtain the initial recognition model.

[0067] 3. Execute offline model training phase: Use offline data (first training data set) to train the initial recognition model.

[0068] 4. Perform online optimization: Use the initial recognition model to perform recognition, record the enhancement parameters and scores, calculate the individual recognition results, and calculate the online recognition accuracy, using this as the score corresponding to the individual data. Record the individual fusion enhancement parameter set and score, and record the total time consumed by all offline model training phases and online optimization phases.

[0069] 5. Repeat the above processing steps to complete the online accuracy calculation and recording of all data.

[0070] 3. Repeat the above steps until all fusion enhancement parameters are traversed.

[0071] Parameter optimization can be achieved by calculating the average accuracy of all data for each single-pass fusion enhancement parameter group, with the average accuracy ranging from 0 to 100. Cluster the data for all single-pass fusion enhancement parameter groups. After clustering, find the cluster with the highest accuracy at the cluster center. From this cluster, find the single-pass fusion enhancement parameter group with the shortest search time, and select it as the preferred single-pass fusion enhancement parameter group.

[0072] Specifically, in some embodiments, the method of traversing the fusion enhancement parameter space to determine the current optimal fusion enhancement parameter may employ the following two methods to optimize the parameter group, including:

[0073] Method 1: Calculate the average accuracy of the data of each fusion enhancement parameter group in the evaluation phase, screen out the fusion enhancement parameter group with no significant difference from the highest accuracy, and select the one with the shortest time consumption as the optimal fusion enhancement parameter group; or calculate the average accuracy of the data of each fusion enhancement parameter group in the evaluation phase, perform K-means clustering on the average accuracy of the data of each fusion enhancement parameter group in the evaluation phase, and select the fusion enhancement parameter group with the shortest time consumption in the category with the highest cluster center.

[0074] In the embodiment, method 1 or method 2 can be used alone to select the optimal single fusion enhancement parameter group. Alternatively, both methods can be used together to find the optimal single fusion enhancement parameter group with the shortest time consumption from the two methods, and use it as the final optimal single fusion enhancement parameter group.

[0075] As an example, method one includes:

[0076] 1. Each single fusion enhancement parameter group contains Online evaluation scores are obtained by applying the above method to each set of data. The average online evaluation scores of all single fusion enhancement parameter groups are calculated, and the online evaluation scores of all fusion enhancement parameter groups corresponding to the maximum online evaluation score are found, which are recorded as .

[0077] 2. The nth single fusion enhancement parameter group corresponds to Online assessment scores ,and A significance test is performed to detect whether there is a significant difference between the two sets of data. Specifically, the significance test can use the ttest method, ignoring the normal distribution of the sample.

[0078] 3. In all Among the groups without significant differences, the single fusion enhancement parameter group with the shortest average time consumption was found as the optimal fusion enhancement parameter group.

[0079] As an example, method two includes:

[0080] 1. Calculate the average of all online evaluation scores for each single fusion enhancement parameter group.

[0081] 2. Cluster all online assessment scores. Specifically, the clustering method can be K-means clustering, with 10 initial cluster points set at 100%, 90%, 80%, and 10%.

[0082] 3. Find the class with the highest cluster center and mark the single fusion enhancement parameter group in all classes;

[0083] 4. Repeat steps 2-3. After times, the number of times the mark is found exceeds All single fusion enhancement parameter groups. The number of elements selected as candidates is 0, then find The elements selected as the finalists are repeated until the number is not 0.

[0084] 5. Find the element with the shortest time consumption in this category as the preferred single fusion enhancement parameter group.

[0085] As examples, data augmentation methods include the following aspects:

[0086] 1. Noise enhancement: Noise enhancement adds noise interference to the data, mainly to improve the classifier's anti-interference ability. In the embodiment, noise enhancement is to add noise interference of selected frequency bands to the data.

[0087] In some embodiments, noise enhancement is performed by adding noise interference from the first and / or third frequency bands of the selected frequency bands to the data, while avoiding the primary frequency bands and brain regions associated with motor imagery EEG responses. This noise enhancement avoids the primary frequency bands and brain regions associated with motor imagery EEG responses, increasing data while minimizing the loss of valid information, thereby improving classification accuracy.

[0088] In a specific embodiment, noise enhancement may include the following steps:

[0089] 1. Generate noise template for non-main frequency band:

[0090] (1) For each single offline data in the preprocessed offline data set, perform time-frequency conversion on each lead;

[0091] (2) According to the fusion enhancement parameter group, the energy of the corresponding frequency band is retained. The frequency bands include the first frequency band, the second frequency band, and the third frequency band. You can choose whether to retain each frequency band or remove them all.

[0092] (3) The frequency domain data is restored to the time domain and re-filtered with a 3 Hz-35 Hz bandpass filter to form a noise template set 1. The dimension of the noise template set is: [lead, data point].

[0093] (4) Averaging is performed in the dimension of a single offline data to obtain a noise template.

[0094] For example, when the sampling rate is 250 Hz, the EEG data is taken 4 seconds after the start of imagination, the number of leads is 16, and the noise template dimension is 16×1000.

[0095] 2. Add noise to the data:

[0096] The expression for adding noise interference of the selected frequency band to the data in all leads is as follows:

[0097] ;

[0098] in is the data after adding noise interference of the selected frequency band, i is the lead number, is the original data of lead i, is the i-th lead data in the noise template, is the first noise addition coefficient, Add a coefficient for the second noise.

[0099] In some embodiments, based on empirical values, Can be set to 0.9. Can be set to 0.1.

[0100] In the embodiment, the noise enhancement related fusion enhancement parameters include: whether to enhance the noise in the first frequency band, whether to enhance the noise in the second frequency band, whether to enhance the noise in the third frequency band, 、 wait.

[0101] 2. Phase Enhancement: Phase enhancement involves applying corresponding phase offsets to the original data of the first, second, and third frequency bands, respectively, while restoring the frequency domain data to the time domain to obtain the corresponding first time domain data. The first time domain data are synthesized and re-bandpass filtered to form a phase-enhanced data set, and the data dimension remains consistent with the original data.

[0102] Among them, the second frequency band contains most of the EEG activities of motor imagery. In some embodiments, it is adjusted as a whole to ensure the true phase of the data and improve the recognition accuracy of the classifier.

[0103] As an example, phase enhancement includes the following steps:

[0104] (1) Perform time-frequency conversion on each lead of each single offline data in the preprocessed offline data set;

[0105] (2) According to the fusion enhancement parameter group, corresponding phase offsets are applied in the three corresponding frequency bands, and the frequency domain data is restored to the time domain;

[0106] ;

[0107] in is the xth data of the jth frequency band of the i-th lead after phase enhancement, is the xth frequency domain data point of the jth frequency band of the i-th lead, is the phase change of the j-th frequency band, is the amplitude of the xth frequency domain data point in the jth frequency band of the i-th lead.

[0108] After converting the frequency domain data of all frequency bands, new time domain data is formed.

[0109] (3) Synthesize the three frequency band data:

[0110] ;

[0111] in is all the time domain data of lead i after phase enhancement, is all the time domain data after phase enhancement of the first frequency band of lead i, is all the time domain data after phase enhancement of the second frequency band of lead i, is all the time domain data after phase enhancement of the third frequency band of lead i.

[0112] (4) Bandpass filtering is performed again to form a phase-enhanced data set with the same data dimension as the original data.

[0113] Phase enhancement related fusion enhancement parameters include: first band offset phase, second band offset phase, and third band offset phase. The offset phase is an enumerated value and can be (-π / 2, 0, π / 2). After enhancement, the total data volume is twice the original data volume. If all parameters are 0, phase enhancement is not performed and the data volume remains unchanged.

[0114] 3. Frequency band enhancement: Frequency band enhancement involves performing time-frequency conversion on the original data of the first, second, and third frequency bands, applying amplitude scaling accordingly, restoring the frequency domain data to the time domain, and obtaining the corresponding second time domain data. The second time domain data are synthesized and re-bandpass filtered to form a frequency band-enhanced data set, with the data dimension remaining consistent with the original data.

[0115] The second frequency band contains most of the EEG activities of motor imagery. In some embodiments, it is adjusted as a whole to ensure the true phase of the data and improve the recognition accuracy of the classifier.

[0116] As examples, these include:

[0117] (1) Perform time-frequency conversion on each lead of each single offline data in the preprocessed offline data set;

[0118] (2) According to the fusion enhancement parameter group, corresponding amplitude changes are applied in the three corresponding frequency bands, and the frequency domain data is restored to the time domain;

[0119] ;

[0120] in, is the xth data of the jth frequency band of the i-th lead after frequency band enhancement, is the xth frequency domain data point of the jth frequency band of the i-th lead, is the phase change of the j-th frequency band, is the amplitude of the xth frequency domain data point in the jth frequency band of the i-th lead, is the amplitude change degree of the jth frequency band; after converting the frequency domain data of all frequency bands, new time domain data are formed.

[0121] (3) Synthesize the three frequency band data:

[0122] ;

[0123] in, is all the time domain data of lead i after frequency band enhancement, All time domain data after frequency band enhancement for the first frequency band of lead i, All time domain data after frequency band enhancement for the second frequency band of lead i, All time domain data after frequency band enhancement of the third frequency band of the i-th lead.

[0124] (4) Re-bandpass filtering is performed to form a frequency band enhanced data set with the same data dimension as the original data.

[0125] Frequency band enhancement related fusion enhancement parameters: Amplitude change range for the first frequency band, Amplitude change range for the second frequency band, and Amplitude change range for the third frequency band. Amplitude change is an enumeration value and can be (0.8, 1, or 1.2). The amount of data after enhancement is double the original data. When all parameters are 1, frequency band enhancement is not performed and the data volume remains unchanged.

[0126] Channel Enhancement: Channel enhancement involves swapping the electrode positions of the left and right brain regions and / or the anterior and posterior brain regions to generate new data. In this embodiment, data channels can be swapped. Channel changes must conform to the general patterns of motor imagery EEG activity. This ensures data validity while increasing data volume and balancing data distribution across different tasks, ultimately improving classifier recognition accuracy.

[0127] In some embodiments, channel enhancement may employ at least one of the following solutions:

[0128] ① Swap the left and right brain regions: keep the position of the electrodes in the central brain area unchanged (such as: Pz, Cz, etc.), swap the positions of the electrodes in the left and right brain regions (such as: swap the positions of C3 and C4), and change the data labels at the same time (such as: change the label from left hand to right hand), generate new data, and balance the data volume of different tasks (such as left and right hand).

[0129] Specifically, when the EEG leads are C3, C4, Cz, Oz, O1, O2, Pz, P1, and P2, the left motor brain area data leads are C3, O1, and P1, the right motor brain area leads are C4, O2, and P2, and the middle area leads are Pz, Cz, and Oz. As an example, when swapping, Pz, Cz, and Oz remain unchanged, and the other leads are mirror-swapped: C3 and C4 are swapped, O1 and O2 are swapped, and P1 and P2 are swapped. In addition, the corresponding labels of the data are swapped. If the original label is left-hand motor imagery, the new label is right-hand motor imagery. If the original label is resting state, the new label is still resting state.

[0130] ② Interchange between front and back brain regions: Interchange the EEG data of the frontal lobe region with the EEG data of the occipital lobe region to generate new data, while avoiding the main motor brain areas. While ensuring data quality, it increases the data volume, which is conducive to improving the accuracy of the classifier.

[0131] Specifically, when the EEG leads are C3, C4, Cz, Oz, O1, O2, FPz, FP1, and FP2, the frontal leads are FPz, FP1, and FP2, and the occipital leads are Oz, O1, and O2. For example, when swapping leads, C3, C4, and Cz remain unchanged, while the leads in the other two regions are swapped: FP1 and O1, FP2 and O2, and FPz and Oz. The data labels remain unchanged.

[0132] Channel enhancement related fusion enhancement parameters: whether to swap leads between left and right brain regions, and whether to swap leads between front and back brain regions.

[0133] In some embodiments, if the data distribution is uneven, for example, in a binary classification task of left / right hand classification, the collected data contains 20 samples of left-hand data and only 10 samples of right-hand data. Then, 10 left-hand samples can be randomly selected and swapped between the left and right brain regions. After the swap, the left and right hand data are both 20, achieving a balanced sample size.

[0134] 5. Cascade enhancement: Cascade enhancement is channel enhancement applied to data after noise enhancement, phase enhancement, or frequency band enhancement to further balance the sample size.

[0135] In a specific embodiment, the above enhancement methods can enhance the original data separately or perform superposition enhancement.

[0136] In the embodiment, the channel enhancement method can be used as a cascade method to act on data processed by other enhancement methods to achieve a balance in the sample size of different tasks, which is beneficial to improving the classifier performance.

[0137] Serial enhancement related fusion enhancement parameters: whether to enhance noise, whether to enhance phase, whether to enhance frequency band, whether to enhance channels, and whether to retain original data.

[0138] In some embodiments, if the data distribution is uneven, for example, in a binary classification task of left / right hand classification, the collected data contains 20 samples of left-hand data and only 10 samples of right-hand data. Then, 10 left-hand samples can be randomly selected and swapped between the left and right brain regions. After the swap, the left and right hand data are both 20, achieving a balanced sample size.

[0139] This application performs data enhancement, which can effectively improve the recognition accuracy of the classifier.

[0140] Example 2: Based on the same inventive concept as the training method of the fusion enhanced online motor imagery intention recognition system provided in the above embodiment, the embodiment of the present application also provides a fusion enhanced online motor imagery intention recognition system, such as Figure 3 As shown, it includes: data acquisition module, data enhancement module, model training module and enhanced parameter optimization module.

[0141] A data collection module is used to collect a first training data set and a second training data set of a user performing motor imagery;

[0142] a data enhancement module configured to generate a first enhanced training data set using a fusion enhancement method based on the initial fusion enhancement parameters for the first training data set; the fusion enhancement method includes at least one of noise enhancement, phase enhancement, frequency band enhancement, and channel enhancement, and each enhancement method is implemented individually or in series;

[0143] A model training module is configured to train an initial intent recognition model based on a first enhanced training data set in a first training phase; and to recognize a second training data set using the trained initial intent recognition model in an evaluation phase;

[0144] The data enhancement module is further used to generate a second enhanced training data set using a fusion enhancement method based on the initial fusion enhancement parameters for the data with correct recognition results during the evaluation phase;

[0145] The model training module is further configured to retrain the initial intent recognition model based on the first enhanced training data set and the second enhanced training data set in a second training phase to obtain an optimized intent recognition model;

[0146] The enhancement parameter optimization module is used to traverse the fusion enhancement parameter space to determine the current optimal fusion enhancement parameters during the process of retraining the initial intent recognition model based on the first enhancement training data set and the second enhancement training data set. The fusion enhancement parameters include noise enhancement parameters, phase enhancement parameters, frequency band enhancement parameters, channel enhancement parameters and / or series enhancement parameters.

[0147] This embodiment provides a fusion-enhanced online motor imagery intention recognition system. By integrating multiple data enhancement methods, it effectively expands the scale and balances the distribution of motor imagery EEG data without relying on additional collected data, solving the problem of insufficient training data in clinical scenarios. In the online phase, through dynamic fusion of correctly identified data and model retraining, the model can continuously adapt to the time-varying nature of EEG signals, improving recognition stability. The parameter optimization mechanism, through global traversal and screening, ensures recognition effectiveness while taking into account processing efficiency. Overall, it enhances the system's adaptability to the EEG characteristics of different users and at different time periods, optimizing the reliability and practicality of motor imagery intention recognition.

[0148] The above is a detailed introduction to the fusion-enhanced online motor imagery intention recognition system and its training method provided by this application. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the concept of this application and should not be understood as limiting the scope of protection of this application.

Claims

1. A training method for an integrated and enhanced online motor imagery intention recognition system, characterized in that: include: Collecting a first training data set and a second training data set of a user performing motor imagery; For the first training data set, based on the initial fusion enhancement parameters, a fusion enhancement method is used to generate a first enhanced training data set; The fusion enhancement method includes at least one of noise enhancement, phase enhancement, frequency band enhancement and channel enhancement methods, and each enhancement method is implemented separately or in series; In a first training phase, an initial intent recognition model is trained based on the first enhanced training dataset; In the evaluation phase, the trained initial intent recognition model is used to identify the second training data set, and for data with correct recognition results, the fusion enhancement method is used based on the initial fusion enhancement parameters to generate a second enhanced training data set; In a second training phase, the initial intent recognition model is retrained based on the first enhanced training data set and the second enhanced training data set to obtain an optimized intent recognition model; In the process of retraining the initial intent recognition model based on the first enhanced training data set and the second enhanced training data set, the fusion enhancement parameter space is traversed to determine the current optimal fusion enhancement parameters, and the fusion enhancement parameters include noise enhancement parameters, phase enhancement parameters, frequency band enhancement parameters, channel enhancement parameters and / or series enhancement parameters.

2. The training method of the fusion-enhanced online motor imagery intention recognition system according to claim 1 is characterized in that: The first training data set is offline data of the user performing a motor imagery task, and the second training data set is online data of the user performing a motor imagery task once.

3. The training method of the fusion-enhanced online motor imagery intention recognition system according to claim 1 is characterized in that: The noise enhancement is to add noise interference of selected frequency bands to the data.

4. The training method of the fusion-enhanced online motor imagery intention recognition system according to claim 1 is characterized in that: The phase enhancement is to apply corresponding phase shifts to the original data of the first frequency band, the second frequency band and the third frequency band respectively, and restore the frequency domain data to the time domain to obtain the corresponding first time domain data; The first time domain data are synthesized and band-pass filtered again to form a phase-enhanced data set, and the data dimension is consistent with the original data.

5. The training method for the fusion-enhanced online motor imagery intention recognition system according to claim 1 is characterized in that: The frequency band enhancement is to perform time-frequency conversion on the original data of the first frequency band, the second frequency band and the third frequency band respectively, and apply amplitude scaling accordingly, restore the frequency domain data to the time domain, and obtain corresponding second time domain data; The second time domain data are synthesized and band-pass filtered again to form a frequency-band enhanced data set, and the data dimension is consistent with the original data.

6. The training method of the fusion-enhanced online motor imagery intention recognition system according to claim 1 is characterized in that: The channel enhancement is to interchange the electrode positions of the left and right brain regions and / or the electrode positions of the front and back brain regions to generate channel-enhanced data.

7. The training method of the fusion-enhanced online motor imagery intention recognition system according to claim 1 is characterized in that: The method further includes preprocessing the first training data set and / or the second training data set, specifically including: Use 3Hz-35Hz bandpass filtering and 50Hz notch filtering; When the single-lead standard deviation is greater than 100 μV or the absolute value is greater than 200 μV, the data volume exceeds the preset ratio, or the frequency band energy deviation value exceeds the preset multiple of the standard deviation, the data is discarded.

8. The training method for the fusion-enhanced online motor imagery intention recognition system according to claim 1, characterized in that: Traverse the fusion enhancement parameter space to determine the current optimal fusion enhancement parameters, including: Calculate the average accuracy of each fusion enhancement parameter group during the evaluation phase, select the fusion enhancement parameter group with no significant difference from the highest accuracy, and select the one with the shortest time consumption as the optimal fusion enhancement parameter group; Alternatively, the average accuracy of each fusion enhancement parameter group data in the evaluation phase is calculated, and K-means clustering is performed on the average accuracy of each fusion enhancement parameter group data in the evaluation phase, and the fusion enhancement parameter group with the shortest time consumption in the highest category of the cluster center is selected.

9. The training method for the fusion-enhanced online motor imagery intention recognition system according to claim 1, characterized in that: Each enhancement method is implemented in series as a series enhancement, wherein the series enhancement is the channel enhancement acting on the data after at least two fusion enhancement methods of noise enhancement, phase enhancement and frequency band enhancement are used to further balance the sample size.

10. A fusion-enhanced online motor imagery intention recognition system, characterized in that: include: A data collection module is used to collect a first training data set and a second training data set of a user performing motor imagery; A data enhancement module, configured to generate a first enhanced training data set using a fusion enhancement method based on initial fusion enhancement parameters for the first training data set; The fusion enhancement method includes at least one of noise enhancement, phase enhancement, frequency band enhancement and channel enhancement methods, and each enhancement method is implemented separately or in series; A model training module is configured to train an initial intent recognition model based on the first enhanced training data set in a first training phase; and to recognize the second training data set using the trained initial intent recognition model in an evaluation phase; The data enhancement module is further configured to generate a second enhanced training data set using the fusion enhancement method based on the initial fusion enhancement parameters for the data with correct recognition results during the evaluation phase; The model training module is further configured to, in a second training phase, retrain the initial intent recognition model based on the first enhanced training data set and the second enhanced training data set to obtain an optimized intent recognition model; An enhancement parameter optimization module is used to traverse the fusion enhancement parameter space to determine the current optimal fusion enhancement parameters during the process of retraining the initial intent recognition model based on the first enhancement training data set and the second enhancement training data set. The fusion enhancement parameters include noise enhancement parameters, phase enhancement parameters, frequency band enhancement parameters, channel enhancement parameters and / or series enhancement parameters.

Images