Method, device and equipment for issuing continuous peripheral control instructions based on electroencephalogram signals

Abstract

The present invention provides a method, device and equipment for issuing continuous peripheral control instructions based on electroencephalogram signals, wherein the method comprises: inputting real-time whole-brain signal data into an external control decoding model to obtain motion categories and motion parameters; converting motion categories and motion parameters into external control instructions, and issuing real-time instructions based on the external control instructions; the external control decoding model comprises a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder. Thus, the conversion from discrete motion imagination instructions to continuous motion imagination instructions is realized, and accurate and continuous control of external devices can be achieved. By guiding the user to imagine to a corresponding degree according to different external motion parameters, and using external motion as feedback, the accuracy of fine motion imagination classification is improved, and the fluency of electroencephalogram control of external devices and the user's interactive experience are significantly enhanced.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to a method, device and equipment for issuing continuous peripheral control instructions based on electroencephalogram signals. Background Art

[0002] Brain-computer interface (BCI) is a system that aims to establish a bidirectional information pathway between the brain and external environmental devices to control external devices or repair neural functions without relying on peripheral nerve tissue and muscle tissue. The system analyzes and interprets the signals generated by brain activity and provides necessary feedback stimulation, thereby achieving precise control of external devices or assisting in the reconstruction of neural functions. The core of BCI technology is to convert the brain's neural activity into recognizable and executable signals to complete a variety of output tasks. As the only advanced paradigm that does not rely on external stimulation and directly reflects the user's active motor consciousness, the brain-computer interface system based on motor imagery (MI) not only demonstrates its important value in exploring the brain's cognitive process, but also plays a key role in controlling external devices and promoting clinical treatment and rehabilitation. For patients who are paralyzed, amputated, or have impaired neurological function due to trauma, the motor imagery brain-computer interface system provides an innovative and effective solution. By accurately decoding the user's motor imagery intention, the MI-BCI system enables patients to control prostheses or other assistive devices, effectively compensating for their lost motor function. This technology not only improves patients' ability to take care of themselves, but also greatly improves their quality of life.

[0003] Benefiting from its non-invasiveness, portability and high cost-effectiveness, scalp electroencephalography (EEG) has become the most commonly used signal acquisition method in brain-computer interface systems. Currently, many brain-controlled devices use EEG signals as data input sources. In motor imagery, the signals generated by the brain are a combination of multiple rhythmic signals, and their activation patterns are highly similar to those during actual movement execution. These signals show significant contralateral effects in event representation, and different motor imagery intentions show significantly different event-related desynchronization and synchronization phenomena in energy amplitude. Common motor imagery tasks include the imagination of large limbs such as the right hand, left hand, and feet, while there are also studies covering more delicate movements, such as the movement imagery of elbows, fists, and fingers.

[0004] In summary, with the continuous development and diversification of MI-BCI systems, users' control requirements for brain-controlled devices are also increasing. However, existing motor imagery research mainly uses single discrete motor commands as input, and the model decoding results are also limited to the prediction of motor commands, resulting in the control of peripherals remaining at the discrete level, and unable to achieve fine-grained control of peripherals, such as speed and position adjustments. In addition, it is often difficult to obtain ideal decoding effects by directly performing continuous motor imagery tasks. Summary of the invention

[0005] The present invention provides a method, device and equipment for issuing continuous peripheral control instructions based on electroencephalogram signals, so as to solve the defects in the prior art that the existing motor imagery research mainly uses a single discrete motion instruction as input, and the model decoding result is also limited to the prediction of the motion instruction, resulting in the control of the peripheral device only staying at the discrete level, and unable to achieve fine-grained control of the peripheral device, such as the adjustment of speed and position, and it is often difficult to obtain an ideal decoding effect when directly performing continuous motor imagery tasks.

[0006] The present invention provides a method for issuing continuous peripheral control instructions based on electroencephalogram signals, comprising:

[0007] Obtain real-time whole-brain signal data;

[0008] Inputting the real-time whole-brain signal data into an external control decoding model to obtain the motion category and motion parameters output by the external control decoding model;

[0009] Converting the motion category and the motion parameter into a peripheral control instruction, and issuing a real-time instruction based on the peripheral control instruction;

[0010] The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder;

[0011] The multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimensionality reduction module is used to reduce the dimensionality of each frequency band data feature in the multi-band data features to obtain reduced dimensionality features, and the multi-head Transformer decoder is used to decode based on the reduced dimensionality features to obtain the motion category and the motion parameters.

[0012] According to a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention, the training steps of the multi-head Transformer decoder include:

[0013] Determine an initial multi-head Transformer decoder, and obtain sample whole-brain signal data, as well as motion category labels and motion parameter labels corresponding to the sample whole-brain signal data through a peripheral control training experiment;

[0014] Performing signal preprocessing on the sample whole-brain signal data to obtain preprocessed whole-brain signal data; the signal preprocessing sequentially includes bandpass filtering operation, downsampling operation, application of common mean reference method and detrending operation;

[0015] Performing multi-band feature extraction on the pre-processed whole-brain signal data to obtain a first multi-band feature, and performing dimensionality reduction on the first multi-band feature to obtain a first sample dimensionality reduction feature;

[0016] Inputting the first sample dimensionality reduction feature into the initial multi-head Transformer decoder to obtain the predicted motion category and predicted motion parameters output by the initial multi-head Transformer decoder;

[0017] Based on the difference between the predicted motion category and the motion category label, and based on the difference between the predicted motion parameter and the motion parameter label, a target loss is determined, and based on the target loss, parameters of the initial multi-head Transformer decoder are iterated to obtain the multi-head Transformer decoder.

[0018] According to a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention, the method further includes:

[0019] Inputting the sample whole-brain signal data into a bite decoding module to obtain a bite classification result output by the bite decoding module;

[0020] Based on the bite classification result, the predicted peripheral control instruction is switched between a controllable state and an uncontrollable state, and the predicted peripheral control instruction is obtained by converting the predicted motion category and the predicted motion parameter.

[0021] According to a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention, the multi-head Transformer decoder includes a Transformer category decoder;

[0022] The training steps of the Transformer class decoder include:

[0023] Determine an initial Transformer category decoder, and obtain a neighboring sample whole-brain signal data set, and a true value of a motion category corresponding to the neighboring sample whole-brain signal data set;

[0024] Performing the signal preprocessing on the adjacent sample whole-brain signal data group to obtain preprocessed adjacent whole-brain signal data;

[0025] Performing multi-band feature extraction on the pre-processed adjacent whole-brain signal data to obtain a second multi-band feature, and performing dimensionality reduction on the second multi-band feature to obtain a second sample dimensionality reduction feature;

[0026] Inputting the second sample dimensionality reduction feature into the initial Transformer category decoder to obtain a predicted motion category result output by the initial Transformer category decoder;

[0027] Based on the difference between the predicted motion category result and the true value of the motion category, a classification loss is determined, and based on the classification loss, parameters of the initial Transformer category decoder are iterated to obtain the Transformer category decoder.

[0028] According to a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention, the method further includes:

[0029] Obtaining sample EEG signal data of the subject and obtaining an initial subject training model, wherein the initial subject training model includes a signal preprocessing module, a category centroid distance calculation module, an average classification accuracy determination module, and a distance feedback value mapping module;

[0030] Based on the sample EEG signal data and the initial subject training model, the brain activity pattern of the subject is guided to obtain a subject training model;

[0031] The signal preprocessing module is used to perform the signal preprocessing on the sample EEG signal data to obtain preprocessed EEG signal data;

[0032] The category centroid distance calculation module is used to respectively determine the first category centroid distance and the second category centroid distance between the preprocessed EEG signal data and the adjacent sample whole brain signal data group;

[0033] The average classification accuracy determination module is used to determine a first average classification accuracy and a second average classification accuracy of cross-validation based on the neighboring sample whole-brain signal data group and the Transformer category decoder;

[0034] The distance feedback value mapping module is used to determine a feedback value based on the first average classification accuracy, the second average classification accuracy, the first category center of gravity distance and the second category center of gravity distance, and guide the brain activity pattern of the subject based on the feedback value.

[0035] According to a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention, the steps for determining the first category center of gravity distance and the second category center of gravity distance are as follows:

[0036] Determine the Riemann centroids of each category in the neighboring sample whole-brain signal data group, and determine a first distance mean, a first standard deviation, and a second distance mean and a second standard deviation between the preprocessed EEG signal data and the Riemann centroids in the tangent space of the category to which the preprocessed EEG signal data belongs;

[0037] Projecting the Riemann features of the preprocessed EEG signal data onto the Riemann center of gravity in the tangent space of the category to which the preprocessed EEG signal data belongs, to obtain a Riemann tangent space feature;

[0038] Determining a first Riemann distance and a second Riemann distance based on the Riemann cut space feature and the Riemann center of gravity of each category;

[0039] The first category centroid distance is determined based on the first Riemann distance, the first distance mean and the first standard deviation, and the second category centroid distance is determined based on the second Riemann distance, the second distance mean and the second standard deviation.

[0040] According to a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention, the multi-head Transformer decoder includes an encoding module and a decoding module;

[0041] The encoding module includes an input embedding layer, a position encoding layer, an output embedding layer and an encoder;

[0042] The decoding module comprises a motion category decoding branch connected to the encoding module, and a motion parameter decoding branch connected to the motion category decoding branch;

[0043] The motion category decoding branch includes a first decoder and a first activation layer;

[0044] The motion parameter decoding branch comprises a second decoder and a second activation layer.

[0045] The present invention also provides a device for issuing continuous peripheral control instructions based on EEG signals, comprising:

[0046] An acquisition unit, used for acquiring real-time whole-brain signal data;

[0047] An input unit, used to input the real-time whole-brain signal data into an external control decoding model to obtain the motion category and motion parameters output by the external control decoding model;

[0048] An instruction issuing unit, used for converting the motion category and the motion parameter into a peripheral control instruction, and issuing a real-time instruction based on the peripheral control instruction;

[0049] The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder;

[0050] The multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimensionality reduction module is used to reduce the dimensionality of each frequency band data feature in the multi-band data features to obtain reduced dimensionality features, and the multi-head Transformer decoder is used to decode based on the reduced dimensionality features to obtain the motion category and the motion parameters.

[0051] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for issuing continuous peripheral control instructions based on electroencephalogram signals as described in any one of the above-mentioned methods is implemented.

[0052] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for issuing continuous peripheral control instructions based on EEG signals as described in any one of the above is implemented.

[0053] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for issuing continuous peripheral control instructions based on electroencephalogram signals.

[0054] The method, device and equipment for issuing continuous peripheral control instructions based on EEG signals provided by the present invention input real-time whole-brain signal data into a peripheral control decoding model to obtain motion categories and motion parameters output by the peripheral control decoding model, then convert the motion categories and motion parameters into peripheral control instructions, and issue real-time instructions based on the peripheral control instructions. The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder, thereby realizing the conversion from discrete motion imagination instructions to continuous motion imagination instructions, and being able to realize accurate and continuous control of external devices, that is, realizing fine-grained control of external devices, such as speed and position adjustment, by guiding the user to imagine to a corresponding degree according to different peripheral motion parameters, and using the peripheral motion as feedback, thereby improving the accuracy of fine motion imagination classification, and significantly enhancing the fluency of EEG control of peripherals and the user's interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 It is a flow chart of a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention;

[0057] Figure 2 It is a structural schematic diagram of the peripheral control decoding model provided by the present invention;

[0058] Figure 3 It is an experimental flow chart of the peripheral control training provided by the present invention;

[0059] Figure 4 It is an experimental flow chart of subject training provided by the present invention;

[0060] Figure 5 It is a structural diagram of a multi-head Transformer decoder provided by the present invention;

[0061] Figure 6 It is a structural diagram of the Transformer class decoder provided by the present invention;

[0062] Figure 7 It is a flow chart of the peripheral control training provided by the present invention;

[0063] Figure 8It is a schematic diagram of the process of subject training provided by the present invention;

[0064] Fig. 9 It is a structural schematic diagram of a device for issuing continuous peripheral control instructions based on EEG signals provided by the present invention;

[0065] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, rather than to describe a specific order or precedence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type.

[0068] The present invention provides a method for issuing continuous peripheral control instructions based on electroencephalogram signals. Figure 1 FIG. 1 is a flow chart of a method for issuing continuous peripheral control instructions based on EEG signals provided by the present invention. Figure 1 As shown, the method includes:

[0069] Step 110, acquiring real-time whole-brain signal data.

[0070] Specifically, real-time whole-brain signal data can be obtained, where the real-time whole-brain signal data refers to data that needs to be decoded for movement intention and clenching data.

[0071] The embodiment of the present invention uses an EEG amplifier system with 64 leads to collect real-time whole-brain signal data. The placement of electrodes strictly follows the standard layout of the international 10-20 system. The forehead is selected as the ground wire, and the top of the head is used as the reference electrode. When preprocessing each segment of real-time whole-brain signal data, a bandpass filter of 0.5 to 40 Hz is first applied. The purpose of this is to minimize noise interference while retaining those frequency bands that may be related to movement rhythm. Subsequently, the sampling rate of the data is reduced to 200 Hz. This process not only reduces the storage space occupied, but also reduces the complexity of subsequent calculation processing. In addition, the full-lead common average reference method is used to eliminate possible common mode noise, and baseline drift correction is performed on each lead to eliminate slow wave drift interference caused by the distraction of the subject. This series of sophisticated data processing steps ensures the quality of the final EEG signal and provides a solid foundation for subsequent analysis.

[0072] That is, real-time whole-brain signal data can be obtained by preprocessing operations, where the preprocessing operations may include bandpass filtering from 0.5 to 40 Hz, downsampling to 200 Hz, application of common mean reference method, detrending processing, etc., so as to optimize data quality.

[0073] Among them, the formula for preprocessing the real-time whole brain signal data S(t) is as follows:

[0074]

[0075] Among them, S(t) represents the real-time whole brain signal data, Represents real-time whole-brain signal data after preprocessing.

[0076] Step 120, inputting the real-time whole-brain signal data into an external control decoding model to obtain the motion category and motion parameters output by the external control decoding model;

[0077] Step 130, converting the motion category and the motion parameter into an external device control instruction, and issuing a real-time instruction based on the external device control instruction;

[0078] The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder;

[0079] The multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimensionality reduction module is used to reduce the dimensionality of each frequency band data feature in the multi-band data features to obtain reduced dimensionality features, and the multi-head Transformer decoder is used to decode based on the reduced dimensionality features to obtain the motion category and the motion parameters.

[0080] Specifically, after obtaining the real-time whole-brain signal data, the real-time whole-brain signal data can be input into the peripheral control decoding model to obtain the motion category and motion parameters output by the peripheral control decoding model, and then the motion category and the motion parameters are converted into peripheral control instructions, and real-time instructions are issued based on the peripheral control instructions. That is, the decoded peripheral control instructions are sent to the manipulator or other peripherals, which can realize accurate and continuous control of the manipulator or other peripherals, for example, involving the adjustment of speed and position, which significantly enhances the fluency of EEG control of peripherals and the user's interactive experience.

[0081] Figure 2 is a schematic diagram of the structure of the peripheral control decoding model provided by the present invention, such as Figure 2 As shown, the peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature concatenation and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder.

[0082] Here, the multi-band feature generator is used to extract the data of each frequency band in the real-time whole brain signal data. In order to improve the classification performance and fully combine the information of different frequency bands, a bandpass filter is used in the embodiment of the present invention to extract multiple multi-scale frequency band data of 0.5-40 Hz respectively. The specific steps are as follows:

[0083] 18 frequency bands with different bandwidths of 4HZ, 8HZ, 16HZ, and 36HZ are defined:

[0084]

[0085] Here, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band. imput , the characteristics S of each frequency band can be extracted inp n :

[0086] S input n =Bandpass(S input ,F n )

[0087] Each frequency band data will be extracted through the Riemann feature extraction module to extract the Riemann feature C of the current frequency band input n .

[0088]

[0089] The feature concatenation and vectorization module is used to concatenate the Riemann features to obtain multi-band data features. input n , vectorize, reduce dimension and concatenate them.

[0090] Considering the C input n is a symmetric positive definite matrix. To eliminate redundancy and simplify feature extraction and subsequent classification processes, it is vectorized and only the upper triangular elements are retained:

[0091]

[0092] For each frequency band data feature ζ input n , use principal component analysis (PCA) to reduce the dimension of , retaining 95% of its explainable variance characteristics, denoted by d f is the feature dimension after dimensionality reduction:

[0093]

[0094] Next, the embodiment of the present invention concatenates the features after dimension reduction of the 18 frequency bands to obtain the output of the feature concatenation and vectorization module:

[0095]

[0096] The dimension reduction module is used to reduce the dimension of multi-band data features to obtain reduced-dimensional features.

[0097] Subsequently, in order to further reduce the data dimension, the embodiment of the present invention refers to the common space modality method to perform combined dimensionality reduction on the frequency band features.

[0098] The peripheral control decoding model is applied to the subject training experiment and the peripheral training experiment respectively. Considering the peripheral training experiment and the peripheral control, the short-term data collected in real time, the input feature is PCA_ζ, and the dimension reduction matrix of the data of the corresponding training group is used for dimension reduction. Let g be the corresponding group (in the peripheral control, all training data are regarded as one group):

[0099] η g =W g MultF_ζ

[0100] Here, a multi-head Transformer decoder is used to decode based on the dimensionality reduction features to obtain motion categories and motion parameters.

[0101] Among them, the multi-head Transformer category decoder consists of a motion category decoding branch and a motion parameter decoding branch, that is, the Transformer category decoder is its branch.

[0102] The model structure of the multi-head Transformer category decoder mainly includes the input embedding layer (InputEmbedding), the positional encoding layer (Positional Encoding), the output embedding layer (Output Embedding), as well as the encoder (Encoder) and the decoder (Decoder). Here, the encoder and decoder are composed of a multi-head attention module, multiple fully connected layers and a Normalization layer.

[0103] The method provided by the embodiment of the present invention inputs real-time whole-brain signal data into the peripheral control decoding model, obtains the motion category and motion parameters output by the peripheral control decoding model, and then converts the motion category and motion parameters into peripheral control instructions, and issues real-time instructions based on the peripheral control instructions. The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder, thereby realizing the conversion from discrete motion imagination instructions to continuous motion imagination instructions, and being able to realize accurate and continuous control of external devices, that is, realizing fine-grained control of external devices, such as speed and position adjustment, by guiding the user to imagine to a corresponding degree according to different peripheral motion parameters, and using the peripheral movement as feedback, thereby improving the accuracy of fine motion imagination classification, and significantly enhancing the fluency of EEG control of peripherals and the user's interactive experience.

[0104] Based on the above embodiment, the training steps of the multi-head Transformer decoder include:

[0105] Step 210, determining an initial multi-head Transformer decoder, and obtaining sample whole-brain signal data, as well as motion category labels and motion parameter labels corresponding to the sample whole-brain signal data through an external device control training experiment;

[0106] Step 220, performing signal preprocessing on the sample whole-brain signal data to obtain preprocessed whole-brain signal data; the signal preprocessing sequentially includes bandpass filtering operation, downsampling operation, application of common mean reference method and detrending operation;

[0107] Step 230, performing multi-band feature extraction on the pre-processed whole-brain signal data to obtain a first multi-band feature, and performing dimensionality reduction on the first multi-band feature to obtain a first sample dimensionality reduction feature;

[0108] Step 240, inputting the first sample dimensionality reduction feature into the initial multi-head Transformer decoder to obtain the predicted motion category and predicted motion parameters output by the initial multi-head Transformer decoder;

[0109] Step 250, based on the difference between the predicted motion category and the motion category label, and based on the difference between the predicted motion parameter and the motion parameter label, determine the target loss, and iterate the parameters of the initial multi-head Transformer decoder based on the target loss to obtain the multi-head Transformer decoder.

[0110] It should be noted that the following is the source of training sample data required for training the multi-head Transformer decoder for peripheral control - peripheral control training experiment. If a peripheral control experiment reaches the evaluation index, all the data of this experiment will be used to train the model.

[0111] In order to better obtain the multi-head Transformer decoder, training can be performed based on the following steps:

[0112] The initial multi-head Transformer decoder can be determined, and the sample whole-brain signal data and the motion category label and motion parameter label corresponding to the sample whole-brain signal data can be obtained through the peripheral control training experiment. The parameters of the initial multi-head Transformer decoder here can be randomly generated or preset.

[0113] The sample whole-brain signal data here is collected by letting the subjects watch the peripheral movements of different intensities and guiding the subjects to perform movement imagination of different intensities. The sample whole-brain signal data includes training data of movement categories and movement intensities. (For example, in a single trial, let a certain part of the peripheral move from slow to fast speed to guide the subjects to perform movement imagination of the corresponding movement category, and the intensity of imagination gradually increases.) This process uses the direct visual guidance of the peripheral to provide intuitive feedback, helping the subjects to more accurately adjust and control their movement imagination, and improve the accuracy and continuity of imagination. Compared with traditional motor imagination EEG training collection experiments guided by display screen pictures or videos, this method reduces the difference between the environment during training collection and the control of the peripheral, and reduces the risk of domain offset between training data and real-time data.

[0114] Figure 3It is an experimental flow chart of the peripheral control training provided by the present invention, such as Figure 3 As shown in the figure, in the peripheral control training experiment, the experiment was divided into 8 groups, each containing 64 trials. Within each group of experiments, each trial was assigned a motor imagery task, accompanied by one of the four randomly selected change patterns, so each trial was randomly selected in terms of movement category and change pattern, but the motor imagery task and imagined change pattern accounted for an equal proportion. These change patterns were: gradual increase, gradual decrease, increase first and then decrease, and decrease first and then increase, each of which represented a different continuous change mode of movement parameters. Participants were guided by visual interaction with the peripheral, while the task instructions were provided by audio. Each trial was divided into three stages: first, the preparation stage, in which the upcoming motor imagery task and its change pattern were introduced through voice instructions, which lasted for 3 seconds; then the task imagery stage, which started with a short beep and lasted for 6 seconds, during which the peripheral movement parameters would be adjusted second by second according to the predetermined change pattern; and finally, the rest stage, which lasted for 3 seconds, during which participants were required to remain still to rest.

[0115] For each set of peripheral control training experimental data, the primary features of the data are extracted by using a multi-band feature generator, a Riemann feature extraction module, feature vectorization, and a feature concatenation method. Then, these features are used to calculate the cross-validation accuracy of each group through dimensionality reduction processing and a Transformer category classifier; on the other hand, the features are input into the Transformer motion parameter decoder to obtain the cross-validation average mean square error between the actual motion parameters. Finally, these two indicators are comprehensively analyzed to calculate the average cross-validation accuracy of each group and the average mean square error of each category as key indicators for evaluating the training effect. Once these performance indicators reach the predetermined threshold, it means that the participants have reached the expected standards in peripheral control, thus marking the successful completion of the peripheral control training experiment.

[0116] After all experiments are completed, collect the data from all groups and organize them in the following format:

[0117] D1={(X 11 ,y 11 ),…,(X 1N ,y 1N )}

[0118] …

[0119] F8={(X 81 ,y 81 ),…,(X 8N ,y 8N )}

[0120] Among them, D1 to D8 represent the data sets of the first to eighth groups of experiments respectively. For the i-th data set D i , X ij represents the data of the jth trial in the i-th group, and y ij is the category label corresponding to this trial.

[0121] After acquiring the sample whole-brain signal data, the sample whole-brain signal data can be subjected to signal preprocessing to obtain preprocessed whole-brain signal data, wherein the signal preprocessing sequentially includes bandpass filtering operation, downsampling operation, application of common mean reference method and detrending operation.

[0122] For each trial data of the sample whole brain signal data Each trial data is sliced ​​into short-term data segments of 1 second in length and converted into 6 segments:

[0123] Slice(X) = {S 1 ,S 2 ,S 3 ,S 4 ,S 5 ,S 6}

[0124] in, Represents the kth short-term data segment.

[0125] Each data segment was subjected to a series of preprocessing steps, including bandpass filtering from 0.5 to 40 Hz, downsampling to 200 Hz, application of common mean referencing, and detrending, to optimize data quality.

[0126] Among them, the signal preprocessing of the sample whole brain signal data is performed, and the formula for obtaining the preprocessed whole brain signal data is as follows:

[0127]

[0128] Among them, S represents the sample whole brain signal data, Represents preprocessed whole-brain signal data.

[0129] Then, multi-band feature extraction is performed on the pre-processed whole-brain signal data to obtain a first multi-band feature, and dimension reduction is performed on the first multi-band feature to obtain a first sample dimension reduction feature.

[0130] For the first multi-band data feature Then, principal component analysis was used to reduce the dimension of each frequency band, retaining the characteristics of 95% of the explained variance, denoted by d f is the feature dimension after dimensionality reduction:

[0131]

[0132] Subsequently, in order to reduce the data dimension, the embodiment of the present invention refers to the common space modality method to perform combined dimension reduction on the frequency band features.

[0133] Let S be the kth short-term data segment of the jth trial in the i-th group of data. ij k After the multi-band feature generator, Riemann feature extraction module, feature vectorization and feature concatenation, the feature obtained is PCA_ζ ij k .

[0134] First, calculate the sum of the PCA features of each group of short-term data:

[0135]

[0136] Calculate the sum of the PCA features of each group of short-term data:

[0137]

[0138] To M i Perform eigendecomposition and calculate its whitening matrix P i , and whiten the mean of each category in each group:

[0139] M i =U i Λ i U i T

[0140]

[0141] S ic =P i M ic P i T ,c=1,2

[0142] At this time, the whitened category mean S of the i-th group of data i1 , S i2 They share the same eigenvector, and the sum of the eigenvalues ​​corresponding to the same eigenvector is 1. This ensures that when the variance of one type of signal is maximized, the variance of the other type of signal is minimized.

[0143] Then, the whitened category means of each group of data are feature decomposed:

[0144] S ic =B ic Λ ic B ic T ,c=1,2

[0145] B i1 =Bi2 =B i

[0146] Extract the eigenvectors corresponding to the largest 4 eigenvalues ​​of the first category and integrate them into Finally, the dimension reduction matrix W of each set of data is obtained i :

[0147]

[0148] And reduce the dimension of the first multi-band feature to obtain the first sample dimension reduction feature η ij k :

[0149] η ij k =W i PCA_ζ ij k

[0150] After obtaining the first sample dimensionality reduction feature, the first sample dimensionality reduction feature may be input into an initial multi-head Transformer decoder to obtain a predicted motion category and predicted motion parameters output by the initial multi-head Transformer decoder.

[0151] Here, the initial multi-headed Transformer decoder includes an input embedding layer, a position encoding layer, and an output embedding layer, wherein the order of appearance of short-term data is converted into position encoding, and the output embedding uses the mean value of the motion parameters under the category. After obtaining the predicted motion category and the predicted motion parameter, the target loss can be determined based on the difference between the predicted motion category and the motion category label, and based on the difference between the predicted motion parameter and the motion parameter label. After obtaining the target loss, the initial multi-headed Transformer decoder can be parameter iterated based on the target loss, and the initial multi-headed Transformer decoder after completing the parameter iteration is used as the multi-headed Transformer decoder.

[0152] It can be understood that the greater the difference between the predicted motion category and the motion category label, the greater the category classification loss; the smaller the difference between the predicted motion category and the motion category label, the smaller the category classification loss.

[0153] It can be understood that the greater the difference between the predicted motion parameter and the motion parameter label, the greater the parameter decoding loss; and the smaller the difference between the predicted motion parameter and the motion parameter label, the smaller the parameter decoding loss.

[0154] Among them, the formula for determining the target loss based on the difference between the predicted motion category and the motion category label, and based on the difference between the predicted motion parameter and the motion parameter label is as follows:

[0155]

[0156]

[0157]

[0158] in, represents the target loss, represents the category classification loss, represents the parameter decoding loss, α and β are the weight coefficients of the two loss terms, which are used to balance the importance of the two tasks, N is the number of samples, and predict_class ij Indicates the predicted motion category, i.e. predict_class ij is the probability that the model predicts that the i-th sample is of the j-th class, truth_class ij Represents the unique hot encoding representation of the true label of the i-th sample. If the i-th sample belongs to class j, then truth_class ij =1, otherwise 0; predict_param i Indicates the predicted motion parameters, i.e. predict_param i is the model's predicted value of the motion parameters for the i-th sample, and truth_param i is the corresponding motion parameter label.

[0159] Based on the above embodiment, the method further includes:

[0160] Step 310, inputting the sample whole-brain signal data into a teeth-gripping decoding module to obtain a teeth-gripping classification result output by the teeth-gripping decoding module;

[0161] Step 320: based on the clenching classification result, switch the predicted peripheral control instruction between a controllable state and an uncontrollable state, wherein the predicted peripheral control instruction is obtained by converting the predicted motion category and the predicted motion parameter.

[0162] Specifically, in the peripheral control training process, a small amount of clenching data is collected while obtaining the sample whole-brain signal data, thereby forming a training data set for the clenching decoding module to train the clenching decoding model. A small amount of the sample whole-brain signal data can also be extracted, and half of the length (i.e., 0.5s) is intercepted and input into the clenching decoding module to obtain the clenching classification result output by the clenching decoding module. Here, the clenching classification result includes clenching and not clenching.

[0163] In the teeth clenching detection, the embodiment of the present invention collects the EEG signal x(t) once every 0.5s to detect whether the subject is clenching his teeth. The collection time is half of the motor imagery data, which ensures the switch control of the peripherals.

[0164] Here, the clenching decoding module includes a high-frequency power spectrum extraction module, a chi-square test feature selection module and a clenching discrimination classifier.

[0165] High frequency power spectrum extraction module:

[0166] First, Fast Fourier Transform (FFT) is applied to each channel of the input sample whole brain signal data x(t) to obtain the frequency domain signal X(f):

[0167]

[0168] Next, calculate the total energy of the signal in the frequency range of 40-100Hz for each channel. The energy can be obtained by calculating the square absolute value of the signal:

[0169]

[0170] Among them, E c represents the total energy of the cth channel within the specified frequency range, X c (f) is the frequency domain signal of the cth channel at frequency f.

[0171] Then, the embodiment of the present invention converts the energy sum E of each channel into c Integrate into a single feature vector:

[0172]

[0173] Chi-square test feature selection module:

[0174] From the collected motor imagery dataset, a set of N cells with the same size as the non-biting signal training dataset is randomly selected. TOOTH The high-frequency power spectrum extraction module is used to extract its features to form the training feature data set F MI , and the feature set of the tooth biting signal training set is F TOOTH . Concatenate the two to form the feature set F and assign the motor imagery feature label 0 and the clenching feature label 1 to form the label set labels:

[0175]

[0176]

[0177] Use the chi-square test to select the top 5 features most relevant to the label and obtain the selected feature selection set F selected And the selection function

[0178] F selected ,S=Chi_squared_test(F,labels,k=5)

[0179] Here, k represents the number of selected features.

[0180] For the real-time collected data x(t) features, the trained selection function is used Select to obtain the selected feature F selected .

[0181] Teeth-gripping discriminant classifier:

[0182] For the training data set, a classifier is constructed using linear discriminant analysis, and the classifier LDA is trained by finding the optimal data projection to maximize the inter-class distance and minimize the intra-class distance.

[0183]

[0184] The selected features of the real-time collected data x(t) are classified using the trained classifier to obtain the teeth clenching classification result (teeth clenching and no teeth clenching).

[0185] y=LDA(F selected )

[0186] After obtaining the bite classification result, the predicted peripheral control instruction can be switched between the controllable state and the uncontrollable state based on the bite classification result, that is, the bite classification result will directly control the motion switch of the peripheral, thereby realizing the asynchronous control of the system. Among them, the predicted peripheral control instruction is obtained based on the predicted motion category and the predicted motion parameter conversion.

[0187] The method provided in the embodiment of the present invention establishes a two-channel peripheral control system for short-term EEG signals based on a trained model. On the one hand, the system detects the subject's clenching action through a teeth-gritting decoding module to switch the peripheral control switch. On the other hand, the decoding module is used to convert the decoded motion category and motion parameters into peripheral control commands, thereby realizing the issuance of continuously variable control instructions. Compared with the common synchronous control method, the asynchronous control model is realized using two-channel control logic, which is suitable for a wider range of application scenarios. In addition, unlike other current discrete parameter control methods, the present invention realizes continuous control of the subject's subjective motion intensity.

[0188] Based on the above embodiment, the multi-head Transformer decoder includes a Transformer class decoder;

[0189] The training steps of the Transformer class decoder include:

[0190] Step 410, determining an initial Transformer category decoder, and obtaining a neighboring sample whole-brain signal data set, and a true value of a motion category corresponding to the neighboring sample whole-brain signal data set;

[0191] Step 420, performing the signal preprocessing on the adjacent sample whole-brain signal data group to obtain preprocessed adjacent whole-brain signal data;

[0192] Step 430, performing multi-band feature extraction on the pre-processed adjacent whole-brain signal data to obtain a second multi-band feature, and performing dimensionality reduction on the second multi-band feature to obtain a second sample dimensionality reduction feature;

[0193] Step 440, inputting the second sample dimensionality reduction feature into the initial Transformer category decoder to obtain a predicted motion category result output by the initial Transformer category decoder;

[0194] Step 450: determine a classification loss based on the difference between the predicted motion category result and the true value of the motion category, and perform parameter iteration on the initial Transformer category decoder based on the classification loss to obtain the Transformer category decoder.

[0195] Specifically, the initial Transformer category decoder can be determined, and the neighboring sample whole-brain signal data group and the true value of the motion category corresponding to the neighboring sample whole-brain signal data group can be obtained.

[0196] It should be noted that the embodiments of the present invention relate to the technology of multi-joint continuous motion imagination control, such as hand (left or right hand) clenching, shoulder and elbow abduction, thigh and calf swinging and other motion imagination, and these motions can be adapted according to the specific peripherals to be controlled. To illustrate the application of the present invention, two types of motion imagination control mechanical exoskeletons are used as scenarios.

[0197] First, suitable participants were selected from the eligible subjects and given detailed task instructions and preparation instructions. After ensuring that the participants were fully prepared, the customized training experiment program was immediately started. The training experiment design included 10 experimental groups, each containing 50 trials, in which various motor imagery tasks were presented in a random order.

[0198] Figure 4It is the experimental flow chart of the subject training provided by the present invention, such as Figure 4 As shown in the figure, the specific single trial process is divided into three main stages: the first is the preparation stage, in which a red background frame and task prompts are displayed for 2 seconds to help participants concentrate; the second is the task imagination stage, in which the background turns green and lasts for 6 seconds, during which feedback is provided every second according to the model evaluation to guide the participants (note that no feedback is provided in the initial experimental group in order to establish baseline data); the last is the rest stage, which lasts for 2 seconds, allowing participants to take a short break between trials.

[0199] No feedback is provided in the initial stage of training (the first two groups of experiments), and it is only used for baseline training of the model. In subsequent training, based on the accuracy of the two most recent trial data, the Riemann feature distance from the current short-term data to the center of gravity of the two corresponding categories is weighted, and the length of the feedback bar represents the weighted distance and is fed back to the subject in real time. The longer the feedback bar, the closer the current trial is to the direction of high classification accuracy.

[0200] This process is designed to accurately record and analyze the participants' EEG responses in order to optimize the performance of the motor imagery task. Each stage of the experiment is designed to improve the quality and efficiency of data collection by finely controlling the experimental conditions. After each experiment, the average accuracy and average feedback value of the participants will be summarized and evaluated. When these key performance indicators reach the preset standard threshold, it indicates that the participants have reached the expected level in motor imagery ability and their control over the imagined force or speed, thus marking the successful completion of the subject's training experiment.

[0201] During the subject training experiment, the training data set that the feedback value generation model of each short-term data S(t) relies on is constructed based on the data of the two most recent groups of experiments, that is, it is constructed based on the whole-brain signal data set of neighboring samples (it is worth noting that the initial two groups of data were not used for feedback generation).

[0202] On this basis, after each group of experiments, collect the previous two groups of data from the current group and organize them into the following form:

[0203] D1={(X 11 ,y 11 ),…,(X 1N ,y 1N )}

[0204] D2={(X 21 ,y 21 ),…,(X 2N ,y 2N )}

[0205] Among them, X 1jis the data of the jth trial in the first group of two groups of data, counting forward from the current group. At the same time, y 1j The motion category label corresponding to this trial. Similarly, X 2j represents the data of the jth trial of the second group of two groups of data counting forward from the current group, and y 2j The motion category label corresponding to this trial.

[0206] Then, the adjacent sample whole brain signal data group can be subjected to signal preprocessing to obtain preprocessed adjacent whole brain signal data, wherein the signal preprocessing includes bandpass filtering operation, downsampling operation, application of common mean reference method and detrending operation in sequence.

[0207] Then, multi-band feature extraction is performed on the pre-processed adjacent whole-brain signal data to obtain a second multi-band feature, and dimension reduction is performed on the second multi-band feature to obtain a second sample dimension reduction feature.

[0208] Specifically, for the second multi-band data feature Then, principal component analysis was used to reduce the dimension of each frequency band, retaining the characteristics of 95% of the explained variance, denoted by d f is the feature dimension after dimensionality reduction:

[0209]

[0210] Subsequently, in order to reduce the data dimension, the embodiment of the present invention refers to the common space modality method to perform combined dimension reduction on the frequency band features.

[0211] Let S be the kth short-term data segment of the jth trial in the i-th group of data. ij k After the multi-band feature generator, Riemann feature extraction module, feature vectorization and feature concatenation, the feature obtained is PCA_ζ ij k .

[0212] First, calculate the sum of the PCA features of each group of short-term data:

[0213]

[0214] Calculate the sum of the PCA features of each group of short-term data:

[0215]

[0216] To M i Perform eigendecomposition and calculate its whitening matrix P i , and whiten the mean of each category in each group:

[0217] M i =Ui Λ i U i T

[0218]

[0219] S ic =P i M ic P i T ,c=1,2

[0220] At this time, the whitened category mean S of the i-th group of data i1 , S i2 They share the same eigenvector, and the sum of the eigenvalues ​​corresponding to the same eigenvector is 1. This ensures that when the variance of one type of signal is maximized, the variance of the other type of signal is minimized.

[0221] Then, the whitened category means of each group of data are feature decomposed:

[0222] S ic =B ic Λ ic B ic T ,c=1,2

[0223] B i1 =B i2 =B i

[0224] Extract the eigenvectors corresponding to the largest 4 eigenvalues ​​of the first category and integrate them into Finally, the dimension reduction matrix W of each set of data is obtained i :

[0225]

[0226] And reduce the dimension of the second multi-band feature to obtain the second sample dimension reduction feature η ij k :

[0227] η ij k =W i PCA_ζ ij k

[0228] After obtaining the second sample dimensionality reduction feature, the second sample dimensionality reduction feature may be input into the initial Transformer category decoder to obtain the predicted motion category result output by the initial Transformer category decoder.

[0229] After obtaining the predicted motion category result, the classification loss can be determined based on the difference between the predicted motion category result and the true value of the motion category, and the parameters of the initial Transformer category decoder are iterated based on the classification loss. The initial Transformer category decoder after the parameter iteration is used as the Transformer category decoder.

[0230] It can be understood that the greater the difference between the predicted motion category result and the true value of the motion category, the smaller the classification loss; the greater the difference between the predicted motion category result and the true value of the motion category, the smaller the classification loss.

[0231] Among them, based on the difference between the predicted motion category result and the true value of the motion category, the formula for determining the classification loss is as follows:

[0232]

[0233] in, represents classification loss, N is the number of samples, predict_class ij Indicates the predicted sports category result, i.e. predict_class ij is the probability that the model believes that the i-th sample is of class j, truth_class ij Represents the true value of the motion category, i.e. truth_class ij is the one-hot encoding representation of the true label of the i-th sample. If the i-th sample belongs to class j, then truth_class ij =1, otherwise 0.

[0234] Based on the above embodiment, the method further includes:

[0235] Step 510, obtaining sample EEG signal data of the subject, and obtaining an initial subject training model, wherein the initial subject training model includes a signal preprocessing module, a category centroid distance calculation module, an average classification accuracy determination module, and a distance feedback value mapping module;

[0236] Step 520, guiding the brain activity pattern of the subject based on the sample EEG signal data and the initial subject training model to obtain a subject training model;

[0237] The signal preprocessing module is used to perform the signal preprocessing on the sample EEG signal data to obtain preprocessed EEG signal data;

[0238] The category centroid distance calculation module is used to respectively determine the first category centroid distance and the second category centroid distance between the preprocessed EEG signal data and the adjacent sample whole brain signal data group;

[0239] The average classification accuracy determination module is used to determine a first average classification accuracy and a second average classification accuracy of cross-validation based on the neighboring sample whole-brain signal data group and the Transformer category decoder;

[0240] The distance feedback value mapping module is used to determine a feedback value based on the first average classification accuracy, the second average classification accuracy, the first category center of gravity distance and the second category center of gravity distance, and guide the brain activity pattern of the subject based on the feedback value.

[0241] Specifically, the following is the training process for the subjects, and it is necessary that the subjects can master the peripheral control decoding model only after training.

[0242] The sample EEG signal data of the subject can be obtained, and an initial subject training model can be obtained, wherein the initial subject training model includes a signal preprocessing module, a category centroid distance calculation module, an average classification accuracy determination module and a distance feedback value mapping module.

[0243] Then, the brain activity pattern of the subject is trained based on the sample EEG signal data and the initial subject training model to obtain the subject training model.

[0244] It can be understood that the EEG signals generated by the subjects directly by imagining the movement of limb joints are pre-processed and then output instructions to control peripherals through the continuously decodable motion parameter model.

[0245] Here, the signal preprocessing module is used to perform signal preprocessing on the sample EEG signal data to obtain preprocessed EEG signal data.

[0246] The category centroid distance calculation module is used to determine the first category centroid distance and the second category centroid distance between the preprocessed EEG signal data and the adjacent sample whole brain signal data group. Here, the first category centroid distance can be used Indicates that the second category centroid distance can be used express.

[0247] Here, the average classification accuracy determination module is used to determine the first average classification accuracy and the second average classification accuracy of the 5-fold cross-validation based on the neighboring sample whole-brain signal data group and the Transformer category decoder, wherein the first average classification accuracy can be represented by p1 and the second average classification accuracy can be represented by p2.

[0248] The distance feedback value mapping module is used to determine the feedback value based on the first average classification accuracy, the second average classification accuracy, the first category center of gravity distance and the second category center of gravity distance, and train the brain activity pattern of the subject based on the feedback value.

[0249] The formula for determining the feedback value based on the first average classification accuracy, the second average classification accuracy, the first category centroid distance, and the second category centroid distance is as follows:

[0250]

[0251] Among them, f v Indicates the feedback value, represents the centroid distance of the first category, represents the centroid distance of the second category, p1 represents the first average classification accuracy, and p2 represents the second average classification accuracy.

[0252] The method provided by the embodiment of the present invention measures the fine-grained feedback value based on the distance in the Riemann feature space. This method can capture and record the whole brain signal in real time, and then extract the multi-band Riemann features with decisive significance from these whole brain signals, and feed back the effect of the current collected signal to the subject. In this way, the embodiment of the present invention ensures that the features extracted from the whole brain signal not only contain a rich amount of information, but also can accurately reflect the brain activity pattern of the subject during the training process, guide the subject to stimulate more distinguishable EEG signals and improve motor imagination ability, while providing a solid foundation for subsequent continuous control of peripherals.

[0253] Based on the above embodiment, the steps for determining the first category centroid distance and the second category centroid distance are as follows:

[0254] Step 610, determining the Riemann centroids of each category in the neighboring sample whole brain signal data group, and determining a first distance mean, a first standard deviation, and a second distance mean and a second standard deviation between the preprocessed EEG signal data and the Riemann centroids in the tangent space of the category to which the preprocessed EEG signal data belongs;

[0255] Step 620, projecting the Riemann features of the preprocessed EEG signal data onto the Riemann centroid in the tangent space of the category to which the preprocessed EEG signal data belongs, to obtain the Riemann tangent space features;

[0256] Step 630, determining a first Riemann distance and a second Riemann distance based on the Riemann cut space feature and the Riemann centroids of the categories;

[0257] Step 640: determine the first category centroid distance based on the first Riemann distance, the first distance mean and the first standard deviation, and determine the second category centroid distance based on the second Riemann distance, the second distance mean and the second standard deviation.

[0258] Specifically, first, determine the Riemann centroid G of each category in the neighboring sample whole brain signal data set ic, where i refers to the two nearest neighbor training data sets, where i=1 represents the first set of the two sets of data, and i=2 represents the second set (the same below). c represents the centroid of the cth category:

[0259]

[0260] Among them, d R (C1, C2) represents the Riemann distance between the Riemann features C1 and C2, λ i C1 -1 The characteristic value of C2, the distance calculation formula is:

[0261]

[0262] Next, the Riemann features of each short-term data segment sample in each group of data are projected onto the tangent space of the Riemann centroid of the group of data:

[0263]

[0264] In the Riemann tangent space with G as the tangent point, since it is a flat Euclidean space, the distance measure between any two points P1 and P2 is:

[0265]

[0266] Using the above formula, calculate the mean distance between all short-term sample data segments of each group and the centroid in the tangent space of the corresponding category. and standard deviation σ ic :

[0267]

[0268]

[0269] That is, the first distance mean, the first standard deviation, and the second distance mean and the second standard deviation between the preprocessed EEG signal data and the Riemann center of gravity in the tangent space of the category to which the preprocessed EEG signal data belongs can be determined.

[0270] It should be noted that the feedback given to the subjects by the preprocessed EEG signal data is real-time.

[0271] Remember C S(t) is the output of S(t) after the Riemann feature extraction module, and the Riemann cut space feature is calculated after projecting it to the Riemann centroid of the category to which each group of training data belongs Where c refers to the category to which S(t) belongs:

[0272]

[0273] Furthermore, based on the Riemann cut space feature and the Riemann center of gravity G of each class ic , determine the first Riemann distance d1 and the second Riemann distance d2, where c refers to the category to which S(t) belongs:

[0274]

[0275]

[0276] Finally, the first category centroid distance can be determined based on the first Riemann distance, the first distance mean and the first standard deviation. Based on the second Riemann distance, the second distance mean and the second standard deviation, the second category centroid distance is determined The formula is as follows:

[0277]

[0278]

[0279] That is, the mean of each group obtained by using the Riemann cut space projection module and standard deviation σ ic Normalize d1 and d2.

[0280] Based on the above embodiment, the multi-head Transformer decoder includes an encoding module and a decoding module;

[0281] The encoding module includes an input embedding layer, a position encoding layer, an output embedding layer and an encoder;

[0282] The decoding module comprises a motion category decoding branch connected to the encoding module, and a motion parameter decoding branch connected to the motion category decoding branch;

[0283] The motion category decoding branch includes a first decoder and a first activation layer;

[0284] The motion parameter decoding branch comprises a second decoder and a second activation layer.

[0285] Specifically, Figure 5 is a schematic diagram of the structure of the multi-head Transformer decoder provided by the present invention, such as Figure 5 As shown, Figure 6 is a schematic diagram of the structure of the Transformer class decoder provided by the present invention, such as Figure 6 As shown, the multi-head Transformer decoder includes an encoding module and a decoding module, wherein the encoding module includes an input embedding layer, a position encoding layer, an output embedding layer and an encoder.

[0286] Among them, the input embedding layer (Input Embedding):

[0287] Peripheral control training data:

[0288] Consider the input trial data After preprocessing, it is decomposed into six time period data sets {S 1 ,S 2 ,S 3 ,S 4 ,S 5 ,S 6 The data of each time period is processed by feature vectorization, and the resulting feature set is:

[0289] {MultF_ζ 1 ,...,MultF_ζ 6}

[0290] Subsequently, these feature sets are concatenated and reshaped in time series to construct a multi-Slice feature representation that takes the time dimension into account:

[0291]

[0292] Next, to generate multi-scale temporal features, a multi-time scale variation T is applied n , where n is the scale factor. This enables the model to effectively process real-time short-term data of different lengths:

[0293]

[0294] For the short-term whole-brain signal data S(t) collected in real time:

[0295] Collect the input short-term whole-brain signal data S(t) in real time, and splice all the previous continuous short-term whole-brain signal data that are consistent with the current data classification results. Suppose there are m in total: {S(t-m+1),...,S(t-1),S(t)}. After these data are processed by feature vectorization, the feature set formed is:

[0296] {MultF_ζ t-m+1 ,...,MultF_ζ t}

[0297] Then, these feature sets are concatenated and reshaped into time series as input embeddings:

[0298]

[0299] Positional Encoding:

[0300] The purpose of the positional encoding layer is to provide timing information for the peripheral control training data and the short-term data S(t) collected in real time, so that the model can capture the time order in different short-term data.

[0301] Peripheral control training data:

[0302] For multi-slice feature input embedding of scale n Assume that the short-term data positions corresponding to each component are {x, x+1,…, x+n-1}, then the corresponding triangular position code is The code pos for each position i i for:

[0303]

[0304] For short-term data S(t) collected in real time:

[0305] Suppose its input embedding is It consists of m continuous short-term data features that are consistent with the current data classification results. The triangular position encoding of these data is

[0306] Output Embedding:

[0307] The design of the output embedding layer is dedicated to processing motion parameters related to motion tasks. Due to their continuous nature and differences between different tasks, these parameters need to be normalized and quantized first in order to match the input embedding layer.

[0308] Quantization encoding:

[0309] For N types of imagination tasks, consider the motion task of category c, if its motion parameters are in the range For the motion parameters para under this category c ,First, perform normalization and map it to a value in the [0,1] interval:

[0310]

[0311] Furthermore, according to the dimension d = (9 × c (c + 1)) of the input and output embeddings, the normalized motion parameters are mapped to [0, N d-1 -1] range, and fill the first digit with 0, and convert it to a d-dimensional N-ary sequence:

[0312]

[0313] In addition, in view of the multi-category nature of the task, the embodiment of the present invention defines a category code BaseN_ClassF to encode the N-ary sequence of the parameter to ensure that the coding spacing of the same normalized motion parameters in different categories is maximized. In the two types of tasks, this strategy is equivalent to using a complementary operation for the same parameter encoding in different categories: for the same parameter encoding in two different categories, the encoding of one category is kept unchanged, while the encoding of the other category is complemented:

[0314]

[0315] The above operation is recorded as the Quant() function.

[0316] Peripheral control training data:

[0317] For multi-slice feature input of scale n, the model output is delayed by 1s compared to the input. Therefore, it is necessary to perform a right shift operation on the parameters: {Nop,c1,c2,...,c n-1}, Nop is a placeholder, and its quantized value is a d-dimensional zero vector. After that, these values ​​are quantized and concatenated in turn to generate the output embedding features:

[0318]

[0319] For the short-term whole-brain signal data S(t) collected in real time:

[0320] For the input short-term whole-brain signal data S(t), the motion parameter decoding results that are consistent with the current data classification results before the t time point are integrated and recorded as {c t-m ,c t-m+1 ,...,c t-1 ,Nop}. Then they are quantized and concatenated in turn to form the output embedding feature:

[0321]

[0322] The encoder is composed of a multi-head attention module, multiple fully connected layers and a Normalization layer. The decoding module includes a motion category decoding branch connected to the encoding module, and a motion parameter decoding branch connected to the motion category decoding branch. Here, the motion category decoding branch includes a first decoder and a first activation layer. The first decoder may include a multi-head attention module, multiple fully connected layers and a Normalization layer. The first activation layer may use a tanh function, an ELU (Exponential Linear Unit) activation function, a Softmax activation function, a Maxout activation function, etc. The embodiment of the present invention does not specifically limit this.

[0323] The motion parameter decoding branch includes a second decoder and a second activation layer, and the second decoder may include a multi-head attention module, multiple fully connected layers and a Normalization layer. The second activation layer may use a tanh function, an ELU activation function, a Softmax activation function, a Maxout activation function, etc., which is not specifically limited in the embodiment of the present invention.

[0324] Based on any of the above embodiments, the present invention provides a method for issuing continuous peripheral control instructions based on EEG signals, the steps of which are as follows:

[0325] The first step is to obtain real-time whole-brain signal data.

[0326] In the second step, the real-time whole-brain signal data is input into the peripheral control decoding model to obtain the motion category and motion parameters output by the peripheral control decoding model.

[0327] The third step is to convert the motion category and motion parameters into peripheral control instructions, and issue real-time instructions based on the peripheral control instructions.

[0328] Among them, the peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder.

[0329] The multi-band feature generator is used to extract data of each frequency band in the real-time whole-brain signal data. The Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band. The feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features. The dimensionality reduction module is used to reduce the dimensionality of the data features of each frequency band in the multi-band data features to obtain reduced dimensionality features. The multi-head Transformer decoder is used to decode based on the reduced dimensionality features to obtain motion categories and motion parameters.

[0330] Figure 7 is a flow chart of the peripheral control training provided by the present invention, such as Figure 7 As shown, the training steps of peripheral control training include:

[0331] S1. Determine the initial multi-head Transformer decoder, and obtain sample whole-brain signal data, as well as motion category labels and motion parameter labels corresponding to the sample whole-brain signal data through peripheral control training experiments.

[0332] S2. Preprocessing the sample whole-brain signal data to obtain preprocessed whole-brain signal data. The signal preprocessing includes bandpass filtering, downsampling, applying a common mean reference method, and detrending.

[0333] S3. Perform multi-band feature extraction on the pre-processed whole-brain signal data to obtain a first multi-band feature, and perform dimensionality reduction on the first multi-band feature to obtain a first sample dimensionality reduction feature.

[0334] S4. Input the dimension reduction feature of the first sample into the initial multi-head Transformer decoder to obtain the predicted motion category and predicted motion parameters output by the initial multi-head Transformer decoder.

[0335] S5. Based on the difference between the predicted motion category and the motion category label, and based on the difference between the predicted motion parameter and the motion parameter label, determine the target loss, and iterate the parameters of the initial multi-head Transformer decoder based on the target loss to obtain a multi-head Transformer decoder.

[0336] S6. Input the sample whole-brain signal data into the teeth-gripping decoding module to obtain the teeth-gripping classification result output by the teeth-gripping decoding module.

[0337] S7. Based on the bite classification result, switch the predicted peripheral control instruction to convert between a controllable state and an uncontrollable state, wherein the predicted peripheral control instruction is obtained based on the predicted motion category and the predicted motion parameter conversion.

[0338] Here, the multi-head Transformer decoder includes a Transformer class decoder;

[0339] Figure 8 It is a flow chart of the subject training provided by the present invention, such as Figure 8 As shown, the training steps of the subject training include:

[0340] S1. Determine the initial Transformer category decoder, and obtain the neighboring sample whole-brain signal data set, and the true value of the motion category corresponding to the neighboring sample whole-brain signal data set.

[0341] S2. Perform the signal preprocessing on the adjacent sample whole-brain signal data group to obtain preprocessed adjacent whole-brain signal data.

[0342] S3. Perform multi-band feature extraction on the pre-processed adjacent whole-brain signal data to obtain a second multi-band feature, and perform dimensionality reduction on the second multi-band feature to obtain a second sample dimensionality reduction feature.

[0343] S4. Input the dimension reduction feature of the second sample into the initial Transformer category decoder to obtain the predicted motion category result output by the initial Transformer category decoder.

[0344] S5. Based on the difference between the predicted motion category result and the true value of the motion category, determine the classification loss, and iterate the parameters of the initial Transformer category decoder based on the classification loss to obtain the Transformer category decoder.

[0345] S6. Obtain sample EEG signal data of the subject and obtain an initial subject training model, wherein the initial subject training model includes a signal preprocessing module, a category centroid distance calculation module, an average classification accuracy determination module and a distance feedback value mapping module.

[0346] S7. Based on the sample EEG signal data and the initial subject training model, the subject's brain activity pattern is trained to obtain a subject training model.

[0347] The signal preprocessing module is used to perform signal preprocessing on sample EEG signal data to obtain preprocessed EEG signal data.

[0348] The category centroid distance calculation module is used to respectively determine the first category centroid distance and the second category centroid distance between the preprocessed EEG signal data and the adjacent sample whole-brain signal data group.

[0349] The average classification accuracy determination module is used to determine a first average classification accuracy and a second average classification accuracy of cross-validation based on the neighboring sample whole-brain signal data group and the Transformer category decoder.

[0350] The distance feedback value mapping module is used to determine the feedback value based on the first average classification accuracy, the second average classification accuracy, the first category center of gravity distance and the second category center of gravity distance, and train the brain activity pattern of the subject based on the feedback value.

[0351] Here, the steps for determining the first category centroid distance and the second category centroid distance are as follows:

[0352] Determine the Riemann centroids of each category in the neighboring sample whole-brain signal data group, and determine the first distance mean, first standard deviation, and second distance mean and second standard deviation of the Riemann centroids in the tangent space between the preprocessed EEG signal data and the category to which the preprocessed EEG signal data belongs.

[0353] The Riemann features of the preprocessed EEG signal data are projected onto the Riemann center of gravity in the tangent space of the category to which the preprocessed EEG signal data belongs, to obtain the Riemann tangent space features.

[0354] Based on the characteristics of the Riemann cut space and the Riemann center of gravity of each category, the first Riemann distance and the second Riemann distance are determined.

[0355] A first category centroid distance is determined based on the first Riemann distance, the first distance mean, and the first standard deviation, and a second category centroid distance is determined based on the second Riemann distance, the second distance mean, and the second standard deviation.

[0356] Among them, the internal structure of the multi-head Transformer decoder covers the following key components:

[0357] Multi-band Riemann dimensionality reduction feature extraction: This feature extraction strategy is closely linked to the features that the feedback bar relies on, focusing on capturing multi-band EEG responses related to motor imagery. By adopting PCA dimensionality reduction technology and borrowing the idea of ​​CSP (Common Spatial Patterns) to find the optimal projection space, the model effectively retains the key features that best reflect the differences between categories, providing a solid foundation for subsequent efficient decoding.

[0358] Shared codec parameters and multi-head decoding mechanism: Considering that both the movement category and movement intensity information are response codes of the movement rhythm in the same period, the model designs a shared codec parameter mechanism and a multi-head decoding mechanism with differentiated output branches. This design not only promotes the extraction of common features of motor imagery and the differentiated learning of feature constraint relationships under different task requirements, but also significantly improves the overall learning efficiency of the model by establishing a joint loss function. This innovation enables the model to directly handle the dual tasks of category classification and intensity decoding.

[0359] Position encoding and multi-level attention mechanism: In view of the dependencies between continuous EEG signal data segments, the model introduces position encoding and multi-level attention mechanism. This enables the model to effectively capture the complex relationship between data segments, thereby ensuring high-precision decoding of motor imagery categories and intensities.

[0360] Output coding: In order to enhance the distinguishability of motion parameters of different categories, the model adopts a special output coding strategy. This strategy aims to maximize the code distance of the same normalized motion parameters in different categories in the coding space, and enhances the model penalty for classification errors.

[0361] It should be noted that the subject training: During the process of motor imagination, the participants need to complete the imagination task corresponding to the control requirements of the external device, in which the imagination process of the task is guided by a feedback bar based on the multi-band Riemann dimensionality reduction feature. Compared with the event-related desynchronization intensity (ERD) of a single lead as a simple feedback indicator to measure the effect of motor imagination, the feedback mechanism proposed in the embodiment of the present invention is based on the weighted distance between the current data of the whole brain and the most recent data in the Riemann feature layer, which can provide more detailed feedback values ​​to participants, and can accurately capture the changes in motor imagination strategies, providing a more refined evaluation framework than traditional methods. At the same time, this feedback mechanism matches the decoding features used in the subsequent imagination control peripheral link, effectively guiding the improvement of the accuracy of the subject's imagination, and can achieve better peripheral control effects.

[0362] The method provided by the embodiment of the present invention measures the fine-grained feedback value based on the distance in the Riemann feature space. This method can capture and record the whole brain signal in real time, and then extract the multi-band Riemann features with decisive significance from these whole brain signals, and feed back the effect of the current collected signal to the subject. In this way, the present invention ensures that the features extracted from the whole brain signal not only contain a rich amount of information, but also can accurately reflect the brain activity pattern of the subject during the training process, guide the subject to stimulate more distinguishable EEG signals and improve motor imagination ability, while providing a solid foundation for subsequent continuous control of peripherals.

[0363] The method provided by the embodiment of the present invention adopts the Riemann distance measurement method to improve the ability to distinguish EEG signals, and effectively cultivates the user's ability to accurately imagine different movement strengths and speeds by combining innovative training strategies. Importantly, this strategy realizes the conversion from discrete movement imagination instructions to continuous movement imagination instructions. Furthermore, based on this training mechanism, the present invention also proposes a new EEG signal decoding method, which can realize accurate and continuous control of external devices, significantly enhancing the fluency of EEG control of external devices and the user's interactive experience.

[0364] The embodiment of the present invention quantifies the characteristics of short-term EEG signals and the distance to the class center of gravity, and uses visual feedback to guide users to adjust their imagination skills, thereby cultivating users' imagination of movement force or speed. Subsequently, by guiding users to imagine to a corresponding degree according to different peripheral motion parameters, and using the peripheral movement as feedback, sufficient training samples are collected to build a robust continuous control decoding model, and finally achieve continuous control of the peripheral. The system further gives patients who have lost their autonomous movement ability the ability to control peripherals through motor imagination, thereby significantly improving their daily lives.

[0365] The following is a description of the continuous peripheral control instruction issuing device based on EEG signals provided by the present invention. The continuous peripheral control instruction issuing device based on EEG signals described below and the continuous peripheral control instruction issuing method based on EEG signals described above can be referenced to each other.

[0366] Based on any of the above embodiments, the present invention provides a device for issuing continuous peripheral control instructions based on EEG signals. Fig. 9 is a schematic diagram of the structure of the device for issuing continuous peripheral control instructions based on EEG signals provided by the present invention, such as Fig. 9 As shown, the device comprises:

[0367] An acquisition unit 910 is used to acquire real-time whole-brain signal data;

[0368] An input unit 920, used to input the real-time whole-brain signal data into the peripheral control decoding model to obtain the motion category and motion parameters output by the peripheral control decoding model;

[0369] An instruction issuing unit 930, configured to convert the motion category and the motion parameter into a peripheral control instruction, and issue a real-time instruction based on the peripheral control instruction;

[0370] The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder;

[0371] The multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimensionality reduction module is used to reduce the dimensionality of each frequency band data feature in the multi-band data features to obtain reduced dimensionality features, and the multi-head Transformer decoder is used to decode based on the reduced dimensionality features to obtain the motion category and the motion parameters.

[0372] The device provided by the embodiment of the present invention inputs real-time whole-brain signal data into the peripheral control decoding model, obtains the motion category and motion parameters output by the peripheral control decoding model, and then converts the motion category and motion parameters into peripheral control instructions, and issues real-time instructions based on the peripheral control instructions. The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder, thereby realizing the conversion from discrete motion imagination instructions to continuous motion imagination instructions, and being able to realize accurate and continuous control of external devices, that is, realizing fine-grained control of external devices, such as speed and position adjustment, by guiding the user to imagine to a corresponding degree according to different peripheral motion parameters and using the peripheral motion as feedback, thereby improving the accuracy of fine motion imagination classification, and significantly enhancing the fluency of EEG control of peripherals and the user's interactive experience.

[0373] Based on any of the above embodiments, the training step of the multi-head Transformer decoder includes:

[0374] Determine an initial multi-head Transformer decoder, and obtain sample whole-brain signal data, as well as motion category labels and motion parameter labels corresponding to the sample whole-brain signal data through a peripheral control training experiment;

[0375] Performing signal preprocessing on the sample whole-brain signal data to obtain preprocessed whole-brain signal data; the signal preprocessing sequentially includes bandpass filtering operation, downsampling operation, application of common mean reference method and detrending operation;

[0376] Performing multi-band feature extraction on the pre-processed whole-brain signal data to obtain a first multi-band feature, and performing dimensionality reduction on the first multi-band feature to obtain a first sample dimensionality reduction feature;

[0377] Inputting the first sample dimensionality reduction feature into the initial multi-head Transformer decoder to obtain the predicted motion category and predicted motion parameters output by the initial multi-head Transformer decoder;

[0378] Based on the difference between the predicted motion category and the motion category label, and based on the difference between the predicted motion parameter and the motion parameter label, a target loss is determined, and based on the target loss, parameters of the initial multi-head Transformer decoder are iterated to obtain the multi-head Transformer decoder.

[0379] Based on any of the above embodiments, a conversion unit is further included, and the conversion unit is specifically used for:

[0380] Inputting the sample whole-brain signal data into a bite decoding module to obtain a bite classification result output by the bite decoding module;

[0381] Based on the bite classification result, the predicted peripheral control instruction is switched between a controllable state and an uncontrollable state, and the predicted peripheral control instruction is obtained by converting the predicted motion category and the predicted motion parameter.

[0382] Based on any of the above embodiments, the multi-head Transformer decoder includes a Transformer class decoder;

[0383] The training steps of the Transformer class decoder include:

[0384] Determine an initial Transformer category decoder, and obtain a neighboring sample whole-brain signal data set, and a true value of a motion category corresponding to the neighboring sample whole-brain signal data set;

[0385] Performing the signal preprocessing on the adjacent sample whole-brain signal data group to obtain preprocessed adjacent whole-brain signal data;

[0386] Performing multi-band feature extraction on the pre-processed adjacent whole-brain signal data to obtain a second multi-band feature, and performing dimensionality reduction on the second multi-band feature to obtain a second sample dimensionality reduction feature;

[0387] Inputting the second sample dimensionality reduction feature into the initial Transformer category decoder to obtain a predicted motion category result output by the initial Transformer category decoder;

[0388] Based on the difference between the predicted motion category result and the true value of the motion category, a classification loss is determined, and based on the classification loss, parameters of the initial Transformer category decoder are iterated to obtain the Transformer category decoder.

[0389] Based on any of the above embodiments, it also includes a subject training unit, and the subject training unit is specifically used for:

[0390] Obtaining sample EEG signal data of the subject and obtaining an initial subject training model, wherein the initial subject training model includes a signal preprocessing module, a category centroid distance calculation module, an average classification accuracy determination module, and a distance feedback value mapping module;

[0391] Based on the sample EEG signal data and the initial subject training model, the brain activity pattern of the subject is guided to obtain a subject training model;

[0392] The signal preprocessing module is used to perform the signal preprocessing on the sample EEG signal data to obtain preprocessed EEG signal data;

[0393] The category centroid distance calculation module is used to respectively determine the first category centroid distance and the second category centroid distance between the preprocessed EEG signal data and the adjacent sample whole brain signal data group;

[0394] The average classification accuracy determination module is used to determine a first average classification accuracy and a second average classification accuracy of cross-validation based on the neighboring sample whole-brain signal data group and the Transformer category decoder;

[0395] The distance feedback value mapping module is used to determine a feedback value based on the first average classification accuracy, the second average classification accuracy, the first category center of gravity distance and the second category center of gravity distance, and guide the brain activity pattern of the subject based on the feedback value.

[0396] Based on any of the above embodiments, the steps of determining the first category centroid distance and the second category centroid distance are as follows:

[0397] Determine the Riemann centroids of each category in the neighboring sample whole-brain signal data group, and determine a first distance mean, a first standard deviation, and a second distance mean and a second standard deviation between the preprocessed EEG signal data and the Riemann centroids in the tangent space of the category to which the preprocessed EEG signal data belongs;

[0398] Projecting the Riemann features of the preprocessed EEG signal data onto the Riemann center of gravity in the tangent space of the category to which the preprocessed EEG signal data belongs, to obtain a Riemann tangent space feature;

[0399] Determining a first Riemann distance and a second Riemann distance based on the Riemann cut space feature and the Riemann center of gravity of each category;

[0400] The first category centroid distance is determined based on the first Riemann distance, the first distance mean and the first standard deviation, and the second category centroid distance is determined based on the second Riemann distance, the second distance mean and the second standard deviation.

[0401] Based on any of the above embodiments, the multi-head Transformer decoder includes an encoding module and a decoding module;

[0402] The encoding module includes an input embedding layer, a position encoding layer, an output embedding layer and an encoder;

[0403] The decoding module comprises a motion category decoding branch connected to the encoding module, and a motion parameter decoding branch connected to the motion category decoding branch;

[0404] The motion category decoding branch includes a first decoder and a first activation layer;

[0405] The motion parameter decoding branch comprises a second decoder and a second activation layer.

[0406] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10 As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute a method for issuing continuous peripheral control instructions based on EEG signals, the method comprising: obtaining real-time whole-brain signal data; inputting the real-time whole-brain signal data into a peripheral control decoding model to obtain the motion category and motion parameters output by the peripheral control decoding model; converting the motion category and the motion parameters into peripheral control instructions, and issuing real-time instructions based on the peripheral control instructions; the peripheral control decoding model comprises a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, and a dimensionality reduction module. and a multi-head Transformer decoder; the multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimensionality reduction module is used to reduce the dimensionality of each frequency band data feature in the multi-band data features to obtain reduced dimensionality features, and the multi-head Transformer decoder is used to decode based on the reduced dimensionality features to obtain the motion category and the motion parameters.

[0407] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0408] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for issuing continuous peripheral control instructions based on EEG signals provided by the above methods, which method includes: obtaining real-time whole-brain signal data; inputting the real-time whole-brain signal data into a peripheral control decoding model to obtain the motion category and motion parameters output by the peripheral control decoding model; converting the motion category and the motion parameters into peripheral control instructions, and issuing real-time instructions based on the peripheral control instructions; the peripheral control decoding model includes multiple A frequency band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimension reduction module and a multi-head Transformer decoder; the multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice each Riemann feature to obtain a multi-band data feature, the dimension reduction module is used to reduce the dimension of each frequency band data feature in the multi-band data feature to obtain a reduced dimension feature, and the multi-head Transformer decoder is used to decode based on the reduced dimension feature to obtain the motion category and the motion parameter.

[0409] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for issuing continuous peripheral control instructions based on EEG signals provided by the above-mentioned methods, the method comprising: acquiring real-time whole-brain signal data; inputting the real-time whole-brain signal data into a peripheral control decoding model to obtain motion categories and motion parameters output by the peripheral control decoding model; converting the motion categories and motion parameters into peripheral control instructions, and issuing real-time instructions based on the peripheral control instructions; the peripheral control decoding model comprises a multi-band feature generator, a Riemann feature extractor, and a decoder. Module, feature splicing and vectorization module, dimensionality reduction module and multi-head Transformer decoder; the multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimensionality reduction module is used to reduce the dimensionality of the data features of each frequency band in the multi-band data features to obtain reduced dimensionality features, and the multi-head Transformer decoder is used to decode based on the reduced dimensionality features to obtain the motion category and the motion parameters.

[0410] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0411] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0412] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for issuing continuous peripheral control instructions based on EEG signals, characterized in that: include: Obtain real-time whole-brain signal data; Inputting the real-time whole-brain signal data into an external control decoding model to obtain the motion category and motion parameters output by the external control decoding model; Converting the motion category and the motion parameter into a peripheral control instruction, and issuing a real-time instruction based on the peripheral control instruction; The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder; The multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimension reduction module is used to reduce the dimension of each frequency band data feature in the multi-band data features to obtain reduced dimension features, and the multi-head Transformer decoder is used to decode based on the reduced dimension features to obtain the motion category and the motion parameter; The training steps of the multi-head Transformer decoder include: Determine an initial multi-head Transformer decoder, and obtain sample whole-brain signal data, as well as motion category labels and motion parameter labels corresponding to the sample whole-brain signal data through a peripheral control training experiment; Performing signal preprocessing on the sample whole-brain signal data to obtain preprocessed whole-brain signal data; the signal preprocessing sequentially includes bandpass filtering operation, downsampling operation, application of common mean reference method and detrending operation; Performing multi-band feature extraction on the pre-processed whole-brain signal data to obtain a first multi-band feature, and performing dimensionality reduction on the first multi-band feature to obtain a first sample dimensionality reduction feature; Inputting the first sample dimensionality reduction feature into the initial multi-head Transformer decoder to obtain the predicted motion category and predicted motion parameters output by the initial multi-head Transformer decoder; Determine a target loss based on a difference between the predicted motion category and the motion category label, and based on a difference between the predicted motion parameter and the motion parameter label, and perform parameter iteration on the initial multi-head Transformer decoder based on the target loss to obtain the multi-head Transformer decoder; The multi-head Transformer decoder includes a Transformer class decoder; The training steps of the Transformer class decoder include: Determine an initial Transformer category decoder, and obtain a neighboring sample whole-brain signal data set, and a true value of a motion category corresponding to the neighboring sample whole-brain signal data set; Performing the signal preprocessing on the adjacent sample whole-brain signal data group to obtain preprocessed adjacent whole-brain signal data; Performing multi-band feature extraction on the pre-processed adjacent whole-brain signal data to obtain a second multi-band feature, and performing dimensionality reduction on the second multi-band feature to obtain a second sample dimensionality reduction feature; Inputting the second sample dimensionality reduction feature into the initial Transformer category decoder to obtain a predicted motion category result output by the initial Transformer category decoder; Determine a classification loss based on a difference between the predicted motion category result and the true value of the motion category, and perform parameter iteration on the initial Transformer category decoder based on the classification loss to obtain the Transformer category decoder; The method further comprises: Obtaining sample EEG signal data of the subject and obtaining an initial subject training model, wherein the initial subject training model includes a signal preprocessing module, a category centroid distance calculation module, an average classification accuracy determination module, and a distance feedback value mapping module; Based on the sample EEG signal data and the initial subject training model, the brain activity pattern of the subject is guided to obtain a subject training model; The signal preprocessing module is used to perform the signal preprocessing on the sample EEG signal data to obtain preprocessed EEG signal data; The category centroid distance calculation module is used to respectively determine the first category centroid distance and the second category centroid distance between the preprocessed EEG signal data and the adjacent sample whole brain signal data group; The average classification accuracy determination module is used to determine a first average classification accuracy and a second average classification accuracy of cross-validation based on the neighboring sample whole-brain signal data group and the Transformer category decoder; The distance feedback value mapping module is used to determine a feedback value based on the first average classification accuracy, the second average classification accuracy, the first category center of gravity distance and the second category center of gravity distance, and guide the brain activity pattern of the subject based on the feedback value.

2. The method for issuing continuous peripheral control instructions based on EEG signals according to claim 1, characterized in that: The method further comprises: Inputting the sample whole-brain signal data into a bite decoding module to obtain a bite classification result output by the bite decoding module; Based on the bite classification result, the predicted peripheral control instruction is switched between a controllable state and an uncontrollable state, and the predicted peripheral control instruction is obtained by converting the predicted motion category and the predicted motion parameter.

3. The method for issuing continuous peripheral control instructions based on EEG signals according to claim 1, characterized in that: The steps for determining the first category centroid distance and the second category centroid distance are as follows: Determine the Riemann centroids of each category in the neighboring sample whole-brain signal data group, and determine a first distance mean, a first standard deviation, and a second distance mean and a second standard deviation between the preprocessed EEG signal data and the Riemann centroids in the tangent space of the category to which the preprocessed EEG signal data belongs; Projecting the Riemann features of the preprocessed EEG signal data onto the Riemann center of gravity in the tangent space of the category to which the preprocessed EEG signal data belongs, to obtain a Riemann tangent space feature; Determining a first Riemann distance and a second Riemann distance based on the Riemann cut space feature and the Riemann center of gravity of each category; The first category centroid distance is determined based on the first Riemann distance, the first distance mean and the first standard deviation, and the second category centroid distance is determined based on the second Riemann distance, the second distance mean and the second standard deviation.

4. The method for issuing continuous peripheral control instructions based on EEG signals according to any one of claims 1 to 3, characterized in that: The multi-head Transformer decoder includes an encoding module and a decoding module; The encoding module includes an input embedding layer, a position encoding layer, an output embedding layer and an encoder; The decoding module comprises a motion category decoding branch connected to the encoding module, and a motion parameter decoding branch connected to the motion category decoding branch; The motion category decoding branch includes a first decoder and a first activation layer; The motion parameter decoding branch comprises a second decoder and a second activation layer.

5. A device for issuing continuous peripheral control instructions based on EEG signals, characterized in that: include: An acquisition unit, used for acquiring real-time whole-brain signal data; An input unit, used to input the real-time whole-brain signal data into an external control decoding model to obtain the motion category and motion parameters output by the external control decoding model; An instruction issuing unit, used for converting the motion category and the motion parameter into a peripheral control instruction, and issuing a real-time instruction based on the peripheral control instruction; The peripheral control decoding model includes a multi-band feature generator, a Riemann feature extraction module, a feature splicing and vectorization module, a dimensionality reduction module and a multi-head Transformer decoder; The multi-band feature generator is used to extract the data of each frequency band in the real-time whole-brain signal data, the Riemann feature extraction module is used to extract the Riemann features corresponding to the data of each frequency band, the feature splicing and vectorization module is used to splice the Riemann features to obtain multi-band data features, the dimension reduction module is used to reduce the dimension of each frequency band data feature in the multi-band data features to obtain reduced dimension features, and the multi-head Transformer decoder is used to decode based on the reduced dimension features to obtain the motion category and the motion parameter; The multi-head Transformer decoder training unit is also included, and the multi-head Transformer decoder training unit is specifically used to: Determine an initial multi-head Transformer decoder, and obtain sample whole-brain signal data, as well as motion category labels and motion parameter labels corresponding to the sample whole-brain signal data through a peripheral control training experiment; Performing signal preprocessing on the sample whole-brain signal data to obtain preprocessed whole-brain signal data; The signal preprocessing includes, in sequence, a bandpass filtering operation, a downsampling operation, an application of a common mean reference method, and a detrending operation; Performing multi-band feature extraction on the pre-processed whole-brain signal data to obtain a first multi-band feature, and performing dimensionality reduction on the first multi-band feature to obtain a first sample dimensionality reduction feature; Inputting the first sample dimensionality reduction feature into the initial multi-head Transformer decoder to obtain the predicted motion category and predicted motion parameters output by the initial multi-head Transformer decoder; Determine a target loss based on a difference between the predicted motion category and the motion category label, and based on a difference between the predicted motion parameter and the motion parameter label, and perform parameter iteration on the initial multi-head Transformer decoder based on the target loss to obtain the multi-head Transformer decoder; The multi-head Transformer decoder includes a Transformer class decoder; The training steps of the Transformer class decoder include: Determine an initial Transformer category decoder, and obtain a neighboring sample whole-brain signal data set, and a true value of a motion category corresponding to the neighboring sample whole-brain signal data set; Performing the signal preprocessing on the adjacent sample whole-brain signal data group to obtain preprocessed adjacent whole-brain signal data; Performing multi-band feature extraction on the pre-processed adjacent whole-brain signal data to obtain a second multi-band feature, and performing dimensionality reduction on the second multi-band feature to obtain a second sample dimensionality reduction feature; Inputting the second sample dimensionality reduction feature into the initial Transformer category decoder to obtain a predicted motion category result output by the initial Transformer category decoder; Determine a classification loss based on a difference between the predicted motion category result and the true value of the motion category, and perform parameter iteration on the initial Transformer category decoder based on the classification loss to obtain the Transformer category decoder; The device also includes: Obtaining sample EEG signal data of the subject and obtaining an initial subject training model, wherein the initial subject training model includes a signal preprocessing module, a category centroid distance calculation module, an average classification accuracy determination module, and a distance feedback value mapping module; Based on the sample EEG signal data and the initial subject training model, the brain activity pattern of the subject is guided to obtain a subject training model; The signal preprocessing module is used to perform the signal preprocessing on the sample EEG signal data to obtain preprocessed EEG signal data; The category centroid distance calculation module is used to respectively determine the first category centroid distance and the second category centroid distance between the preprocessed EEG signal data and the adjacent sample whole brain signal data group; The average classification accuracy determination module is used to determine a first average classification accuracy and a second average classification accuracy of cross-validation based on the neighboring sample whole-brain signal data group and the Transformer category decoder; The distance feedback value mapping module is used to determine a feedback value based on the first average classification accuracy, the second average classification accuracy, the first category center of gravity distance and the second category center of gravity distance, and guide the brain activity pattern of the subject based on the feedback value.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for issuing continuous peripheral control instructions based on electroencephalogram signals as described in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for issuing continuous peripheral control instructions based on EEG signals as described in any one of claims 1 to 4 is implemented.

Images