An EEG signal feature enhancement method based on reinforcement learning for joint denoising and spatio-temporal relationship modeling

Abstract

The present invention relates to a method for enhancing electroencephalogram (EEG) signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling, including: acquiring multi-channel EEG signals; making interference removal decisions on the acquired multi-channel EEG signals based on multi-agent reinforcement learning to obtain clean signals; and using spatio-temporal attention modeling to restore the details of the clean signals to obtain enhanced EEG signals. By selecting and optimizing the denoising process of EEG signals based on the reinforcement learning mechanism, clean signals with multiple types of irrelevant interferences removed are obtained, improving the overall signal-to-noise ratio and significant effect of EEG features. On this basis, spatio-temporal relationship modeling introducing the transformer attention mechanism is adopted, and by utilizing the inherent temporal continuity and spatial correlation of the EEG test signal flow, signals with higher spatial resolution are reconstructed, expanding the channel features of the EEG source data. Finally, the purpose of improving the signal spatial resolution and thus the recognition accuracy is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram (EEG) intelligent recognition, and relates to a method for enhancing EEG signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling. Background Art

[0002] EEG signals are electrical signals collected and recorded on the scalp using non-invasive flexible electrodes. These electrical signals are formed by the summation of postsynaptic potentials synchronously generated by a large number of neurons during brain activity, and are the overall reflection of the physiological activities of brain nerve cells on the cerebral cortex or the scalp surface. When a test subject imagines limb movement without actual limb movement, electrical signals are still generated between neurons. When the energy of these signals accumulates beyond a certain threshold, EEG signals are generated. The EEG signals generated by motor imagery have the characteristics of event-related synchronization and desynchronization. By analyzing the motor imagery EEG signals and classifying the features of the EEG signals, the movement intention of the imaginer can be judged, thereby realizing the control of external devices.

[0003] The current research focus on the intelligent processing of EEG signals is the removal of various interferences and artifacts in EEG signals, the intelligent selection of efficient channels, and the enhancement of EEG features required for classification, so as to enable the efficient implementation of the subsequent intelligent classification and recognition process of EEG signals. Existing artifact removal algorithms can only remove single-type clutter signals, and the superposition of multiple clutter suppression algorithms will also affect the brain source signals. Existing EEG signal enhancement methods can strengthen features such as local relevant potentials in brain regions in EEG signals, but lack the enhancement of temporal information and EEG details in spatial regions outside the acquisition electrodes. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method for enhancing EEG signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling. The technical problems to be solved by the present invention are realized through the following technical solutions:

[0005] An embodiment of the present invention provides a method for enhancing EEG signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling, including the following steps:

[0006] S1. Obtain multi-channel EEG signals;

[0007] S2. Make an interference removal decision on the obtained multi-channel EEG signals based on multi-agent reinforcement learning to obtain clean signals;

[0008] S3. Use spatio-temporal attention modeling to restore the details of the clean signals to obtain enhanced EEG signals.

[0009] In an embodiment of the present invention, step S2 includes:

[0010] Step S21: Input the clean signal into a cascaded network containing all denoising algorithms for the first denoising;

[0011] Step S22: Evaluate the result of the first denoising;

[0012] Step S23: Provide feedback on the result based on the score of the first evaluation result and then guide the change of the network structure;

[0013] Step S24: Use the changed network structure to construct a new denoising strategy to process the clean signal to obtain the second denoising result;

[0014] Step S25: Evaluate the obtained second denoising result;

[0015] Step S26: Provide feedback on the result based on the score of the second evaluation result and then guide the change of the network structure;

[0016] Repeat the above steps to obtain the optimal joint denoising method for the multi-channel EEG signals.

[0017] In an embodiment of the present invention, step S22 includes:

[0018] Step S221: Use a cross-validation strategy to evaluate the algorithm model.

[0019] In an embodiment of the present invention, step S221 includes:

[0020] Step S2211: For each subject, after dividing all their EEG signals into N equal data subsets, use one subset as the test set and the other N - 1 subsets as the training set;

[0021] Step S2212: Evaluate the algorithm model according to the test set and the training set to obtain the score of one subject's data;

[0022] Step S2213: Take the average score of all subjects as the final score, where the average score of cross-validation is used as the score of one subject.

[0023] In an embodiment of the present invention, step S3 includes:

[0024] Step S31: Apply a spatial attention transformer in the feature channel dimension for encoding spatial features;

[0025] Step S32: Slice the data in the time dimension for attention transformation to obtain attention features containing temporal correlations;

[0026] Step S33: Enhance the spatial features under the guidance of temporal features;

[0027] Step S34: Obtain the enhanced EEG signals based on the features in both the spatial and temporal dimensions.

[0028] In an embodiment of the present invention, step S31 includes:

[0029] Step S311: Independently partition and encode the clean signals;

[0030] Step S312: Extract features from the EEG signals of multiple independent channels after spatial encoding;

[0031] Step S313: Compress the extracted feature set into a feature vector through a compression module;

[0032] Step S314: Aggregate the multi-dimensional features through the activation operation of the activation module on the compressed feature vector, where different channels correspond to their respective activation factors.

[0033] In an embodiment of the present invention, the expression of the compression module is:

[0034]

[0035] In the formula, X ∈ R H×V×C is the output of the feature map extracted from the previous layer of the module; X c ∈ R H×V , c ∈ [1, 2, ··· C], represents the feature map of the c-th channel in X, and x c (i, j) represents the data point at the position (i, j) in X c ; F sq (·) represents the feature compression along the spatial dimension, and compresses each two-dimensional feature X c into a real number z c , and this real number z c has a global perception field to a certain extent, and the dimension output matches the number of feature channel inputs.

[0036] In an embodiment of the present invention, the expression of the activation operation of the activation module is:

[0037] S = σ(W2δ(W1Z))

[0038] In the formula, S is the activation factor, δ represents the sigmoid function operation, Z is the spatial quantity obtained after the compression of H and V; W1 and W2 respectively represent the total correlation of two different channels;

[0039] Obtain the aggregation result of the multi-dimensional features through the activation factor;

[0040] The expression for the aggregation of the multi-dimensional features is as follows:

[0041]

[0042] Among them, after the multi-dimensional feature aggregation, the input vector x c changes to s c represents the activation factor of the c channel, and x c represents the value of the input matrix in the c channel.

[0043] In an embodiment of the present invention, step S32 includes:

[0044] Step S321: Encoding and marking the position of the clean signal in time series;

[0045] Step S322: Compressing the clean signal after encoding and marking into one-dimensional data;

[0046] Step S323: Dividing the one-dimensional data into multiple small slices with overlap in the time dimension;

[0047] Step S324: Using a multi-head attention permissive model for the multiple small slices to obtain attention features including time series correlation.

[0048] In an embodiment of the present invention, step S34 includes:

[0049] Step S341: Combining the features in the spatial and temporal dimensions;

[0050] Step S342: Segmenting the combined features to obtain multiple time series segments;

[0051] Step S343: Jointly performing flattened block mapping reconstruction on multiple independent channels for the multiple time series segments to obtain enhanced electroencephalogram signals.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] The purpose of the present invention is to overcome the deficiencies of the above-mentioned existing methods, and propose interference removal of electroencephalogram raw signals based on reinforcement learning and extraction and reconstruction enhancement of the spatio-temporal relationship of electroencephalogram signals based on transformers, so as to realize high-quality electroencephalogram signal restoration and improve the intelligent classification ability of electroencephalogram signals.

[0054] Optimize the selection of the denoising process for EEG signals based on the reinforcement learning mechanism to obtain clean signals that remove various types of irrelevant interferences, and improve the overall signal-to-noise ratio and significance of EEG features. On this basis, aiming at the high specificity among subjects during the EEG extraction process, adopt the spatio-temporal relationship modeling by introducing the transformer attention mechanism, utilize the inherent temporal continuity and spatial correlation of the EEG test signal flow, and reconstruct signals with higher spatial resolution to expand the channel features of the EEG source data. Finally, achieve the purpose of improving the signal spatial resolution and thus the recognition accuracy.

[0055] Through the following detailed description with reference to the accompanying drawings, other aspects and features of the present invention become apparent. However, it should be understood that the drawings are only designed for the purpose of explanation and not as a limitation of the scope of the present invention, as it should refer to the appended claims. It should also be understood that, unless otherwise indicated, the drawings are not necessarily drawn to scale, and they only attempt to conceptually illustrate the structures and processes described herein. Brief Description of the Drawings

[0056] Figure 1 It is a schematic flow chart of a method for enhancing EEG signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling provided by an embodiment of the present invention;

[0057] Figure 2 It is a schematic flow chart of a method for enhancing EEG signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling provided by an embodiment of the present invention;

[0058] Figure 3 It is a schematic structural diagram of a denoising module provided by an embodiment of the present invention;

[0059] Figure 4 It is a schematic flow chart of enhancing EEG spatial features under spatio-temporal feature modeling provided by an embodiment of the present invention;

[0060] Figure 5 It is a schematic flow chart of channel attention feature extraction provided by an embodiment of the present invention;

[0061] Figure 6 It is a schematic flow chart of time attention feature extraction provided by an embodiment of the present invention;

[0062] Figure 7 It is a schematic structural diagram of a decoder provided by an embodiment of the present invention. Detailed Embodiments

[0063] The following further describes the present invention in detail with specific embodiments, but the embodiments of the present invention are not limited thereto.

[0064] Embodiment 1

[0065] Please refer toFigure 1 and Figure 2 , Figure 1 and Figure 2 FIG. is a schematic flow chart of a method for enhancing electroencephalogram (EEG) signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling provided by an embodiment of the present invention. The present invention provides a method for enhancing EEG signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling, the method comprising the following steps:

[0066] S1. Obtain multi-channel EEG signals;

[0067] As shown in FIG., multi-channel EEG signals are collected from a subject through a motor imagery induction paradigm and a multi-channel non-invasive acquisition device to obtain multi-channel EEG signals. Figure 2

[0068] S2. Make an interference removal decision for the obtained multi-channel EEG signals based on multi-agent reinforcement learning to obtain clean signals.

[0069] Make an interference removal decision for the obtained multi-channel EEG signals based on multi-agent reinforcement learning, extract features such as artifacts from the overall multi-channel EEG signals, and adaptively construct corresponding denoising modules for joint noise removal, avoiding the interference information that affects multi-channel feature classification introduced by existing methods when removing a single type of artifact. The adaptability of existing methods to various noises is insufficient, which may cause the bioelectric signals after denoising to be severely distorted and cannot fully balance the local features and global features of the signals.

[0070] In one embodiment, step S2 includes:

[0071] Step S21. Input the clean signals into a cascaded network containing all denoising algorithms for the first denoising;

[0072] For different forms of environmental motion noise, electromyogram (EMG) artifact-like noise, and obvious emotional fluctuation-like interference of EEG testers, there are respective targeted denoising mechanisms. At the same time, in different scenarios, different orders of removing the three types of interference will also lead to different denoising effects. Therefore, in this step, reinforcement learning is used to construct a reward and punishment mechanism for the entire denoising process, adjust the structure of the model for a combined denoising module containing three denoising mechanisms, and then train an end-to-end structural decision method for the denoising module of EEG signals of different subjects in different scenarios.

[0073] ​During the training process, first, a basic denoising network containing all denoising method modules is established to form an initial denoising strategy. Then, combined with the evaluation method, after obtaining the reward and punishment results, iterative search is carried out to find the optimal denoising network structure for the current input EEG signal sequence. After the training process using a large-scale dataset, a network model can be obtained that can quickly make optimal denoising method decisions for different subjects under various interference conditions such as multiple environments, and can efficiently remove artifacts such as emotional clutter and noise during the EEG test process.

[0074] As Figure 3 shown, a complete denoising network consists of three layers, and these three layers respectively represent multiple processing processes for three categories of interference: EEG emotional clutter, EMG artifacts, and environmental noise. First, the EEG signal is input into a cascaded network containing all denoising algorithms. After one denoising, step S22 is executed.

[0075] Step S22: Evaluate the obtained first denoising result;

[0076] Step S23: After the result feedback based on the score of the first evaluation result, guide the change of the network structure;

[0077] After the result feedback based on the score of the first evaluation result, guide the change of the network structure to obtain a new denoising strategy.

[0078] Step S24: Use the changed network structure to construct a new denoising strategy to process the clean signal to obtain the second denoising result;

[0079] Step S25: Evaluate the obtained second denoising result;

[0080] The evaluation method of this step is similar to that of the upper step S12, and will not be elaborated here.

[0081] Use the changed network structure to construct a new denoising strategy to process the EEG signals of the same batch, and also evaluate the obtained results and provide feedback to guide the optimization of the network sequential structure.

[0082] Step S26: After the result feedback based on the score of the second evaluation result, guide the change of the network structure;

[0083] Repeat the above steps to obtain the optimal joint denoising method for multi-channel EEG signals.

[0084] Repeat the iteration according to this process, and finally obtain the optimal joint denoising method for this batch of EEG signals.

[0085] By training on a large number of datasets, an adaptive denoising structure decision that adapts to different scenarios and different subjects can be obtained, and a denoising strategy can be dynamically generated in actual applications for efficient denoising to restore clean EEG signals with high signal-to-noise ratio and saliency.

[0086] During the training process, the correction of the denoising strategy largely depends on the evaluation method. To effectively utilize a large number of datasets to obtain results with strong generality, the cross-validation method is adopted.

[0087] In one embodiment, step S22 includes:

[0088] Step S221: Use the cross-validation strategy to evaluate the algorithm model.

[0089] Specifically, step S221 includes:

[0090] Step S2211: For each subject, after dividing all of their EEG signals into N equal data subsets, use one of the subsets as the test set, and the other N - 1 subsets form the training set;

[0091] Step S2212: Evaluate the algorithm model based on the test set and the training set to obtain a score for the data of one subject;

[0092] Step S2213: Use the average score of all subjects as the final score, where the average score of the cross-validation is used as the score of one subject.

[0093] After obtaining the clean EEG signals through the joint denoising method, use the cross-validation strategy to evaluate the optimization degree of the algorithm model for the original EEG signals. This strategy uses a dataset covering all data to train the model, which can provide more reliable accuracy. For each subject, after dividing all of their EEG signals into N equal data subsets, let one of the subsets be used as the test set, and the other N - 1 subsets form the training set. This process is repeated N times to obtain a score for the data of one subject. Use the average score of the cross-validation as the result of one subject, and then use the average score of all subjects as the final score.

[0094] S3: Adopt spatio-temporal attention modeling to perform detail restoration on the clean signal to obtain enhanced EEG signals.

[0095] Aiming at the high specificity among subjects during the EEG extraction process, introduce the spatio-temporal relationship modeling of the transformer attention mechanism, and utilize the inherent temporal continuity and spatial correlation of the EEG test signal flow to reconstruct signals with higher spatial resolution.

[0096] After making corresponding decisions for individual multi-channel signals using reinforcement learning and invoking the corresponding denoising process, a highly expressed clean signal can be obtained, but there will still be a certain degree of signal attenuation. To further enhance the feature information of EEG signals in the time and spatial dimensions, a spatio-temporal relationship model of EEG is established using the transformer attention mechanism to achieve joint temporal continuity and spatial correlation for enhancing spatial resolution.

[0097] Specifically, as Figure 4 shown, step S3 includes:

[0098] Step S31: Apply a spatial attention transformer in the feature channel dimension for encoding spatial features;

[0099] Specifically, step S31 includes:

[0100] Step S311: Independently partition and encode the clean signal;

[0101] Step S312: Extract features from the EEG signals of multiple independent channels after spatial encoding;

[0102] Step S313: Compress the extracted feature set into a feature vector through a compression module;

[0103] Step S314: Aggregate multi-dimensional features of the compressed feature vector through the activation operation of the activation module, where different channels correspond to their respective activation factors.

[0104] As Figure 5 shown, the Transformer spatial encoder consists of squeeze-and-excitation, which can extract channel attention features, thereby improving the expressive ability of the framework and increasing the sensitivity of the model to information features. The intra-block and inter-block attention mechanisms are added to the dense fusion module, enabling the network to selectively amplify valuable feature channels based on global information and suppress useless feature channels.

[0105] The clean EEG signal after denoising processing is still in the form of multi-channel data. To perform spatial encoding of the transformer, first, the multi-channel signal is independently partitioned and encoded, then features are extracted from the EEG signals of multiple independent channels after spatial encoding, the extracted feature set is input into the compression module to be compressed into a feature vector, and the compressed features are aggregated into multi-dimensional features through an excitation module that assigns corresponding activation factors to different channels.

[0106] Among them, the expression of the compression module is:

[0107]

[0108] In the formula, X ∈ RH×V×C is the output of the feature map extracted from the previous layer of the module. H and V represent two coordinate axes in the two-dimensional space, and C represents the channel dimension; X c ∈R H × V , where c ∈ [1, 2, ··· C], represents the feature map of channel c in X, and x c (i, j) represents the data point at position (i, j) in X c ; F sq (·) represents the feature compression along the spatial dimension, which compresses each two-dimensional feature X c into a real number z c , and this real number z c has a global perception field to a certain extent, and the dimension output matches the number of feature channel inputs.

[0109] Among them, the expression of the activation operation of the activation module is:

[0110] S = σ(W2δ(W1Z))

[0111] In the formula, S is the activation factor, δ represents the sigmoid function operation, and Z is the spatial quantity obtained after the compression of H and V; W1 and W2 respectively represent the total correlation between two different channels. Among them, the two groups of parameters and simulate the correlation between channels.

[0112] Among them, the activation module forms a bottleneck, controls the parameters by using two fully connected layers. Between the fully connected layers, it returns to the channel dimension of the transformed input through the dimensionality reduction layer, and adjusts through the activation factor to obtain the aggregation result of multi-dimensional features;

[0113] The expression of the aggregation of multi-dimensional features is:

[0114]

[0115] Among them, after the aggregation of multi-dimensional features, the input vector x c changes to s c represents the activation factor of channel c, and x c represents the value of the input matrix in channel c.

[0116] Through the above spatial coding method, the channel attention features of multi-dimensional EEG signals can be effectively extracted, fully reflecting the global features of EEG signals in space and the correlation between local features. In addition to spatial features, EEG test signals also have the temporal variation characteristics of a complete dynamic behavior. By establishing temporal dependence relationships, the relationships between various parts of the signal sequence can be more effectively utilized, providing parameters for the enhancement and expansion of spatial features. To achieve this goal, such as Figure 6As shown, the multi - head temporal attention mechanism is used to perceive the global temporal dependence of EEG signals.

[0117] Step S32: Slice the data in the time dimension for attention transformation to obtain attention features containing temporal correlations;

[0118] Specifically, as Figure 6 shown, step S32 includes:

[0119] Step S321: Encode and mark the positions of the clean signal in time series;

[0120] Step S322: Compress the encoded and marked clean signal into one - dimensional data;

[0121] Step S323: Divide the one - dimensional data into multiple small slices with overlap in the time dimension;

[0122] Step S324: Use the multi - head attention mechanism for multiple small slices to obtain attention features containing temporal correlations.

[0123] First, after encoding and marking the positions of the data in time series, the data is compressed into one - dimensional data of 1×T. Then, the data is divided into multiple small slices with overlap in the time dimension (non - overlapping division will lose some context information). The attention mechanism is used to obtain representations suitable for classification by perceiving global temporal features.

[0124] Different from the spatial transformer, the multi - head attention mechanism allows the model to learn dependencies from different perspectives. The input is divided into h smaller parts, and attention is performed in parallel. The outputs are concatenated and linearly transformed to obtain the original size. This process can be expressed as:

[0125] MHA(X Q ,X K .X V )=[head0,...,head h-1 W o

[0126] head i =Attention(X Q W i Q ,X K W i K ,X V W i V )

[0127] where Q, K, and V represent the vector query, key, and value respectively, and Denote the query, key, and value matrices obtained after linearly transforming the input state vector. Denote the linear transformation for obtaining the final output, head i Denote the attention matrix of the i-th part, and MHA is the final vector representation obtained after aggregating each part. The feed-forward module contains two fully connected layers, and the GeLU activation function is connected behind the multi-head attention to enhance the model's perception and non-linear learning ability. The input and output sizes of the feed-forward feature block are the same, and the internal size is multiplied. Layer normalization is set before the multi-head attention and the feed-forward module. Residual connections are also used for better training. The repetition of the multi-head attention and the feed-forward module three times is used to obtain the overall effect.

[0128] Step S33: Under the guidance of the temporal features, enhance the spatial features;

[0129] Step S34: Obtain the enhanced EEG signals according to the features in both the spatial and temporal dimensions.

[0130] Specifically, as Figure 7 shown, step S34 includes:

[0131] Step S341: Combine the features in both the spatial and temporal dimensions;

[0132] Step S342: Divide the combined features into multiple temporal segments;

[0133] Step S343: Jointly perform flattened block mapping reconstruction on N independent channels with multiple temporal segments to obtain the enhanced EEG signals.

[0134] Spatial encoding and temporal encoding respectively extract and model the features of different dimensions of a segment of input multi-channel EEG signals and generate augmentations, obtaining a multi-channel EEG sequence with enhanced spatial resolution (expansion of the number of electrode channels). The decoder part first divides the combined features into segments, and jointly performs flattened block mapping reconstruction on N independent channels with the temporal segments to obtain the restored EEG signals.

[0135] After signal reconstruction, the discriminator extracts the differences between the features obtained by reconstruction and the features of the matched high-quality EEG signals (clean EEG signals with a wider number of channels, higher sampling frequency, and preprocessed when the subjects are the same), guiding the reconstruction optimization of the spatial and temporal encodings in the generator.

[0136] For multi-channel long-sequence EEG signals, the effectiveness of the position induction deviation generated by the spatial transformer block sequence is combined with the performance of the temporal segment position attention encoding generated by the spatial transformer module. By comparing the signal generated by the decoder with the signal obtained under ideal conditions in the contrast sampling of the same subject, the cross-entropy loss is calculated and jointly trained to obtain the network parameters that can be used for the inverse modeling and restoration of EEG signals.

[0137] Among them, spatial attention transformers are applied in the feature channel dimension for encoding spatial features. The feature channel attention block assigns weights to different channels, enabling the model to selectively focus on more relevant channels, and then channel compression is performed to reduce the computational cost. The data is sliced in the time dimension for attention transformation to obtain attention features containing temporal correlations. Guided by the temporal features, the spatial features are enhanced. After combining the features in the spatial and temporal dimensions, the transformer decoder is used to map and reconstruct the enhanced independent parts to obtain the enhanced EEG signals.

[0138] The purpose of the present invention is to overcome the deficiencies of the above-mentioned existing methods, propose the removal of interference from raw EEG signals based on reinforcement learning and the extraction, reconstruction, and enhancement of the spatio-temporal relationship of EEG signals based on transformers, and achieve high-quality EEG signal restoration to improve the intelligent classification ability of EEG signals.

[0139] Based on the reinforcement learning mechanism, the denoising process of EEG signals is selected and optimized to obtain clean signals that remove various types of irrelevant interferences, and the overall signal-to-noise ratio and significance effect of EEG features are improved. On this basis, aiming at the high specificity among subjects during the EEG extraction process, a spatio-temporal relationship modeling introducing the transformer attention mechanism is adopted. Utilizing the inherent temporal continuity and spatial correlation of the EEG test signal flow, a signal with higher spatial resolution is reconstructed to expand the channel features of the brain source data. Finally, the purpose of improving the signal spatial resolution and thus the recognition accuracy is achieved.

[0140] In the traditional deep learning processing methods for bioelectric data, the method steps using recurrent neural networks or long short-term memory networks cannot be parallelized and have low efficiency, and the processing method using a convolutional neural network as the basic framework will lose some time series information. At the same time, EEG signals have the characteristics of multi-channel and multi-dimension. Some existing multi-classification simple integration strategies that stack the feature channels obtained from multiple results of one-to-many processing largely ignore the importance of different feature channels, so collaborative optimization is not well implemented.

[0141] Existing deep network models with the ability to extract and classify multi-dimensional features can improve the ability to extract and classify target features, but they are vulnerable to individual differences among different testers and clutter interference caused by emotional EEG signals. The present invention uses a deep reinforcement mechanism to suppress various multi-dimensional interferences, highlighting the parts of the EEG signals that contribute greatly to interpretation. By using a generative adversarial mechanism and a spatio-temporal feature modeling model, the channel weighting after self-attention spatial encoding can represent global features while also taking into account the mutual correlation characteristics of adjacent regions. Therefore, compared with existing methods, it can generate a more flexible and dynamic receptive field. On the time scale, due to overlapping sequence slicing and attention extraction, it can achieve adaptive cross-domain connection in the time series of EEG motor imagery signals. On the spatial scale, it can integrate spatial local mutual correlation and global temporal variation features of brain regions to restore and reconstruct EEG signals, effectively enhancing the global spatial distribution and providing high-quality signals for subsequent research.

[0142] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality" means two or more unless otherwise specifically defined.

[0143] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or specific data points described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or specific data points described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0144] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for enhancing electroencephalogram signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling, characterized in that It includes the following steps: S1. Obtain multi-channel electroencephalogram (EEG) signals; S2. Make an interference removal decision for the obtained multi-channel EEG signals based on multi-agent reinforcement learning to obtain clean signals; S3. Use spatio-temporal attention modeling to restore the details of the clean signals to obtain enhanced EEG signals; Step S2 includes: Step S21. Input the clean signals into a cascaded network containing all denoising algorithms for the first denoising; Step S22. Evaluate the result of the first denoising; Step S23. Provide feedback on the result according to the score of the first evaluation result and then guide the change of the network structure; Step S24. Use the changed network structure to construct a new denoising strategy to process the clean signals to obtain the second denoising result; Step S25. Evaluate the obtained second denoising result; Step S26. Provide feedback on the result according to the score of the second evaluation result and then guide the change of the network structure; Repeat the above steps to obtain the optimal joint denoising method for the multi-channel EEG signals; Step S3 includes: Step S31. Apply a spatial attention transformer in the feature channel dimension for encoding spatial features; Step S32. Slice the data in the time dimension for attention transformation to obtain attention features containing temporal correlations; Step S33. Enhance the spatial features under the guidance of the temporal features; Step S34. Obtain enhanced EEG signals according to the features in both spatio-temporal dimensions.

2. The EEG signal feature enhancement method based on reinforcement learning combined with denoising and spatio-temporal relationship modeling according to claim 1, wherein Step S22 includes: Step S221. Use a cross-validation strategy to evaluate the algorithm model.

3. The method for enhancing electroencephalogram signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling according to claim 2, wherein, Step S221 includes: Step S2211. For each subject, after dividing all their EEG signals into N equal data subsets, use one subset as the test set, and the other N - 1 subsets form the training set; Step S2212. Evaluate the algorithm model according to the test set and the training set to obtain the score of one subject's data; Step S2213. Take the average score of all subjects as the final score, where the average score of cross-validation is used as the score of one subject.

4. The EEG signal feature enhancement method based on reinforcement learning combined with denoising and spatio-temporal relationship modeling according to claim 1, characterized in that Step S31 includes: Step S311. Independently partition and encode the clean signals; Step S312. Extract features from the EEG signals of multiple independent channels after spatial encoding; Step S313. Compress the extracted feature set into a feature vector through a compression module; Step S314. Perform aggregation of multi-dimensional features on the compressed feature vector through the activation operation of an activation module, where different channels correspond to their respective activation factors.

5. The method for enhancing electroencephalogram signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling according to claim 4, wherein The expression of the compression module is: where X ∈ R H×V×C is the feature map output extracted from the previous layer of the module. H and V represent two coordinate axes in the two-dimensional space, and C represents the channel dimension; X c ∈ R H×V , c ∈ [1, 2, ··· C], represents the feature map of the c-th channel in X, and x c (i, j) represents the data point at the position (i, j) in X c ; F sq (·) represents the feature compression along the spatial dimension, compressing each two-dimensional feature X c into a real number.

6. The method for enhancing electroencephalogram signal features based on reinforcement learning combined with denoising and spatio-temporal relationship modeling according to claim 5, wherein The expression of the activation operation of the activation module is: S = σ(W2δ(W1Z)); where S is the activation factor, δ represents the sigmoid function operation, Z is the spatial quantity obtained after compression of H and V; W1 and W2 respectively represent the total correlations of two different channels; Obtain the aggregation result of the multi-dimensional features through the activation factor; The expression of the aggregation of the multi-dimensional features is: Among them, after multi-dimensional feature aggregation, the input vector x c changes to s c represents the activation factor of the c-th channel, and x c represents the value of the input matrix in the c-th channel.

7. The EEG signal feature enhancement method based on reinforcement learning combined with denoising and spatio-temporal relationship modeling according to claim 1, characterized in that Step S32 includes: Step S321: Encoding and marking the position of the clean signal in time series; Step S322: Compressing the clean signal after encoding and marking into one-dimensional data; Step S323: Dividing the one-dimensional data into multiple small slices with overlap in the time dimension; Step S324: Using a multi-head attention permissive model for multiple small slices to obtain attention features containing time series correlation.

8. The EEG signal feature enhancement method based on reinforcement learning combined with denoising and spatio-temporal relationship modeling according to claim 1, wherein Step S34 includes: Step S341: Combining the features in both spatial and temporal dimensions; Step S342: Dividing the combined features into segments to obtain multiple time series segments; Step S343: Jointly performing flattened block mapping reconstruction on multiple independent channels for multiple time series segments to obtain enhanced electroencephalogram signals.

Images