Target detection method based on dual reinforcement learning of mutual generation of electroencephalogram data

Abstract

The application discloses a target detection method based on dual reinforcement learning electroencephalogram data mutual generation. The method collects electroencephalogram signal data of multiple subjects when the subjects watch the same label sequence. Then, any two subjects are paired to try to generate electroencephalogram signals mutually, and a network model for generating electroencephalogram signals is trained based on a dual learning method. Then, the electroencephalogram data samples of a pair of subjects are input into the trained generation network to obtain an electroencephalogram data pair. The generated electroencephalogram data of each subject and the original electroencephalogram data are subjected to correlation analysis, and the generated electroencephalogram data with the highest correlation is selected as the final reinforced electroencephalogram data of the subject. The reinforced electroencephalogram data and the original electroencephalogram data are used to train a detection model together, and finally, the detection model is used to detect the target of the electroencephalogram signal of a single subject in the test stage, so as to solve the problem that a single subject is prone to inattention and target loss, resulting in poor detection results.

Description

Technical Field

[0001] This invention belongs to the field of brain-computer interface technology and relates to EEG generation methods, specifically to a target detection method based on the mutual generation of EEG data from dual reinforcement learning. Background Technology

[0002] Brain-computer interfaces (BCIs) can translate brain information into commands, enabling direct communication between the brain and external devices. Their applications include helping to extend the motor abilities of patients with diseases; for example, the gaze-independent BCI system can be used to restore the communication abilities of patients with oculomotor disorders. They also leverage the powerful visual perception of humans to collaborate with machines to achieve intelligent enhancement, such as in the military field for finding target images from a large number of pictures, including target detection and localization. In EEG target interpretation, a reliable method is to use the brain's spatial attention mechanism, which utilizes the spatial distribution characteristics of the brain after an event stimulus to determine the presence of a target in the visual field.

[0003] P300 is an event-related potential (ERP) that occurs during cognitive processes in the brain, primarily related to psychological factors such as expectation, motivation, arousal, and attention. Sutton et al. discovered that when the human brain is stimulated by a low-probability related event, a positive peak with a latency of approximately 300 ms appears in the electroencephalogram (EEG), hence the name P300. As a fundamental characteristic of ERPs, P300 has wide applications in EEG signal research.

[0004] Since P300 is an EEG ERP feature triggered by a low-probability event, the probability of the target appearing during the experiment cannot be too high. This leads to an imbalance in the ratio of positive to negative samples, resulting in an imbalance in sample size. Furthermore, due to the suddenness of the target's appearance, subjects' attention cannot remain consistently focused, causing them to miss the target. Therefore, the detection results are easily influenced by the subject's state. Conducting experiments with multiple subjects can mitigate these problems. Current technologies typically perform feature extraction and classification on multiple subjects separately, then fuse the classification results for decision-making. However, there is no direct interaction between the multiple subjects, thus limiting the improvement in target detection performance. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a target detection method based on the mutual generation of EEG data from dual reinforcement learning. This method utilizes the EEG signals generated when two subjects perform the same experiment to generate a target detection method from each other, thus solving problems such as missed targets, misidentification of targets, and unclear P300 signals that are common with single subjects, thereby improving training effectiveness and target recognition accuracy.

[0006] The target detection method based on the mutual generation of EEG data through dual reinforcement learning includes the following steps:

[0007] Step 1: Acquire spatial localization EEG data

[0008] The Rapid Visual Target Presentation (RSVP) paradigm in brain-computer interfaces (BCI) was employed to collect EEG data from multiple subjects viewing consecutive images. The content of the image sequences may differ, but the label sequences are identical. The collected EEG data from each subject was segmented into N samples. C represents the number of channels in the EEG data, and T represents the duration of a sample. The presence or absence of a target is used as the category label for the sample, with the presence of a target considered a positive sample and the absence of a target considered a negative sample.

[0009] Step 2: Data Preprocessing and EEG Analysis

[0010] Channel selection, filtering, and independent component analysis were performed on the EEG data from step 1 to remove obvious noise signals unrelated to the principal components. All data were then decentered and baseline calibrated.

[0011] Step 3: Dual Reinforcement

[0012] Two subjects were randomly paired and attempted to generate EEG signals for each other. For the EEG data A and B of the two subjects, the EEG data A and B were first input into two structurally identical generative networks Unet-A and Unet-B, respectively, to obtain the corresponding generated EEG data B' and A'. Then, the generated EEG data B' and A' were input into the generative networks Unet-B and Unet-A, respectively, to obtain the corresponding enhanced EEG data A” and B”.

[0013] The loss between the original EEG data and the generated and enhanced EEG data is calculated, and the model parameters of the Unet-A and Unet-B networks are updated to make the network-enhanced EEG data closer to the input real EEG data.

[0014] Step 4: Data Analysis and Sample Pair Selection

[0015] A pair of subject EEG data samples are input into the generative network trained in step 3 to obtain EEG data pairs generated after dual reinforcement learning. Correlation analysis is performed on the generated EEG data and the original EEG data for each subject, and the generated EEG data with the highest correlation is selected as the final reinforcement EEG data for that subject.

[0016] Preferably, the correlation analysis includes Pearman correlation coefficient calculation, Pearson correlation coefficient calculation, multivariate pattern analysis, or canonical correlation analysis.

[0017] Step 5: Model Training

[0018] The enhanced EEG data obtained in step 4 is input together with the original EEG data into a detection model for feature extraction and classification. The classification results are then compared with the labels of the original EEG data to complete the training of the detection model.

[0019] As a preferred approach, the enhanced EEG data obtained in step 4 is input together with the original EEG data into the EEGNet network for feature extraction. The resulting feature vectors are mapped to a low-dimensional vector space to obtain the final decision. Cross-entropy loss is then used for backpropagation to update the model parameters, making the model prediction more accurate.

[0020] Step 6: Target Detection

[0021] The subject's EEG signals were collected and input into the detection model trained in step 5 to obtain the target detection results.

[0022] The present invention has the following beneficial effects:

[0023] This method, building upon single-subject complex target detection, employs multi-subject collaborative detection and interactive EEG data generation among multiple participants. This effectively reduces issues prone to occur with single subjects, such as inattention, target loss, experimental fatigue, and contamination of the collected EEG data, thus significantly improving model training performance. Furthermore, it utilizes dual learning to construct a cycle consistency loss mechanism, enhancing the accuracy of EEG generation and making training more effective. The method of mutual EEG generation also offers a novel solution to the cross-subject problem in brain-computer interfaces. Attached Figure Description

[0024] Figure 1 This is a flowchart of a target detection method based on the mutual generation of EEG data using dual reinforcement learning;

[0025] Figure 2 Screenshot of the experimental paradigm used in the examples;

[0026] Figure 3 This is a schematic diagram of the model structure used for generating EEG data in the embodiment;

[0027] Figure 4 This is a flowchart of the dual reinforcement learning implementation in the example;

[0028] Figure 5 This is a comparison chart of the generated EEG data and the actual EEG data in the example. Detailed Implementation

[0029] The present invention will be further explained below with reference to the accompanying drawings;

[0030] like Figure 1As shown, the target detection method based on the mutual generation of EEG data using dual reinforcement learning has the following specific steps:

[0031] Step 1: Acquire spatial localization EEG data

[0032] Experimental design is required before collecting EEG data. This experiment uses the Rapid Sequence Visual Presentation (RSVP) paradigm in brain-computer interfaces (BCI). Images with or without targets will be randomly selected for presentation. Figure 2 As shown, the target is marked with a box in the left image; however, the box was not used during the experiment. Each image was presented for 500ms, with a total of 5500 images. The ratio of images with and without targets was 1:10. During EEG data acquisition, the subject was approximately 70cm away from the screen displaying the images. A 64-channel Neuroscan device was used for non-invasive acquisition, employing a 10-20 electrode placement method. Before acquisition, the impedance of all electrodes was kept below 15Ω, and the sampling rate was 1000Hz. The subject completed five viewing sessions, each consisting of 110 images. After each set of images was played, a "+" sign appeared in the center of the screen for 5 seconds, allowing the subject a break to refocus. After viewing five sets, the subject could either click the mouse to start the next round of the experiment or choose to rest. A total of four sets of experiments were conducted. During the viewing of the image sequence, the subject needed to constantly monitor whether the target appeared. Multiple subjects participated in the experiment separately, so each subject saw different images during the experiment, but the label sequence remained unchanged.

[0033] Step 2: Data Preprocessing and EEG Analysis

[0034] The acquired EEG data was first subjected to Butterworth bandpass filtering to a frequency range of 0.1-40Hz. Then, the 1000Hz EEG data was downsampled to 200Hz. Finally, the continuous EEG data was divided into multiple segments, and baseline calibration was performed on the 0-1000ms segment following the target's appearance using the 250ms data preceding the target's appearance. Since an excessively high number of channels is detrimental to the generative model's learning, only 17 key channels from the original 62 channels were selected to reduce energy consumption and improve training efficiency during model training.

[0035] Step 3: Dual Reinforcement

[0036] The EEG generation uses a U-shaped network (UNet), such as... Figure 3 As shown, the samples are first fed into the model. It is represented as [m,1,c,t], where m represents the number of samples in a batch, 1 represents the feature dimension of the sample, c represents the number of electrodes, i.e. the number of channels, and t represents the sample length.

[0037] First, calculate the mean and standard deviation (std) for a sample in a batch:

[0038]

[0039]

[0040] Where, q j Let j represent the j-th sample.

[0041] Perform sample standardization:

[0042] q′ j =(q j -mean) / std

[0043] After performing the same processing on EEG data samples from two subjects, the samples were input into two identical Unet networks for compression and extraction operations. The Unet network uses one-dimensional convolution operations, first increasing the dimensionality in the channel dimension and then reducing the dimensionality in the time dimension to obtain the core data features, which are then amplified to produce generated EEG data with the same shape as the input EEG data.

[0044] Building on this, dual learning methods, such as Figure 4 As shown, the actual EEG data of SubA is first input into a generative model G. After extracting the core features of SubA's EEG in model G, these core features are used to generate the generated EEG of SubB. This generated EEG is then input into another, inverse generative model F to extract the core features of SubB's generated EEG, which are then amplified to obtain the generated EEG of SubA, thus forming a loop structure. This loop's feedforward process is denoted as the initial process. Due to the existence of a symmetrical generation process, the two are consistent, and a generation loss is constructed based on this process. and cycle consistency loss The generation loss Using mean square loss and cross-entropy loss To indicate:

[0045]

[0046]

[0047]

[0048]

[0049] in, This represents the cosine similarity between the generated data and the real data. The cycle consistency loss calculates the 2-norm of the data after passing through the entire cycle and the initial input data, i.e., the spatial distance between the two vectors. The generator used in the reverse dual process shares parameters and is updated synchronously with the generator in the initial process. This forms two symmetrical cycles to enhance the generative model. The Spearman correlation coefficient is used to evaluate the generation performance after each training round.

[0050]

[0051] Where ρ is the Spearman correlation coefficient, and d i It is the rank difference of each pair of variables, i.e., d i =rank(X) i )-rank(Y i ), where n is the number of observations. The model that generates the best results is retained.

[0052] Step 4: Data Analysis and Sample Pair Selection

[0053] Using the model trained in step 3, generate test data and calculate the Pearson correlation coefficient r between any two subjects:

[0054]

[0055] Where X i and Y i These are the i-th observations of variables X and Y, respectively. and These are the means of variables X and Y, respectively. This coefficient can be used to assess whether SubA is suitable for generating data from SubB. Representational similarity analysis (RSA) is then used to analyze the neural representations in the generated and real data. Based on the similarity calculation results, the best-performing subject pairs are selected for subsequent classification training.

[0056] The generated EEG data is destandardized using the mean and variance of the original EEG data, and then averaged across all channels to obtain the following: Figure 5 The comparison charts shown depict the generated and original data, with thick lines representing the generated data and thin lines representing the original data. The first chart shows the result without dual reinforcement learning; the generated data only shows a portion of the waveform's trend and does not match the amplitude. The following three charts use the dual reinforcement model, resulting in data that is more accurate in both waveform and amplitude, closely resembling the real data.

[0057] Step 5: EEG Feature Extraction and Classification

[0058] After selecting suitable subject pairs according to step 4, the EEG signals used for training between the two subjects are converted to generate entirely new samples, and all generated samples are used as the training set. The original and generated EEG signals of one subject are shuffled and input into EEGNet to obtain the feature vector of the sample. Specifically, the EEG signal is first amplified through convolutional layer 1, then compressed through convolutional layer 2 to extract channel features, which are then input into the activation function ELU, followed by average pooling. Subsequent operations are similar, performing feature extraction along the time dimension, finally obtaining a one-dimensional column vector as the feature extracted by the network. In addition to convolutional layers, the dense layers also include a dropout layer with a dropout rate of 0.5 to prevent overfitting. The obtained feature vector is mapped to a low-dimensional vector space using a fully connected layer to obtain the final decision vector. The decision vector is output, and the softmax activation function is used to obtain the decision value. This value is compared with the label, and backpropagation is performed using cross-entropy loss to update the model parameters, making the model's predictions more accurate.

[0059]

[0060] p(x i ) represents sample x i The probability of belonging to a positive sample, q(x) i The value represents the probability of belonging to a negative sample. Using cross-entropy loss, an inter-class competition mechanism is employed, allowing the learning of inter-class information. However, cross-entropy loss only considers the accuracy of the predicted probability for the correct label, ignoring the differences in other incorrect labels, which can lead to scattered learned features. This application uses cross-entropy loss for classifier learning and inter-class metric loss for feature extractor, effectively combining the advantages of both and reducing their respective drawbacks on the network classification model.

[0061] Step 6: Target Detection

[0062] Through the above steps, the classifier can accurately classify whether the target has appeared. In offline detection systems, due to the instability of a single subject and the deliberate disguise of the target, there is a high probability of missed detection. This method greatly improves the stability of detection by a single subject by using multi-subject EEG-assisted training and multi-subject EEG information interaction, thus alleviating problems such as missed detection and false detection.

Claims

1. A target detection method based on the mutual generation of EEG data using dual reinforcement learning, employing the RSVP paradigm in brain-computer interfaces, collects EEG data from multiple subjects viewing consecutive image sequences, wherein the consecutive image sequences have the same label; the collected EEG data from each subject is segmented into N samples. Where C represents the number of channels in the EEG data and T represents the time length of a sample, the feature is: The specific steps are as follows: Step 1: Data Preprocessing and EEG Analysis Channel selection, filtering, and independent component analysis were performed on the collected EEG data to remove obvious noise signals unrelated to the principal components. All data were then decentered and baseline calibrated. Step 2, Dual Reinforcement For the EEG data A and B of the two subjects after processing in step 1, the EEG data A and B are first input into two identical generative networks Unet-A and Unet-B, respectively, to obtain the corresponding generated EEG data B' and A'; then the generated EEG data B' and A' are input into the generative networks Unet-B and Unet-A, respectively, to obtain the corresponding enhanced EEG data A” and B”. Using generation loss To characterize the loss between raw EEG data and generated EEG data: Using cycle consistency loss To characterize the loss between raw EEG data and enhanced EEG data: in, This represents the cosine similarity between the generated data and the real data. These represent the generated EEG data and the raw EEG data, respectively. This indicates enhanced EEG data; , These represent the raw EEG data and generated EEG data for the c-th channel at the t-th time period, respectively; update the model parameters of networks Unet-A and Unet-B; Step 3: Data Analysis and Sample Pair Selection Input a pair of EEG data samples from the subjects into the generative network trained in step 2 to obtain EEG data pairs generated after dual reinforcement learning; perform correlation analysis between the generated EEG data and the original EEG data for each subject, and select the generated EEG data with the highest correlation as the final reinforcement EEG data for that subject. Step 4: Model Training The enhanced EEG data obtained in step 3 is input together with the original EEG data into a detection model for feature extraction and classification. The classification results are then compared with the labels of the original EEG data to complete the training of the detection model. Step 5: Target Detection The subject's EEG signals were collected and input into the detection model trained in step 4 to obtain the target detection results.

2. The target detection method based on the mutual generation of EEG data through dual reinforcement learning as described in claim 1, characterized in that: The acquired EEG data were subjected to Butterworth bandpass filtering to 0.1-40Hz, then downsampled to 200Hz, and finally the continuous EEG data were divided into multiple segments. The data segment from 0-1000ms after the target appeared was baseline calibrated using the data from 250ms before the target appeared.

3. The target detection method based on the mutual generation of EEG data through dual reinforcement learning as described in claim 1, characterized in that: The data processed in step 1 is standardized and then input into the generator network.

4. The target detection method based on the mutual generation of EEG data through dual reinforcement learning as described in claim 1, characterized in that: After each training round, the performance of the generated network is evaluated using the Spearman correlation coefficient, and the network parameters with the highest Spearman correlation coefficient are retained.

5. The target detection method based on the mutual generation of EEG data through dual reinforcement learning as described in claim 1, characterized in that: The correlation analysis includes Spearman correlation coefficient calculation, Pearson correlation coefficient calculation, or canonical correlation analysis.

6. The target detection method based on the mutual generation of EEG data through dual reinforcement learning as described in claim 1, characterized in that: EEGNet was used as the detection model.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1 to 6.

Images