Target recognition task state electroencephalogram signal extraction method and system based on reinforcement learning

Abstract

The invention discloses a target recognition task state electroencephalogram signal extraction method and system based on reinforcement learning. The method comprises the following steps: collecting electroencephalogram data when a subject watches monitoring video content, and segmenting the electroencephalogram data in a sliding window mode to make an electroencephalogram sample; the obtained electroencephalogram samples are sequentially input into a feature extraction network, electroencephalogram information of different frequencies is captured through multi-scale feature extraction, high-dimensional data are processed through deep separable convolution, and an attention mechanism is added to focus on key features. The extracted features are used, the importance score of the sample is obtained through a task sensing network, and the importance score reflects task state information contained in each fragment. And based on the obtained importance score, splicing by using a plurality of electroencephalogram fragments highly related to the current task, and realizing target detection by using a classifier. According to the method, automatic extraction of important fragments is realized by utilizing a reinforcement learning method, redundant information is removed, the calculation amount is remarkably reduced, and the reliability of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the field of brain-computer collaboration technology, and specifically relates to a method and system for extracting target recognition task-state EEG signals based on reinforcement learning. Background Art

[0002] With the rapid development of science and technology, surveillance systems are increasingly used in modern society. From urban security and traffic management to operational monitoring of commercial venues, the number and coverage of surveillance cameras are constantly increasing. The massive amount of surveillance video data provides a rich information foundation for security monitoring, behavioral analysis, and event warning, but it also poses significant challenges. Direct processing of massive amounts of video information places extremely high demands on computing models, requiring not only powerful hardware support but also efficient algorithms to process and analyze the data. However, traditional processing methods often suffer from low accuracy and slow processing speed, making them difficult to meet the requirements of real-time performance and accuracy.

[0003] In the biomedical field, electroencephalogram (EEG) signals, as an important physiological signal, contain a wealth of information that can be used to study brain activity, diagnose neurological diseases, and develop brain-computer interfaces. Rapid serial visual presentation (RSVP) is a paradigm widely used in psychology and brain science. It studies the brain's processing of visual information by presenting visual stimuli in rapid succession. In the RSVP task, EEG signals can reflect the brain's neural responses to target stimuli, such as event-related potentials (ERPs). However, EEG signals are characterized by high dimensionality, low signal-to-noise ratio, and significant individual variability, making it difficult for traditional feature extraction methods to effectively extract task-relevant features. EEG data in RSVP tasks often contain a large amount of redundant information. This redundancy not only increases the complexity of data processing but can also interfere with the extraction and analysis of key information. Accurately capturing task-state segments and removing irrelevant information is crucial for improving task analysis performance. Summary of the Invention

[0004] The purpose of the present invention is to address the shortcomings of the existing technology and propose a method and system for extracting EEG signals in target recognition task states based on reinforcement learning. The method is used to efficiently extract EEG features in rapid serial visual presentation (RSVP) tasks during electroencephalogram (EEG) signal processing and RSVP task analysis, and automatically capture task-state fragments through reinforcement learning to improve the accuracy and efficiency of target detection task analysis.

[0005] In a first aspect, the present invention provides a method for extracting EEG signals in a target recognition task state based on reinforcement learning, which comprises: Acquire multiple EEG segments based on EEG data.

[0006] The EEG segments are subjected to feature extraction based on the channel attention mechanism to obtain weighted EEG features.

[0007] The graph structure is constructed using weighted EEG features as graph nodes, and graph node features that fuse local and global information are extracted.

[0008] The importance score corresponding to each graph node feature is extracted through a bidirectional long short-term memory network.

[0009] The importance score corresponding to each graph node feature is input into the reinforcement learning method to generate the action for each graph node feature.

[0010] According to the action of each graph node feature, some weighted EEG features are extracted as a set of key EEG segments.

[0011] Preferably, the total reward function of the reinforcement learning method is ;in, R sim is the similarity reward function within the set; R dif is the inter-set difference reward function. The key EEG segment set is obtained by maximizing the total reward function R.

[0012] As a preference, the similarity reward function within the set R sim The similarity between different EEG segments within the key EEG segment set is represented by 1 minus the mean of the normalized distances between the corresponding vectors of different EEG segments within the key EEG segment set. R dif It is characterized by the degree of difference between the EEG segments in the key EEG segment set and the EEG segments in the remaining EEG segment sets, specifically the mean of the normalized distances between the corresponding vectors of the EEG segments in the key EEG segment set and the EEG segments in the remaining EEG segment sets.

[0013] Preferably, the feature extraction process includes multi-scale convolution, depth-separable convolution and feature extraction based on channel attention mechanism.

[0014] Preferably, the multi-scale convolution uses two parallel convolution layers with different convolution kernel sizes to capture features in time periods of different lengths. The output features of the two convolution layers are normalized and nonlinearly transformed before being concatenated.

[0015] Preferably, the depthwise separable convolution includes depthwise convolution and pointwise convolution.

[0016] Preferably, the process of feature extraction based on the channel attention mechanism is as follows: the features of the depthwise separable convolution output are compressed by adaptive average pooling, and then the attention weight of each channel is obtained through two fully connected layers and an activation function. The attention weight is applied to the EEG segment to obtain the weighted EEG feature.

[0017] Preferably, the EEG data is collected when the subject is watching the target video and is filtered. The EEG data is intercepted by a sliding window to obtain multiple EEG segments.

[0018] In a second aspect, the present invention provides a reinforcement learning-based target recognition task-state EEG signal extraction system, which is used to perform the aforementioned target recognition task-state EEG signal extraction method. The target recognition task-state EEG signal extraction system includes an EEG acquisition module, a preprocessing module, a feature extraction module, and a task perception module. The EEG acquisition module is used to collect EEG signals when the subject is viewing the target video. The preprocessing module is used to filter and segment the EEG signals.

[0019] The feature extraction module includes a multi-scale convolutional layer, a concatenation layer, a depthwise separable convolutional layer, and a channel attention mechanism extraction layer. The multi-scale convolutional layer includes two convolution blocks with different kernel sizes. The concatenation layer is used to concatenate the outputs of different convolution blocks in the multi-scale convolutional layer. The depthwise separable convolutional layer includes a series of depthwise convolution blocks and pointwise convolution blocks. The channel attention mechanism extraction layer includes an adaptive average pooling layer, two fully connected layers, and an activation function.

[0020] The task perception module is used to extract graph node features corresponding to EEG segments, extract importance scores corresponding to graph node features, and extract key EEG segments from all EEG segments through a reinforcement learning method.

[0021] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory stores the computer program; and the processor executes the aforementioned target recognition task-state EEG signal extraction method.

[0022] In a fourth aspect, the present invention provides a readable storage medium storing a computer program; when the computer program is executed by a processor, it is used to implement the aforementioned target recognition task-state EEG signal extraction method.

[0023] The present invention has the following beneficial effects: 1. This invention uses deep reinforcement learning to detect task-state segments in EEG signals for target recognition, without the need for label information. Traditional methods typically rely on fixed labels, but this invention can automatically detect task-related EEG segments and ignore irrelevant information. This allows the model to focus more on the task-related parts, thereby more accurately capturing dynamic EEG changes.

[0024] 2. This invention automatically selects task-state segments to compress high-dimensional data into low-dimensional features, significantly reducing computational and storage requirements. Low-dimensional features can be more easily integrated with data from other modalities (such as eye tracker data and physiological signals), further facilitating the research of cross-modal methods.

[0025] 3. The present invention processes EEG signals through a multi-layer convolution feature extraction scheme method of multi-scale convolution, depthwise separable convolution and channel attention mechanism, which significantly improves the model's ability to characterize EEG signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flowchart of Example 1 of the present invention.

[0027] Figure 2 This is a flow chart of EEG data collection in Example 1 of the present invention.

[0028] Figure 3 This is a flowchart of EEG segment feature extraction in Example 1 of the present invention.

[0029] Figure 4 Schematic diagram of the working process of the task perception module in Example 1 of the present invention. DETAILED DESCRIPTION

[0030] The method of the present invention is described in detail below with reference to the accompanying drawings.

[0031] Example 1 like Figure 1 As shown in the figure, a reinforcement learning-based target recognition task-state EEG signal extraction method is designed and used. A EEG signal extraction model that can effectively locate task-related information in EEG time series is designed and used to obtain better target detection performance.

[0032] The target recognition task state EEG signal extraction method comprises the following steps: Step 1: EEG data collection This example uses videos from multiple surveillance angles as stimuli. The target appears at different time points in the video, and each appearance lasts for the same duration. In this example, the target is a designated person. To increase the difficulty of the task, the target in the video may adopt disguises such as changing clothing to interfere with the subject's judgment. The video also contains other distracting factors such as non-target pedestrians and vehicles.

[0033] This example was conducted in a confined space, and the environment was kept quiet. The distance between the subject and the screen was approximately 75 cm, and the line of sight was level with the center of the screen. Before the start, a verification procedure was performed to ensure the validity of the data and the subject's ability to capture the target. Figure 2 As shown, this embodiment is divided into 4 blocks, each containing 100 video stimulus clips, of which half are target-oriented and half are non-target-oriented. Each video stimulus clip is 10 seconds long and has a resolution of 1920×1080. There is a 2-second interval between two video stimuli, during which a "+" sign appears in the center of the screen to help the subject focus. During this embodiment, the subject is required to stare at the center of the screen until the target appears, and then continue to focus on the target until it disappears. The subject can decide the rest time between each block according to his or her own situation to ensure that he or she maintains a good mental state and avoids excessive fatigue during this embodiment.

[0034] This embodiment uses a 64-lead Neuroscan device to obtain the subject's EEG data, adopts a 10-20 electrode placement system, and sets the data sampling frequency to 1000 Hz to ensure that the impedance of each electrode is below 15 kΩ. The sampling rate is set to 1000 Hz.

[0035] Step 2: Data preprocessing First, the EEG data was filtered using a 0.1-40Hz Butterworth filter. After removing damaged leads (M1 and M2), valid 62-lead data was obtained. The signal was then filtered to 0.1-40Hz. A sliding window approach was used to segment the EEG data into fixed time points. For each video, the EEG data was captured using a fixed-size sliding window.

[0036] In the subsequent steps, the EEG signal extraction model is used to extract key features from the preprocessed EEG data, and the specific process is shown in steps 3 and 4. The EEG signal extraction model includes a feature extraction module and a task perception module.

[0037] Step 3: Feature extraction like Figure 3 As shown in the figure, after the EEG data is segmented, the feature extraction module is used to extract features from the EEG segments. The feature extraction module includes a multi-scale convolution layer, a splicing layer, a depth-wise separable convolution layer, and a channel attention mechanism extraction layer. The channel attention mechanism extraction layer includes a pooling layer, two fully connected layers, and a ReLU activation function. The specific steps of feature extraction are as follows: Step 3.1: Features at different time scales may contain different information. To capture this information, the first convolutional layer of the feature extraction module adopts a multi-scale feature extraction strategy. Specifically, two parallel convolutional layers (Convolutional Layer 1 and Convolutional Layer 2) are used to extract features at different scales. Convolutional Layer 1 uses a larger convolution kernel of size (1, input size / / 2 + 1), which can capture features over longer time periods. Convolutional Layer 2 uses a smaller convolution kernel of size (1, input size / / 4 + 1), which can capture features over shorter time periods. The output features of both convolutional layers are normalized and nonlinearly transformed using batch normalization and GELU activation to enhance feature stability and discriminability. The normalized and nonlinearly transformed output features of the two convolutional layers are then concatenated along the channel dimension to form multi-scale features, which are then input to the depthwise separable convolutional layer.

[0038] In step 3.2, the second convolution layer of the feature extraction module uses depthwise separable convolution, which significantly reduces the number of model parameters while maintaining effective feature extraction. Depthwise separable convolution decomposes the standard convolution into two independent operations: depthwise convolution (performing a separate convolution on each input channel) and pointwise convolution (performing a pointwise convolution on the output of the depthwise convolution). This decomposition not only reduces computational effort but also enhances the model's feature extraction capabilities.

[0039] Step 3.3: To further enhance feature discriminability, the third convolutional layer of the feature extraction module introduces a channel attention mechanism. This mechanism adaptively adjusts the weights of different channels, enabling the model to focus on important channels while suppressing unimportant ones. First, the pooling layer compresses the feature maps output by the depthwise separable convolutional layer into a 1×1 matrix using adaptive average pooling (AdaptiveAvgPool2d) to capture global information. Then, through two fully connected layers (Conv2d) and a ReLU activation function, attention weights are calculated for each channel. Finally, the attention weights are normalized to the range [0, 1] using a Sigmoid activation function and applied to the feature map, enhancing the features of important channels and suppressing those of unimportant ones, resulting in weighted EEG features.

[0040] This attention mechanism can dynamically adjust channel weights, allowing the model to pay more attention to channels that contribute significantly to target recognition, thereby improving the discriminability of features.

[0041] Step 4: Figure 4 As shown in the figure, task perception is performed and a set of key EEG features is extracted.

[0042] Step 4.1. EEG signals are multi-channel time series data with complex spatiotemporal features. Signals from different electrodes may have synergistic or competitive relationships, and traditional methods such as pure time series models are difficult to effectively capture the spatial correlation between these nodes. Segments of EEG data (specific time windows) may have non-independent structured features, which are jointly analyzed here through graph structure modeling. The weighted EEG features corresponding to each EEG segment are regarded as graph nodes. , we define the graph structure by calculating the similarity between nodes and building an adjacency matrix: in, is the edge weight between node i and node j in the adjacency matrix; , is the feature vector of the i-th and j-th EEG segments; is the Euclidean norm; is the bandwidth parameter of the Gaussian kernel, which controls the sensitivity of the adjacency relationship.

[0043] By modeling the spatial and temporal dependencies between graph nodes through graph structure, the graph node features that integrate local and global information are extracted, the ability of the EEG signal extraction model to capture key EEG features is improved, and enhanced feature representation is provided for subsequent processing, assisting the task perception module to more accurately locate task-related EEG signals.

[0044] Step 4.2: In the task perception module, only EEG signals in certain time windows carry important information. The long-term and short-term dependencies of EEG signals are captured through a bidirectional long short-term memory network (Bi-LSTM), and the probability value of each EEG segment is output. , to indicate the importance score of the segment.

[0045] Bidirectional temporal modeling: for feature sequences enhanced by graphs , Bi-LSTM processes the sequence both forward and backward: in, is the input feature vector of the tth time step; is the hidden state of the forward LSTM at time step t; is the hidden state of the backward LSTM at time step t.

[0046] Importance score calculation: Fusing bidirectional hidden state information, outputting the importance probability of each EEG segment through the fully connected layer : in, is a learnable weight matrix; is a learnable bias parameter; is the Sigmoid activation function.

[0047] Step 4.3: Based on the segment importance score, the reinforcement learning agent takes action to select the EEG segment.

[0048] action gives: in, ∈{0,1} indicates whether the tth EEG segment belongs to the key segment; This is Bernoulli sampling.

[0049] mark The weighted EEG features corresponding to the EEG segments with a value of 1 constitute the key EEG segment set y. The weighted EEG features corresponding to the EEG segments with a value of 0 constitute the remaining EEG segment set y'.

[0050] In order to optimize the agent's selection of key segments in the task perception module and ensure coverage of all information points and discriminant points in the data, this embodiment uses a multi-objective reward function that comprehensively considers the similarity of sets and the separation between sets to optimize the training process. The total reward function R is expressed as follows: in, R sim is the similarity reward function within the set, specifically the similarity between different EEG segments within the key EEG segment set y. R dif is the inter-set difference reward function, specifically the degree of difference between the EEG segments in the key EEG segment set y and the EEG segments in the remaining EEG segment set y'. and is a non-negative weight hyperparameter used to balance the importance of different reward items.

[0051] Taking into account the variance of sample data in different dimensions and the correlation between dimensions, the two sample vectors The distance is expressed as in, Represents the inverse matrix of the feature covariance matrix.

[0052] For distance , the normalized distance is obtained by normalization calculation Intra-set similarity reward function R sim and inter-set difference reward function R difThe expression is as follows: in, 、 is the weighted EEG features corresponding to two different EEG segments in the key EEG segment set y. is the weighted EEG feature corresponding to the EEG segments in the remaining EEG segment set y'.

[0053] By maximizing the total reward function R, the task perception module tends to retain a set of key EEG segments with high internal similarity.

[0054] Step 5: Video object detection The key EEG segment set y is input into the target recognition model to obtain the recognition results of whether the target exists in the video and the time period when the target appears.

[0055] Comparative Example 1 A method for extracting EEG signals. The difference between this comparative example and Example 1 is that only steps 1 and 2 are performed, and steps 3 and 4 are not performed, and the EEG features obtained in step 2 are directly used for video target detection.

[0056] The ACC index was used to measure the performance difference between Example 1 and Comparative Example 1. The specific process was as follows: All sample data is divided into positive samples P and negative samples N. The EEGNet model is used as the target recognition model to identify the input EEG data. Therefore, the prediction results of the target recognition model have the following four situations: (1) TP: positive samples are predicted as positive samples; (2) TN: negative samples are predicted as negative samples; (3) FP: negative samples are predicted as positive samples; (4) FN: positive samples are predicted as negative samples.

[0057] ACC stands for recognition accuracy, which represents the ratio of correctly predicted samples to the total number of samples. When the number of samples in each category is similar, the accuracy rate can better reflect the performance of the model. It is calculated as follows: The accuracy rates ACC of Example 1 and Comparative Example 1 are shown in Table 1 below: The following table shows the effect of using this model to classify data: Table 1 Comparison of recognition accuracy between Example 1 and Comparative Example 1 According to Table 1, the RSVP task-state EEG signal extraction method based on reinforcement learning adopted in Example 1 is helpful to improve the accuracy of target recognition.

[0058] Example 2 A reinforcement learning-based target recognition task-state EEG signal extraction system is used to implement the target recognition task-state EEG signal extraction method of Example 1. The target recognition task-state EEG signal extraction system includes an EEG acquisition module, a preprocessing module, a feature extraction module, and a task perception module. The EEG acquisition module is used to collect EEG signals from a subject while viewing a target video. The preprocessing module is used to filter and segment the EEG signals.

[0059] The feature extraction module includes a multi-scale convolutional layer, a concatenation layer, a depthwise separable convolutional layer, and a channel attention mechanism extraction layer. The multi-scale convolutional layer includes two convolution blocks with different kernel sizes. The concatenation layer is used to concatenate the outputs of different convolution blocks in the multi-scale convolutional layer. The depthwise separable convolutional layer includes a series of depthwise convolution blocks and pointwise convolution blocks. The channel attention mechanism extraction layer includes an adaptive average pooling layer, two fully connected layers, and an activation function.

[0060] The task perception module uses a graph structure to treat the weighted EEG features corresponding to each EEG segment as graph nodes and extracts the graph node features corresponding to the EEG segment. A bidirectional long short-term memory network is then used to extract the importance scores corresponding to the graph node features. Reinforcement learning is then used to extract key EEG segments from all EEG segments. These key EEG segments are then input into the target recognition model to perform the target recognition task.

Claims

1. A target recognition task-state EEG signal extraction method based on reinforcement learning, characterized in that: The method comprises: Acquire multiple EEG segments based on EEG data; Perform feature extraction on EEG segments based on the channel attention mechanism to obtain weighted EEG features; Use weighted EEG features as graph nodes to build a graph structure and extract graph node features; Extract the importance score corresponding to each graph node feature through a bidirectional long short-term memory network; The importance score corresponding to each graph node feature is input into the reinforcement learning method to generate the action for each graph node feature; According to the action of each graph node feature, some weighted EEG features are extracted as a set of key EEG segments.

2. The target recognition task-state EEG signal extraction method according to claim 1, characterized in that: The total reward function of the reinforcement learning method ;in, R sim is the similarity reward function within the set; R dif is the inter-set difference reward function; the key EEG segment set is obtained by maximizing the total reward function R.

3. The target recognition task-state EEG signal extraction method according to claim 1, characterized in that: Intra-set similarity reward function R sim Representation of the similarity between different EEG segments within the key EEG segment set; difference reward function between sets R dif It is characterized by the difference between the EEG segments in the key EEG segment set and the EEG segments in the remaining EEG segment sets.

4. The target recognition task-state EEG signal extraction method according to claim 1, characterized in that: The feature extraction process includes multi-scale convolution, depth-separable convolution and feature extraction based on channel attention mechanism.

5. The target recognition task-state EEG signal extraction method according to claim 4, characterized in that: The multi-scale convolution uses two parallel convolution layers with different convolution kernel sizes to capture features in time periods of different lengths; the output features of the two convolution layers are normalized and nonlinearly transformed before being spliced.

6. The method for extracting EEG signals in target recognition task state according to claim 4, characterized in that: The process of feature extraction based on the channel attention mechanism is as follows: the features of the depthwise separable convolution output are compressed by adaptive average pooling, and then the attention weight of each channel is obtained through two fully connected layers and an activation function; the attention weight is applied to the EEG segment to obtain the weighted EEG feature.

7. The method for extracting EEG signals in target recognition task state according to claim 1, characterized in that: The EEG data is collected when the subject watches the target video and is filtered; the EEG data is intercepted through a sliding window to obtain multiple EEG segments.

8. A target recognition task-state EEG signal extraction system based on reinforcement learning, characterized by: Used to execute the target recognition task-state EEG signal extraction method according to claim 1; the target recognition task-state EEG signal extraction system includes an EEG acquisition module, a preprocessing module, a feature extraction module and a task perception module; the EEG acquisition module is used to collect EEG signals when the subject views the target video; the preprocessing module is used to filter and segment the EEG signals; The feature extraction module includes a multi-scale convolution layer, a splicing layer, a depth-wise separable convolution layer and a channel attention mechanism extraction layer; the multi-scale convolution layer includes two convolution blocks with different convolution kernel sizes; the splicing layer is used to splice the outputs of different convolution blocks in the multi-scale convolution layer; the depth-wise separable convolution layer includes a series of depth-wise convolution blocks and a point-by-point convolution block; the channel attention mechanism extraction layer includes an adaptive average pooling layer, two fully connected layers and an activation function; The task perception module is used to extract graph node features corresponding to EEG segments, extract importance scores corresponding to graph node features, and extract key EEG segments from all EEG segments through a reinforcement learning method.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The memory stores a computer program; the processor executes the target recognition task-state EEG signal extraction method according to any one of claims 1 to 7.

10. A readable storage medium storing a computer program; characterized in that: When the computer program is executed by a processor, it is used to implement the target recognition task-state EEG signal extraction method according to any one of claims 1 to 7.

Images