A method and system for fast target detection based on electroencephalogram and image-semantic prior

Abstract

The application discloses a kind of electroencephalogram and image-semantic priori's fast target detection method and system, belong to brain-computer interface technical field.The method includes: obtaining RSVP task under electroencephalogram signal, stimulus image and text description and pre-processing;Through statistical channel attention, transient motion incentive and bidirectional Mamba, extract electroencephalogram neurodynamics feature;Through CLIP visual encoder, combined token space attention focus and text semantic injection, generate semantic anchor feature;Through cross-modal bidirectional condition matching module, by visual semantic driven neural representation refining and neural response driven visual saliency focus iterative alignment, obtain fusion feature;Finally, the detection result is classified and output by gated temporal reconstruction.The application introduces image-semantic priori and physiological perception mechanism, constructs bidirectional cross-modal alignment and gated temporal repair structure, significantly improves the detection accuracy and stability under cross-task zero calibration, with excellent generalization ability and real-time performance.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface and target detection technology, specifically to a rapid target detection method and system that integrates EEG and image-semantic priors. Background Technology

[0002] Brain-computer interface systems based on rapid sequence visual representation (RSVP) achieve target recognition by capturing event-related potentials (ERPs). Existing methods perform well in decoding EEG signals for single tasks, but their decoding performance significantly degrades when faced with new tasks lacking calibration data, hindering rapid deployment and cross-task transfer. Traditional models such as HDCA and MDRM have weak signal-to-noise ratio adaptability and poor cross-task feature transferability. While recent models such as Transformer and TCN have shown improvement, they still struggle to effectively integrate multimodal information, exhibiting semantic mismatch and temporal information loss. Furthermore, existing methods fail to fully utilize prior semantic information from images to guide neural feature extraction, limiting generalization ability across cross-task scenarios. Therefore, a high-precision, robust RSVP decoding method capable of achieving target detection without calibration data is urgently needed. Summary of the Invention

[0003] This invention aims to overcome the performance degradation of RSVP decoding across tasks with zero calibration in existing technologies, and provides a bidirectional enhanced target detection method and system that integrates image-semantic priors and EEG signals. By constructing physiological perception feature extraction, semantic anchor encoding, cross-modal bidirectional alignment and gated temporal reconstruction mechanisms, it achieves high-precision, high-robustness, and rapid target detection without the need for new task calibration.

[0004] This invention proposes a fast target detection method based on EEG and image-semantic prior, implemented using the Semantic-Anchored MambaEEG (SA-MambaEEG) framework, comprising the following steps:

[0005] Step S1: Acquire the user's electroencephalogram (EEG) signals, stimulus images, and task-related text descriptions under the Rapid Sequence Visual Presentation (RSVP) task, and preprocess the EEG signals, stimulus images, and text descriptions. Step S2: Input the preprocessed EEG signals into the physiological perception MambaEEG feature extractor to extract EEG neurodynamic features. Step S3: Encode the preprocessed stimulus images and text descriptions to generate semantic anchor features. Step S4: Input the EEG neurodynamic features and semantic anchor features into the cross-modal bidirectional conditional matching module for iterative alignment and co-evolution to obtain fused features. Step S5: Perform temporal reconstruction fusion and classification on the fused features, and output the target detection results.

[0006] Preferably, the extraction of EEG neurodynamic features in step S2 includes: Step S2.1: Channel reweighting of EEG signals is performed using a statistical channel attention module. The statistical channel attention module simultaneously calculates the global average pooling and global standard deviation pooling of the input features in the time domain, concatenates the two in the channel dimension to obtain a statistical descriptor, generates channel attention weights through two fully connected layers and an activation function, and performs residual connection with the original features to obtain channel-weighted features; Step S2.2: Channel-weighted features are subjected to channel dimensionality reduction convolution and time-domain convolution by a transient motion excitation module, and their first-order temporal difference in the time domain is calculated. The difference features are subjected to global average pooling, and then generated as dynamic excitation weights through a multilayer perceptron and a sigmoid function. These weights are then residually connected with the channel-weighted features to obtain transient enhancement features; Step S2.3: Transient enhancement features are scanned forward and backward using a bidirectional Mamba encoder, and bidirectional contextual features are fused using an element-wise addition method. After layer normalization, EEG neurodynamic features are output.

[0007] Preferably, the calculation formula for the channel attention weights generated by the statistical channel attention module in step S2.1 is as follows: , Where Concat(·) is the feature concatenation operation. For the input EEG features, AvgPool(·) and StdPool(·) are global average pooling and global standard deviation pooling, respectively. Let be the concatenated statistical descriptor, R be the set of real numbers, C be the dimension of the space (the concatenated dimension is C + C = 2C), W1 and W2 be the weight matrices of the fully connected layer, δ(·) be the ReLU activation function, and σ(·) be the Sigmoid activation function. Weighted features for the output channels.

[0008] Preferably, the calculation formula for the dynamic excitation weights generated by the transient motion excitation module in step S2.2 is as follows: , ,in, For time-series difference features, It is a temporal convolution operation. It is channel-reduced convolution. Weighted features for the input channels. For global average pooling, For a multilayer perceptron, σ(·) is the sigmoid activation function. Represents the difference feature at the t-th time step. For the difference feature at the (t-1)th time step, This is for the transient enhancement features of the output.

[0009] Preferably, the bidirectional Mamba encoder adopts a selective state-space model, specifically implemented as follows:

[0010] a. Discretize the continuous-time state-space model using zero-order preservation. The discretization parameter transformation formula is as follows:

[0011] ,

[0012] ,

[0013] in Let A be the discretized state transition matrix, exp(·) be the matrix exponentiation operation, Δ be the discretization time step, A be the state transition matrix, and P be the input projection matrix. is the discretized input projection matrix; I is the identity matrix.

[0014] b. Parameterize the discretized time step Δ, the input projection matrix P, and the projection matrix Ω into the current time step input feature vector. The function, through For the input vector Perform linear encoding operations to generate a linear code structure defined by Δ, P, and Ω, thereby achieving dynamic weight allocation.

[0015] c. In the selective scanning mechanism of the selective state space model, the dynamic adjustment of step size and weight is triggered by the amplitude and rate of change of the input EEG features. When the feature amplitude exceeds the preset threshold of the model, the model increases the attention weight and reduces the discretization time step Δ to achieve focus on significant neural responses.

[0016] Where Linear(x) t ) represents the input vector x t Perform linear encoding operations to obtain the corresponding codewords or encoding results.

[0017] Preferably, the generation of semantic anchor features in step S3 includes: Step S3.1: Extracting the visual token sequence of the stimulus image using a pre-trained CLIP-ViT visual encoder, wherein the CLIP-ViT encoder uses a parameter-freezing method; Step S3.2: Spatially focusing the visual tokens using a token space attention module to generate a spatial attention mask, the calculation formula being: ,in A visual token sequence, , The weight matrix is ​​a linear transformation, δ is the ReLU activation function, and σ is the Sigmoid activation function. Then, the attention mask and the original features are subjected to residual enhancement to obtain the focused visual features: Step S3.3: Input the preprocessed task-related text description into the CLIP text encoder to obtain a global semantic vector. The CLIP text encoder uses a parameter freezing method. The global semantic vector is then fused with the focused visual features element by element through a broadcast mechanism to generate semantic anchor features.

[0018] Preferably, in step S4, the cross-modal bidirectional conditional matching module first maps EEG neurodynamic features and semantic anchor features to a unified feature dimension D' through 1×1 convolutional layers and fully connected layers, and then sets up a bidirectional cross-attention structure with L layers executing in parallel, where D' and L are hyperparameters; the following sub-steps are executed in parallel in each layer:

[0019] Step S4.1: Visual semantic-driven neural representation refinement path, using EEG features as queries and visual features as keys and values, and refining EEG features through scaled dot product attention: ,in For the refined EEG characteristics of layer l, This is a query vector derived from electroencephalogram (EEG) signals. This is the key vector (Key) from the image modality. Let φ(·) be the value vector from the image modality, and let φ(·) be the content-based dynamic aggregation operation.

[0020] Step S4.2: The neural response-driven visual saliency focusing path uses visual features as queries and EEG features as keys and values, calculating the focusing visual features through scaled dot product attention: ,in, Let be the focused visual feature of the l-th layer, and φ(·) be the content-based dynamic aggregation operation. For the query vector of the image modality, The key vectors for EEG modalities. This is the value vector of the EEG modality;

[0021] Step S4.3: The refined EEG features and focused visual features output from each layer are spliced ​​together and then normalized by layer before being used as the input for the next layer.

[0022] The cross-modal bidirectional conditional matching module achieves iterative synergy between visual semantic-driven EEG feature refinement and neural response-driven visual feature focusing through the aforementioned L-layer parallel execution bidirectional cross-attention structure. This progressively eliminates intermodal semantic shifts, enabling efficient fusion and accurate matching of EEG and image-semantic features.

[0023] Preferably, the temporal reconstruction and classification in step S5 includes: fusing features The input-gated temporal reconstruction fusion module includes a lightweight unidirectional Mamba module and a learnable gating module. The hidden layer dimension of the lightweight unidirectional Mamba module is consistent with the dimension D' of the input fusion feature, performing temporal modeling on the fusion feature to generate temporal reconstruction features. Its parameter scale is 1 / 4 of that of the bidirectional Mamba encoder.

[0024] The learnable gating module normalizes the gating parameter γ using the Sigmoid function to generate adaptive weights. It then performs element-wise multiplication of the original fused features and the reconstructed features, followed by residual fusion to obtain the final features. : ,

[0025] in, These are learnable gating parameters used to dynamically control the contribution of reconstructed features to the final output, enabling adaptive residual fusion. The Sigmoid activation function is used. The final features are then processed by global average pooling and input into a fully connected layer, and finally passed through a Softmax classifier to output the object detection results.

[0026] Preferably, the L-layer bidirectional cross-attention structure of the cross-modal bidirectional conditional matching module has residual connections between it to achieve effective gradient backpropagation. Scaling dot product attention is used as a content-based dynamic aggregation operation φ(·) to achieve bidirectional collaboration between EEG refinement based on visual semantic constraints and visual focusing guided by neural response. After each layer of refined EEG features and focused visual features are concatenated, they are normalized and then input into the next layer. This iterative collaboration gradually eliminates the semantic offset between modalities. Furthermore, the cross-modal features are mapped to a unified dimension through 1×1 convolutional layers and fully connected layers, achieving efficient fusion and accurate matching of EEG and image-semantic features.

[0027] The present invention also provides a system for implementing the above method, comprising:

[0028] Data acquisition module: including a multi-channel EEG acquisition cap and image display, used to synchronously acquire the user's EEG signals while performing the RSVP task and synchronously present stimulation images;

[0029] Data preprocessing module: used to perform preprocessing on the raw EEG signal, such as filtering and slicing embedding, and to unify the format of images and text;

[0030] Physiologically Sensing Mamba EEG Feature Extractor: Built-in statistical channel attention module, transient motion excitation module and bidirectional Mamba encoder for extracting robust neurodynamic features from EEG signals;

[0031] Semantic anchor cue encoder: CLIP-ViT visual encoder with built-in frozen parameters, CLIP text encoder and token space attention module, used to generate semantic anchor features that match neural signals;

[0032] Cross-modal bidirectional conditional matching module: contains an L-layer parallel bidirectional cross-attention structure, used to achieve iterative alignment and co-evolution of EEG features and visual semantic features;

[0033] Gated timing reconstruction and fusion module: It has a built-in lightweight one-way Mamba module and a learnable gating module, which are used to repair timing continuity and output gated fusion features;

[0034] Classifier: Contains fully connected layers and a softmax activation function, used to output the final object detection result;

[0035] Central control unit: Used to coordinate the data flow and computing tasks of various modules.

[0036] Beneficial effects:

[0037] This invention addresses the technical pain point of poor detection performance in cross-task zero-calibration scenarios of RSVP brain-computer interfaces. It integrates image-semantic priors and EEG physiological perception mechanisms to construct a full-link architecture of bidirectional cross-modal alignment and gated temporal reconstruction, achieving a synergistic improvement in detection accuracy, stability, and generalization ability. The core beneficial effects are as follows:

[0038] 1. Excellent detection accuracy and stability: The SA-MambaEEG model of this invention achieved an average detection accuracy of 82.7% in 6 sets of uncalibrated cross-task tests, which is the best among all the comparison models; and the standard deviation is only 0.37-0.86, which is far lower than traditional and recently optimized models, with small fluctuations in detection results and outstanding robustness.

[0039] 2. Significantly enhanced cross-task generalization ability: Compared with traditional baseline models such as HDCA and MDRM, the cross-task accuracy is improved by ≥5.2%, solving the problems of weak signal-to-noise ratio adaptation and poor cross-task feature transfer; Compared with recent optimized models such as TCN-T and TFF-Former, it achieves deep alignment between neural representation and visual semantics through bidirectional cross-modal conditional matching, and maintains low-fluctuation detection performance in high-difficulty cross-task scenarios, with a significant improvement in cross-task adaptability under zero calibration.

[0040] 3. Improved efficiency of multimodal information fusion: Abandoning the traditional unidirectional feature mapping method, the L-layer parallel bidirectional cross-attention structure is used to achieve iterative synergy between visual semantic-driven EEG feature refinement and neural response-driven visual feature focusing, progressively eliminating semantic offset between modalities, and achieving efficient fusion and accurate matching of EEG and image-semantic features.

[0041] 4. Combining real-time performance with engineering feasibility: The feature extraction and fusion stages adopt a lightweight design, and the lightweight Mamba module significantly reduces computational overhead. The single-sample processing time on the GPU is ≤38ms, and on edge devices it is ≤50ms, meeting the real-time requirements of RSVP tasks. Each module of the model can be implemented based on a software framework or integrated into a dedicated chip to form a hardware module, supporting multi-platform deployment and having high value for engineering implementation and promotion.

[0042] 5. Feature extraction is more in line with the physiological mechanism of EEG: statistical channel attention fusion second-order statistics realize channel adaptive reweighting, transient motion excitation enhances the capture of transient responses of event-related potentials, bidirectional Mamba selective scanning mechanism accurately focuses on significant neural responses, and physiological perception design makes EEG feature extraction more in line with the brain's visual neural response pattern, thus improving the rationality and effectiveness of decoding. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the fast target detection method based on EEG and image-semantic priors according to the present invention.

[0044] Figure 2 This is a schematic diagram of the structure of a fast target detection system based on EEG and image-semantic priors according to the present invention. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the implementation of the present invention is not limited thereto.

[0046] Example 1: A fast target detection method based on EEG and image-semantic priors

[0047] This embodiment provides a fast target detection method based on EEG and image-semantic prior, which is implemented based on the Semantic-Anchored MambaEEG (SA-MambaEEG) framework. The overall process of the method is as follows: Figure 1 As shown, it includes the following steps:

[0048] Step S1: Data acquisition and preprocessing.

[0049] Acquire the electroencephalogram (EEG) signals, stimulus images, and task-related text descriptions of the user to be tested during the Rapid Sequence Visual Presentation (RSVP) task.

[0050] Specifically, EEG signals are acquired using a multi-channel EEG acquisition device with a sampling rate of 1000Hz. In this embodiment, 32-channel (C=32) scalp electrodes are used to acquire EEG data for 1 second, meaning each sample has 1000 temporal sampling points (T=1000). The raw EEG signals are sliced ​​and embedded to divide them into non-overlapping short time segments, and an initial feature matrix S of dimension [B,C,T] is obtained through linear mapping. eeg Where B is the batch size (B=256 in this embodiment), C is the number of EEG channels, and T is the number of time-domain sampling points. The stimulus images are the images that flash during the RSVP task, with the resolution uniformly adjusted to 224×224. The task-related text descriptions are phrases, such as "target car" or "nontarget".

[0051] The multi-channel EEG acquisition device uses a 32-channel EEG acquisition cap that conforms to the international 10-20 EEG electrode placement standard. The device has a sampling accuracy of ≥16 bits and a common-mode inhibition ratio of ≥110dB to ensure the acquisition accuracy of the original EEG signal. The image display is a high-definition LCD with a refresh rate of ≥144Hz, and the stimulus image presentation duration is 50-200ms, which meets the visual stimulus presentation standard of the RSVP task.

[0052] All preprocessing operations are performed through a computer device, which includes a processor, memory, and input / output interfaces. The memory stores executable preprocessing programs, and the processor calls these programs to perform preprocessing of EEG signals, stimulus images, and text descriptions.

[0053] EEG signal preprocessing: The original 32-channel EEG signal is subjected to 0.1-30Hz bandpass filtering and 50Hz power frequency notch filtering to remove high-frequency noise and power frequency interference; the short time segment length of the slice embedding processing is 100ms, with no overlapping division, and the linear mapping is implemented using a single fully connected layer with linear activation function to ensure that the feature dimension is mapped from the original sampled data to [B,C,T];

[0054] Stimulus image preprocessing: Stimulus images are uniformly converted to RGB three-channel format, and then center-cropped and normalized. The normalized mean is [0.485, 0.456, 0.406], and the standard deviation is [0.229, 0.224, 0.225], which is consistent with the preprocessing rules of the CLIP-ViT model's pre-training data.

[0055] Text description encoding preprocessing: Task-related text descriptions are uniformly expressed in lowercase English. After word segmentation, [CLS] start markers and [SEP] end markers are added to match the input format requirements of the CLIP text encoder.

[0056] Step S2: Physiological perception MambaEEG feature extraction.

[0057] The preprocessed EEG features S eeg The data is input into the physiological perception MambaEEG feature extractor to extract EEG neurodynamic features F. eeg This step further includes:

[0058] S2.1 Statistical Channel Attention (SCA) processing.

[0059] To address the issues of low signal-to-noise ratio in EEG signals and the tendency for information cancellation to occur with traditional global average pooling, we first employ the SCA (Statistical Channel Attention) module to process the input features. Reweighting is performed. This module introduces second-order statistics as a supplement, and aggregates the first moment (mean) and the second moment (standard deviation) to construct a robust channel descriptor.

[0060] For input EEG features (Dimensions [B, C, T]), calculate its global average pooling AvgPool in the time domain. ) and global standard deviation pooling StdPool( The two are then concatenated along the channel dimension to obtain the statistical descriptor Z. stat , is represented as:

[0061]

[0062] Here, `Concat(·)` is a feature concatenation operation, concatenating the mean and standard deviation features into a one-dimensional vector; `AvgPool(·)` is channel-wide average pooling, outputting a real vector of dimension C, specifically averaging the temporal features of each EEG channel to obtain a C-dimensional mean feature vector; `StdPool(·)` is channel-wide standard deviation pooling, outputting a real vector of dimension C, i.e., taking the standard deviation of the temporal features of each EEG channel to obtain a C-dimensional standard deviation feature vector; R represents the set of real numbers, C is the spatial dimension, and the concatenated dimension is C + C = 2C. It is the original EEG feature matrix, a real matrix of dimension [B,C,T], which is the input feature of the SCA (Statistical Channel Attention) module and is composed of the original EEG features after slice embedding.

[0063] Subsequently, Z stat Channel attention weights are generated through two fully connected layers and an activation function. Specifically, the weights are first passed through a weight matrix W1 (dimension [...]). The ReLU activation function δ(·) is mapped to the hidden layer, and then passed through the weight matrix W2 (dimension [C, C]) and the weight matrix W2 (dimension [C, C]). The attention weights are obtained by applying the sigmoid activation function σ(·) and then performing a residual connection with the original features to obtain the channel-weighted features X. sca :

[0064]

[0065] in, The hidden layer dimension of the fully connected layer is set to 64 in this embodiment; the W1 dimension is [2C, 64] and the W2 dimension is [64, C]. The dot (·) indicates channel-wise element-wise multiplication (broadcast operation), specifically broadcasting the C-dimensional attention weights to the [B, C, T] dimension, and then multiplying them with the T-dimensional data. Element-wise multiplication is performed; σ(·) is the Sigmoid activation function, which outputs a real vector in the interval [0,1] and normalizes the attention weights; δ(·) is the ReLU activation function, which outputs a non-negative real vector and introduces a nonlinear transformation.

[0066] By integrating second-order statistics, the SCA module can keenly detect key active channels with low mean but drastic fluctuations, and achieve adaptive reweighting of the signal-to-noise ratio.

[0067] S2.2 Transient Motion Excitation (ME) Processing.

[0068] To enhance the model's sensitivity to transient responses such as event-related potentials (ERPs), channel-weighted features are added. Input is fed into the ME (Motion Excitation) module. This module explicitly models neurodynamics by computing the first-order temporal difference of the feature sequence.

[0069] First, channel-reduced convolution is used. (1×1 convolution) and temporal convolution Extracting temporal difference features using one-dimensional convolution with a kernel size of 3. :

[0070]

[0071] in, It is a temporal convolution operation that operates on the temporal dimension T of EEG features to extract temporal dependency information; It is a channel-reducing convolution, usually a 1×1 convolution, which maps the high-dimensional channel features C to a lower dimension to reduce the amount of subsequent computation. These are channel-weighted EEG features output by the SCA module, with dimensions [B, C, T].

[0072] Then, calculate First-order difference in the time domain Where t is the temporal step index, a positive integer ranging from 1 to t. The differential feature is then subjected to global average pooling (Mean(·)) in the temporal domain to obtain a channel-level transient intensity descriptor. This descriptor is then processed by a multilayer perceptron (MLP) (consisting of two fully connected layers and an activation function) and a sigmoid function to generate dynamic activation weights. Finally, the transient enhancement feature is output through residual connections. :

[0073]

[0074] in, These are the transient enhanced EEG features finally output by the ME module, with dimensions [B,C,T], which are used for subsequent MambaEEG feature extraction. This represents element-wise multiplication; It is a multilayer perceptron; For global average pooling; Let represent the difference feature at the t-th time-domain step, with dimensions [B,C]. The differential feature at the (t-1)th time step is used to calculate the first-order time difference and capture the rate of change of potential.

[0075] This mechanism is similar to high-pass filtering, significantly enhancing transient edge features in the signal and forcing the model to focus on abrupt changes in the neural signal rather than a stable background, thereby improving the sensitivity to capture ERP waveforms. The ME module and SCA module are executed sequentially. First, the EEG features after slice embedding are weighted by the SCA channels, and then the weighted features are sent to the ME module for transient enhancement. The first-order temporal difference calculation of the ME module is performed in the time domain, and zero-padding is used for the boundary time domain steps to avoid loss of feature dimensions.

[0076] S2.3 Bidirectional Mamba Selective State Space Modeling.

[0077] Transient enhancement feature X me The input is fed into a bidirectional Mamba encoder for long-range temporal dependency modeling. Given the long sequence characteristics and non-stationary dynamic changes of EEG signals due to their high sampling rate, the Mamba architecture is introduced, and its selective scanning mechanism is used to achieve global context inference with linear complexity.

[0078] The basic form of a continuous-time state-space model is the state equation:

[0079]

[0080] Where A is the state transition matrix, representing the parameter matrix of the state evolution over time in the state-space model; P and Ω are the projection matrices, representing the core parameter matrices of the state-space model that realize the mapping from input features to the state space. State vector The derivative with respect to time t represents the rate of state evolution over time and is used to model the neurodynamic changes in EEG signals; The continuous-time state vector represents the potential state of the model at time t, encoding the long-range temporal dependency information of the EEG signal. The continuous-time input feature vector corresponds to the transient enhanced EEG features output by the ME module and is the input to the state-space model; The continuous-time output feature vector is the output of the state-space model and contains the modeled EEG neurodynamic information.

[0081] To process sampled EEG signals in a digital system, the continuous system is discretized using the zero-order hold technique. Given a time step Δ (in this embodiment, Δ = 1 / sampling rate = 1 / 1000s, a positive real number), the discretized parameters are transformed as follows:

[0082]

[0083]

[0084] in, is the discretized state transition matrix; exp(·) is the matrix exponentiation operation; Δ is the discretized time step (sampling period), which is numerically equal to the reciprocal of the EEG signal sampling rate and is the key parameter connecting continuous time t and discrete time step k; I is the discretized input projection matrix; I is the identity matrix; For matrix subtraction, the discretized state transition matrix is... Subtracting the identity matrix yields the matrix representation of the "pure state changes". These two formulas solve the problem of the evolution of continuous states within a fixed sampling interval, and are the core steps of discretization, ensuring the accuracy of the discretized model.

[0085] This leads to a discrete form that facilitates recursive computation:

[0086]

[0087]

[0088] in, The hidden state at the current time t is the model's "memory" or encoded representation of the previous t inputs in the sequence; This represents the hidden state of the previous moment t-1, signifying previous memories; This is the input vector at the current time t; This is the model output at the current time t.

[0089] Traditional state-space models have data-independent parameters, making them ill-suited to handling sudden artifacts or specific ERP patterns in EEG signals. Mamba's core innovation lies in its selectivity mechanism, which parameterizes the parameters (Δ, P, Ω) as input x. t Functions:

[0090]

[0091] Among them, Linear(x) t ) represents the input vector x t Performing linear encoding operations yields the corresponding codewords or encoding results. After `Linear(·)`, a linear code structure defined by Δ, P, and Ω is generated. This improvement endows the model with the ability to dynamically allocate weights: for moments containing electrooculogram artifacts or irrelevant background fluctuations, the model can "ignore" the current input by reducing the step size Δ or the weight P, thereby avoiding the state variable h from being affected. t The model is contaminated by noise; however, for moments that elicit a significant response, the model can adaptively "focus" and update its memory.

[0092] Furthermore, this embodiment employs a bidirectional Mamba structure to scan the sequence in both forward and reverse directions. Standard Mamba is a causal model (only able to see the past), but EEG decoding is typically an offline analysis task, where future contextual information is crucial for eliminating current ambiguity. Therefore, we designed a bidirectional Mamba encoder that scans the sequence in both forward and reverse directions. The final output feature is a fusion of bidirectional context: forward and reverse Mamba models the EEG features output from the ME module in terms of time sequence, yielding forward features [B,C,T] and reverse features [B,C,T]. Bidirectional feature fusion is performed using element-wise addition. After fusion, layer normalization operations are used to suppress feature distribution shifts, ultimately outputting EEG neurodynamic features with dimensions [B,C,T]. In the selective scanning mechanism of bidirectional Mamba, the dynamic adjustment of step size and weights is triggered by the amplitude and rate of change of the input EEG features. When the feature amplitude exceeds the set threshold (learned by the model during training), the model increases the attention weights and decreases the step size to focus on significant neural responses.

[0093] Step S3: Semantic anchor point hint encoding.

[0094] The stimulus image and task text description are input into the semantic anchor cue encoder to generate semantic anchor features. This step further includes:

[0095] S3.1 Image visual feature extraction.

[0096] Stimulus image The input is fed into the pre-trained CLIP-ViT model to extract visual token sequences. In this embodiment, CLIP's ViT encoder uses a parameter-freezing method for feature extraction to avoid the pre-trained visual semantic knowledge being destroyed by task fine-tuning. The visual token sequence output by the ViT encoder has dimensions [B, N, D], where N is the number of tokens (the number of tokens generated after the ViT model divides the image into blocks, N=196 in this embodiment), and D is the original feature dimension (D=768 in this embodiment).

[0097] S3.2 Token Spatial Attention (TSA) Focus.

[0098] Considering that EEG-evoked components are primarily driven by core targets, in order to suppress background noise and achieve precise alignment between visual and neural saliency, the visual token sequence is... The input is fed into the TSA (Token Spatial Attention) module. The TSA module captures key semantic regions by dynamically calculating spatial weights.

[0099] Specifically, a spatial attention mask is first generated through two layers of linear transformation and activation functions:

[0100]

[0101] in, The input image features or visual feature maps (from the output of the CNN); The input is a linear transformation weight matrix, used to map image features to an intermediate space; This is a non-linear activation function (ReLU is used in this embodiment) used to introduce non-linear expressive power; The output is a linear transformation weight matrix used to map intermediate features to the weight space of a single channel; σ(·) is the Sigmoid activation function, which compresses the output to the [0,1] interval to obtain the attention weight at each position; The final attention mask represents the importance or attention level of each location in the image.

[0102] Then, residual enhancement is performed between the attention mask and the original features to obtain the focused visual features. :

[0103]

[0104] in, This represents the final feature representation enhanced by the attention mechanism; Θ denotes element-wise multiplication. This design adaptively enhances key information features while preserving the original feature representation, mimicking the "foveal focusing" mechanism of the human visual system in a biological sense.

[0105] By combining the attention weight generation process with the residual enhancement mechanism, the above process can be equivalently represented as:

[0106]

[0107] S3.3 Text Semantic Injection.

[0108] Input the task-related text description into CLIP's text encoder (with parameters frozen) to obtain a global semantic vector. The semantic vector has dimensions [B, D]. This semantic vector is then broadcast together with the focused visual features. Element-wise addition is performed to inject textual semantics into visual features, ultimately yielding the semantic anchor feature F. semantic The TSA module takes [B, N, D] dimensions as input and outputs a focused visual token feature of dimension [B, N, D]. The task text is encoded into a semantic vector of dimension [B, D] by the CLIP text encoder. This semantic vector is then added element-wise to the visual token feature output by the TSA through a broadcast mechanism, thereby injecting text semantics into visual features.

[0109] Step S4: Cross-modal bidirectional conditional matching.

[0110] EEG characteristics and semantic anchor features The input is fed into the cross-modal bidirectional conditional matching module of layer L for iterative alignment. In this embodiment, D'=256. After mapping through a 1×1 convolutional layer and a fully connected layer, the EEG feature dimension is [B,T,D'], the semantic feature dimension is [B,N,D'], and the dimension after feature concatenation is [B,T+N,D']. The number of layers L in this module is a hyperparameter, which is adaptively adjusted according to the dataset size of the RSVP task. In this embodiment, the value is 4. The structure of each layer is completely consistent, and the features are backpropagated between layers through residual connections to avoid the gradient vanishing problem of deep networks.

[0111] First, a 1×1 convolutional layer and a fully connected layer are used to map both features to a unified feature dimension D' (D'=256 in this embodiment). After mapping, the EEG features become [B,T,D'], and the semantic features become [B,N,D'], ensuring dimensional consistency in cross-modal feature interactions.

[0112] Within each layer, two paths are executed in parallel, enabling dynamic recombination and co-evolution of cross-modal features through content relevance:

[0113] S4.1 Visual semantic-driven neural representation refinement path.

[0114] In this approach, EEG representations are used as the query subject to extract discriminative context from visual semantic candidates. Specifically, content indicator vectors are extracted from the temporal structure of EEG features, and their association weights with visual feature segments are calculated. Through this semantic guidance, the originally noisy EEG activations are refined under the constraints of visual priors.

[0115]

[0116] in, The refined EEG features of the lth layer are the output results after passing through the cross-modal attention mechanism; This is a query vector derived from EEG signals, used to retrieve relevant information in other modalities; The key vector (Key) from the image modality is used to calculate the relevance with the query vector; Value is the value vector from the image modality, which is the feature that is finally weighted and fused; φ(·) represents the content-based dynamic aggregation operation (in this embodiment, it is scaled dot product attention, and the specific formula is: Where Q, K, and V are the query, key, and value vectors of the attention mechanism, respectively; This is the transpose of the key vector K; The dimension of the key vector K is used to scale the attention score, avoiding excessively large values ​​due to high dimensionality. `softmax(•)` is a normalization function used to map the attention weights to the [0,1] interval (ensuring the weight sum is 1). Its function is to selectively enhance ambiguous neural activations using visual semantics, accurately mapping them to the neural function subspace triggered by the current visual stimulus. This path uses EEG signals as the "query," performing attention retrieval within image features to obtain enhanced EEG features that integrate image information.

[0117] S4.2 Neural response-driven visual saliency focusing pathway.

[0118] Complementing the previous approach, this approach identifies and reinforces regions in the visual feature space that can effectively elicit neurophysiological responses. Here, visual features act as query cues, while EEG features serve as modulating context, providing inverse attentional guidance for visual representation.

[0119]

[0120] in, φ(·) represents the focused visual features of the l-th layer, which are refined visual features after EEG guidance and are used for subsequent target detection or classification; φ(·) is a content-based dynamic aggregation operation used to realize the dynamic weighting of visual features by EEG. The query vector for the image modality is generated from image features and serves as the subject of the attention query. The key vector is generated from EEG neurodynamic features and is used to calculate the relevance with the query vector. The value vector represents the EEG modality, which is an EEG feature weighted and fused by attention weights. This path uses visual features as the query, performs content-based attention retrieval and dynamic aggregation on the EEG features, and injects the neural response priors of EEG into the visual features, thereby achieving EEG-driven visual feature refinement.

[0121] This mechanism transforms the spatiotemporal frequency characteristics of EEG into semantic modulation factors, forcing the model to focus on the most sensitive visual salient regions to brain responses in high-dimensional space, thereby effectively suppressing task-irrelevant visual redundancy.

[0122] S4.3 Iterative Modal Collaboration and Alignment.

[0123] Two paths are executed simultaneously at each level of the module: the refined EEG features output by the visual semantic-driven EEG refinement path. Focused visual features output by the visual focusing path driven by neural response After each layer is completed, features are concatenated and then fed into the next layer for iterative matching. The concatenated features of each layer have a dimension of [B, T+N, D'], which are normalized and then used as input for the next layer, achieving progressive alignment of cross-modal features.

[0124] The aforementioned bidirectional paths are coupled in parallel at each level, progressively eliminating the semantic offset between EEG and visual modality through multi-scale iterative matching. The resulting fused representation preserves the topological structure of visual semantics while coupling the individual response characteristics of neural signals, achieving deep alignment between neural representation and visual semantics in manifold space.

[0125] After L layers of iteration, the module outputs the final fusion features, which include refined EEG features [B,T,D'] and focused visual features [B,N,D'].

[0126] Step S5: Gated timing reconstruction fusion and classification.

[0127] The two features output from step S4 are concatenated in the time domain to obtain the cross-modal fusion feature. The dimension is [B,T+N,D']. This feature is the input to the gated temporal reconstruction fusion module.

[0128] While cross-modal interactions enhance the discriminative power of features, the nonlinear transformations during deep collaboration often disrupt the fine temporal dependencies in EEG signals, leading to a degradation of the inherent temporal continuity of neural signals. To balance the contradiction between "cross-modal semantic gain" and "preservation of endogenous temporal structure," this embodiment proposes a reconstruction mechanism using a gated temporal reconstruction fusion module (Gated Residual Mamba, GRM).

[0129] First, After layer normalization, the data is fed into an additional, lightweight, unidirectional Mamba module as a temporal manifold repairer to obtain the temporal reconstruction feature x_recon:

[0130]

[0131] in, The input cross-modal fusion feature is the EEG-visual fusion feature obtained after bidirectional cross-modal attention, with dimensions [B,T+N,D']. LayerNorm(·) is a layer normalization operation used to stabilize training, alleviate gradient vanishing, and standardize the fusion feature to prepare for subsequent Mamba processing. The Mamba module for post-processing is a selective state-space model specifically designed for temporal dependency modeling and noise suppression of sequence features. It can effectively capture the long-range temporal dynamics of EEG signals. The reconstructed features obtained after Mamba post-processing are a temporal refinement of the fused features, used to supplement and correct the original fused features.

[0132] Then, adaptive weights are calculated using a learnable gating parameter γ, and these weights are used to adjust the original features. and reconstructed features Perform residual fusion to obtain the final features. :

[0133]

[0134] in, These are learnable gating parameters used to dynamically control the contribution of reconstructed features to the final output, enabling adaptive residual fusion. For the Sigmoid activation function, Compressed to the [0,1] interval, it serves as an adaptive weight, determining the proportion of reconstructed features in the final fusion; The final output of the module is the enhanced feature after temporal modeling and residual reconstruction, which is used for subsequent sequence target detection or classification.

[0135] This mechanism constructs an adaptive information compensation path: gating value Based on the statistical characteristics of the current representation, this design achieves an optimal trade-off between cross-modal enhanced semantic features and Mamba-processed temporal features. In low signal-to-noise ratio environments, this design significantly improves the model's robustness to neural signal fluctuations by strengthening temporal causal constraints, ensuring the consistency of fused features in both the semantic and temporal domains.

[0136] The specific execution process of GRM is as follows:

[0137] 1. Integrate cross-modal features A lightweight unidirectional Mamba block is fed in, the hidden layer dimension of which is consistent with the input feature dimension D'. A causal scanning mechanism is used to repair the temporal features, and the output is a temporal repair feature with dimension [B, T+N, D']. In this embodiment, the parameter size of the lightweight unidirectional Mamba block is 1 / 4 that of the bidirectional Mamba encoder, which reduces the computational cost of the model while ensuring the timing repair effect.

[0138] 2. Calculate the gating parameters Product with temporal repair features and fuse it with the original cross-modal features. By adding elements one by one, the final gated fusion feature is obtained. .

[0139] 3. The vector of dimension [B,D'] is obtained by global average pooling, and then mapped to the classification dimension (target / non-target is 2-dimensional) through a fully connected layer. Finally, it is fed into the Softmax classifier to complete the target detection of the RSVP task.

[0140] Example 2: A Fast Target Detection System Based on EEG and Image-Semantic Prior

[0141] This embodiment provides a system for implementing the above method, such as... Figure 2 As shown, the system includes:

[0142] Data acquisition module: Includes a multi-channel EEG acquisition cap and an image display, used to synchronously acquire the user's EEG signals during RSVP tasks and simultaneously present stimulus images. The multi-channel EEG acquisition cap is a 32-channel cap conforming to the international 10-20 standard, with a sampling accuracy ≥16 bits and a common-mode rejection ratio ≥110dB; the image display is a high-definition LCD with a refresh rate ≥144Hz, and a stimulus image synchronization triggering error ≤1ms.

[0143] Data preprocessing module: Connected to the data acquisition module, it performs preprocessing on the raw EEG signal, such as filtering and slicing embedding, and unifies the format of images and text. This is a software module, stored on a computer-readable storage medium (solid-state drive, read / write speed ≥500MB / s), and is executed by the processor (clock frequency ≥3.0GHz), supporting batch processing (≤1024 samples per batch).

[0144] Physiologically Sensing Mamba EEG Feature Extractor: Connected to the preprocessing module, it incorporates a statistical channel attention module, a transient motion excitation module, and a bidirectional Mamba encoder to extract robust neurodynamic features from EEG signals. Its specific implementation details are as described in step S2 of Example 1.

[0145] Semantic anchor cue encoder: Connected to the preprocessing module, it incorporates a CLIP-ViT visual encoder with built-in frozen parameters, a CLIP text encoder, and a token space attention module to generate semantic anchor features that match the neural signals. Its specific implementation details are as described in step S3 of Example 1.

[0146] The cross-modal bidirectional conditional matching module is connected to both the MambaEEG feature extractor and the semantic anchor cue encoder. It contains an L-layer parallel bidirectional cross-attention structure to achieve iterative alignment and co-evolution of EEG features and visual semantic features. Its specific implementation details are as described in step S4 of Example 1.

[0147] Gated temporal reconstruction and fusion module: Connected to the cross-modal bidirectional conditional matching module, it incorporates a lightweight unidirectional Mamba module and a learnable gating module to repair temporal continuity and output gated fusion features. Its specific implementation details are as described in step S5 of Example 1.

[0148] The physiological perception MambaEEG feature extractor, semantic anchor cue encoder, cross-modal bidirectional conditional matching module, and gated temporal reconstruction fusion module can be software modules (based on the PyTorch / TensorFlow framework, supporting CUDA 11.6 and above) or hardware modules (dedicated ASIC / FPGA chips, integrating corresponding algorithm operation logic, with an operation rate ≥10^9 times / second).

[0149] Classifier: Connected to the gated temporal reconstruction fusion module, it contains fully connected layers and a Softmax activation function to output the final object detection result. This is a software module; the weight matrix of the fully connected layer is stored in GPU memory (≥16GB), the Softmax activation is performed with 32-bit floating-point precision, and the detection result output latency is ≤1ms.

[0150] Central Control Unit: Connected to all the above modules, it coordinates the data flow and computation tasks of each module. It is a microprocessor (ARM Cortex-A72 or higher), with a main frequency of ≥2.0GHz, a built-in real-time operating system (RTOS), supports multi-module task scheduling, and has a scheduling response time of ≤0.1ms.

[0151] The modules are connected via a high-speed serial bus (SPI / I2C) with a data transmission rate of ≥100Mbps. The transmitted data is a feature matrix with floating-point precision of 32. The data acquisition module and the preprocessing module transmit synchronously, while the other modules transmit asynchronously. Data interaction is triggered by the central control unit. Lossless compression is used during feature data transmission with a compression ratio of ≥2:1 and no data loss.

[0152] The specific implementation details of each of the above modules correspond one-to-one with the steps described in Example 1, and will not be repeated here.

[0153] Model training and deployment:

[0154] The end-to-end model described above was trained using the labeled RSVP dataset. The optimizer was Adam, the initial learning rate was set to 1e-4, and the loss function was cross-entropy loss. After training, for new RSVP tasks without calibration data, only the EEG signal, stimulus image, and task description needed to directly perform object detection.

[0155] Training details: The total number of training epochs is 20, the batch size B=256, the weight decay coefficient is 0.001, the gradient clipping threshold is 1.0, and an early stopping strategy (patience=10) is adopted; the training set: validation set: test set = 9:0.5:0.5, the dataset contains 1,065,000 labeled trial samples, covering testers of different genders aged 20-60, and the stimulus images include four categories: people, cars, airplanes, and background images that are not the first three categories.

[0156] Deployment environment: The model can be deployed on CPU (i7-12700K and above), GPU (RTX3090 and above) or edge computing devices (computing power ≥10TOPS, memory ≥4GB); the single sample processing time on the GPU side is ≤38ms and on the edge device side is ≤50ms, which meets the real-time requirements of RSVP tasks.

[0157] Comparison of effects:

[0158] The experimental data comparing the method of this invention with other methods are shown in Table 1:

[0159]

[0160] Table 1

[0161] From Table 1: Accuracy and standard deviation data of cross-task tests without calibration, the objective performance of the SA-MambaEEG model of this invention can be derived as follows:

[0162] Core advantage in accuracy: The model of this invention has the highest accuracy among all compared models in the three cross-task tests of car→plane, people→plane, and plane→car; in the car→people task, it is on par with the TCN-T model in accuracy (79.30%), which is one of the best results for this task.

[0163] Significantly superior stability: In all 6 cross-task tests, the standard deviation of the model of this invention was the smallest among all the comparative models, ranging from 0.37 to 0.86, which is much lower than other models (lowest 1.39, highest 8.93). This indicates that the model has small fluctuations in detection results in cross-task scenarios without calibration, and has outstanding robustness and stability.

[0164] Compared with traditional baseline models such as HDCA, MDRM, EEGNet, and LeeNet, the model of this invention achieves significant improvements in accuracy across all 6 cross-tasks and a substantial reduction in standard deviation, solving the problems of poor feature transferability and insufficient stability of detection results in traditional models.

[0165] Comparison with recent optimized models: Compared with recent optimized models such as TCN-T, TFF-Former, and EEGbaseline, the model of this invention maintains an accuracy advantage in most cross-task scenarios, and the standard deviation is reduced by an order of magnitude. Even in complex cross-task scenarios, it can still maintain stable detection performance, verifying the model's calibration-free cross-task generalization ability.

[0166] Average accuracy: The average cross-task accuracy of the model of this invention is 82.67% for 6 groups, which is 82.7% after rounding, which is the highest average accuracy among all the comparison models.

[0167] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A fast target detection method based on EEG and image-semantic priors, characterized in that, Includes the following steps: Step S1: Acquire the EEG signals, stimulus images, and task-related text descriptions of the user in the Rapid Sequence Visual Presentation (RSVP) task, and preprocess the EEG signals, stimulus images, and text descriptions; S2: Input the preprocessed EEG signals into the physiological perception MambaEEG feature extractor to extract EEG neurodynamic features; S3: Encode the preprocessed stimulus images and text descriptions to generate semantic anchor features; Step S4: Input the EEG neurodynamic features and semantic anchor features into the cross-modal bidirectional conditional matching module, perform iterative alignment and co-evolution to obtain fused features; Step S5: Perform temporal reconstruction fusion and classification on the fused features, and output the target detection results.

2. The method according to claim 1, characterized in that, The extraction of EEG neurodynamic features in step S2 includes: Step S2.1: Channel reweighting of EEG signals is performed through a statistical channel attention module. The statistical channel attention module simultaneously calculates the global average pooling and global standard deviation pooling of the input features in the time domain, concatenates the two in the channel dimension to obtain a statistical descriptor, generates channel attention weights through two fully connected layers and activation functions, and performs residual connection with the original features to obtain channel-weighted features; Step S2.2: Channel-weighted features are subjected to channel dimensionality reduction convolution and time-domain convolution through a transient motion excitation module, and their first-order temporal difference in the time domain is calculated. The difference features are subjected to global average pooling, and then generated into dynamic excitation weights through a multilayer perceptron and a sigmoid function. The dynamic excitation weights are performed into the channel-weighted features through residual connection to obtain transient enhancement features; Step S2.3: Transient enhancement features are scanned forward and backward through a bidirectional Mamba encoder, and bidirectional contextual features are fused by element-wise addition. After layer normalization, EEG neurodynamic features are output.

3. The method according to claim 2, characterized in that, The formula for calculating the channel attention weights generated by the statistical channel attention module in step S2.1 is as follows: , Where Concat(·) is the feature concatenation operation. For the input EEG features, AvgPool(·) and StdPool(·) are global average pooling and global standard deviation pooling, respectively. Let be the concatenated statistical descriptor, R be the set of real numbers, C be the dimension of the space (the concatenated dimension is C + C = 2C), W1 and W2 be the weight matrices of the fully connected layer, δ(·) be the ReLU activation function, and σ(·) be the Sigmoid activation function. Weighted features for the output channels.

4. The method according to claim 2, characterized in that, The calculation formula for the dynamic excitation weights generated by the transient motion excitation module in step S2.2 is as follows: , ,in, For time-series difference features, It is a temporal convolution operation. It is channel-reduced convolution. Weighted features for the input channels. For global average pooling, For a multilayer perceptron, σ(·) is the sigmoid activation function. Represents the difference feature at the t-th time step. For the difference feature at the (t-1)th time step, This is for the transient enhancement features of the output.

5. The method according to claim 2, characterized in that, The bidirectional Mamba encoder adopts a selective state-space model, specifically implemented as follows: a. Discretize the continuous-time state-space model using zero-order preservation. The discretization parameter transformation formula is as follows: ， ， in Let A be the discretized state transition matrix, exp(·) be the matrix exponentiation operation, Δ be the discretization time step, A be the state transition matrix, and P be the input projection matrix. To discretize the input projection matrix; I is the identity matrix; b. Parameterize the discretized time step Δ, the input projection matrix P, and the projection matrix Ω into the current time step input feature vector. The function, through For the input vector Perform linear encoding operations to generate a linear code structure defined by Δ, P, and Ω, and realize dynamic weight allocation; c. In the selective scanning mechanism of the selective state space model, the dynamic adjustment of step size and weight is triggered by the amplitude and rate of change of the input EEG features. When the feature amplitude exceeds the preset threshold of the model, the model increases the attention weight and reduces the discretization time step Δ to achieve focus on significant neural responses. Where Linear(x) t ) represents the input vector x t Perform linear encoding operations to obtain the corresponding codewords or encoding results.

6. The method according to claim 1, characterized in that, The generation of semantic anchor features in step S3 includes: Step S3.1: Extracting the visual token sequence of the stimulus image using a pre-trained CLIP-ViT visual encoder, wherein the CLIP-ViT encoder uses a parameter-freezing method; Step S3.2: Spatially focusing the visual tokens through a token spatial attention module to generate a spatial attention mask, the calculation formula being: ,in A visual token sequence, , The weight matrix is ​​a linear transformation, δ is the ReLU activation function, and σ is the Sigmoid activation function; then, the attention mask and the original features are subjected to residual enhancement to obtain the focused visual features: Step S3.3: Input the preprocessed task-related text description into the CLIP text encoder to obtain a global semantic vector. The CLIP text encoder uses a parameter freezing method. The global semantic vector is then fused with the focused visual features element by element through a broadcast mechanism to generate semantic anchor features.

7. The method according to claim 1, characterized in that, The cross-modal bidirectional conditional matching module in step S4 first maps EEG neurodynamic features and semantic anchor features to a unified feature dimension D' through 1×1 convolutional layers and fully connected layers, and then sets up a bidirectional cross-attention structure with L layers of parallel execution, where D' and L are hyperparameters; the following sub-steps are executed in parallel in each layer: Step S4.1: Visual semantic-driven neural representation refinement path, using EEG features as queries and visual features as keys and values, and refining EEG features through scaled dot product attention: ,in For the refined EEG characteristics of layer l, This is a query vector derived from electroencephalogram (EEG) signals. This is the key vector (Key) from the image modality. Let φ(·) be the value vector from the image modality; Step S4.2: The neural response-driven visual saliency focusing path uses visual features as queries and EEG features as keys and values, and calculates the focusing visual features through scaled dot product attention. ,in, Let be the focused visual feature of the l-th layer, and φ(·) be the content-based dynamic aggregation operation. For the query vector of the image modality, The key vectors for EEG modalities. This is the value vector of the EEG modality; Step S4.3: The refined EEG features and focused visual features output from each layer are spliced ​​together, and after layer normalization, they are used as the input for the next layer. The cross-modal bidirectional conditional matching module achieves iterative synergy between visual semantic-driven EEG feature refinement and neural response-driven visual feature focusing through the aforementioned L-layer parallel execution bidirectional cross-attention structure. This progressively eliminates intermodal semantic shifts, enabling efficient fusion and accurate matching of EEG and image-semantic features.

8. The method according to claim 1, characterized in that, The temporal reconstruction and classification described in step S5 includes: fusing features The input-gated temporal reconstruction fusion module includes a lightweight unidirectional Mamba module and a learnable gating module. The hidden layer dimension of the lightweight unidirectional Mamba module is consistent with the dimension D' of the input fusion feature, and it performs temporal modeling on the fusion feature to generate temporal reconstruction features. Its parameter size is 1 / 4 that of the bidirectional Mamba encoder. The learnable gating module normalizes the gating parameter γ using the Sigmoid function to generate adaptive weights. It then performs element-wise multiplication of the original fused features and the reconstructed features, followed by residual fusion to obtain the final features. : , in, These are learnable gating parameters used to dynamically control the contribution of reconstructed features to the final output, enabling adaptive residual fusion. The Sigmoid activation function is used; the final features are then input into a fully connected layer after global average pooling, and finally output as object detection results through a Softmax classifier.

9. The method according to claim 7, characterized in that, The L-layer bidirectional cross-attention structure of the cross-modal bidirectional conditional matching module has residual connections between its layers to achieve effective gradient backpropagation. Scaling dot product attention is used as a content-based dynamic aggregation operation φ(·) to achieve bidirectional collaboration between EEG refinement based on visual semantic constraints and visual focusing guided by neural response. After each layer of refined EEG features and focused visual features are concatenated and normalized, they are input to the next layer. This iterative collaborative approach progressively eliminates semantic offsets between modalities. Furthermore, cross-modal features are mapped to a unified dimension through 1×1 convolutional layers and fully connected layers, achieving efficient fusion and accurate matching of EEG and image-semantic features.

10. A rapid target detection system for implementing the EEG and image-semantic prior of any one of claims 1-9, characterized in that, include: Data acquisition module: including a multi-channel EEG acquisition cap and image display, used to synchronously acquire the user's EEG signals while performing the RSVP task and synchronously present stimulation images; Data preprocessing module: used to perform preprocessing on the raw EEG signal, such as filtering and slicing embedding, and to unify the format of images and text; Physiologically Sensing Mamba EEG Feature Extractor: Built-in statistical channel attention module, transient motion excitation module and bidirectional Mamba encoder for extracting robust neurodynamic features from EEG signals; Semantic anchor cue encoder: CLIP-ViT visual encoder with built-in frozen parameters, CLIP text encoder and token space attention module, used to generate semantic anchor features that match neural signals; Cross-modal bidirectional conditional matching module: contains an L-layer parallel bidirectional cross-attention structure, used to achieve iterative alignment and co-evolution of EEG features and visual semantic features; Gated timing reconstruction and fusion module: It has a built-in lightweight one-way Mamba module and a learnable gating module, which are used to repair timing continuity and output gated fusion features; Classifier: Contains fully connected layers and a softmax activation function, used to output the final object detection result; Central control unit: Used to coordinate the data flow and computing tasks of various modules.

Images