Electroencephalogram image reconstruction method based on frequency steering and bidirectional diffusion

Abstract

The invention provides an electroencephalogram image reconstruction method based on frequency steering and bidirectional diffusion, and belongs to the technical field of brain-computer interfaces and computer vision. The method comprises the steps that EEG data and corresponding image data are acquired and preprocessed; constructing a reconstruction model comprising a frequency domain-space-time dynamic encoder and a bidirectional submerged space diffusion generator; wherein the frequency domain-space-time dynamics encoder adopts a frequency-oriented Mama architecture, explicitly models neural oscillation dynamics by constructing a block diagonal state matrix, and extracts features in combination with graph convolution and space-time convolution; the bidirectional submerged space diffusion generator comprises a symmetric EEG-to-image submerged space diffusion model and an image-to-EEG submerged space diffusion model, and training is carried out through generative cyclic consistency constraint; and finally, mapping the collected EEG data into image semantic features by using the trained model, and driving a pre-training generation model to reconstruct an image. According to the method, the problems that in the prior art, the electroencephalogram frequency domain specificity dynamic state is ignored, and cross-modal semantic alignment is weak are solved, and the semantic consistency of electroencephalogram decoding and the fidelity of image reconstruction are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of brain-computer interface (BCI), neuroscience and deep learning, and specifically to a method for reconstructing electroencephalogram (EEG) images based on frequency guidance and bidirectional diffusion. Background Technology

[0002] Reconstructing visual semantics perceived or imagined by humans from non-invasive electroencephalography (EEG) signals is an important research direction in neuroscience, artificial intelligence, and brain-computer interfaces. The core challenge lies in establishing a robust mapping between EEG signals and latent space features, which is crucial for revealing the mechanisms of human visual cognition and constructing high-level brain-computer interaction systems.

[0003] Latent space features are low-dimensional vector representations containing core semantic information formed after high-dimensional data such as images are compressed by deep neural networks (such as variational autoencoders VAEs or CLIP models). In image generation tasks, latent space features act as "semantic seeds," determining the category, structure, and key attributes of the final generated image.

[0004] However, due to the inherent characteristics of EEG signals, such as low signal-to-noise ratio, limited spatial resolution, and significant differences across subjects, existing methods still face many challenges in the extraction and alignment of latent spatial features.

[0005] On the one hand, the mapping accuracy between EEG features and visual latent space is insufficient. Most existing EEG coding models struggle to explicitly characterize frequency-specific neurodynamic features highly correlated with visual perception, especially neural oscillation patterns such as Beta waves (13–30 Hz) and Gamma waves (30–60 Hz). This results in feature vectors decoded from EEG not being accurately mapped to the visual latent space coordinates of pre-trained generative models (such as Stable Diffusion), limiting the model's discriminative ability.

[0006] On the other hand, the modal gap hinders robust alignment of latent space features. EEG signals and image latent spaces differ significantly in data distribution and semantic hierarchy. Existing methods largely rely on contrastive learning-based cross-modal alignment strategies, constraining the similarity of different modal features in the latent space solely from a discriminative perspective. This approach lacks generative consistency constraints, failing to effectively constrain the ability of EEG features to reconstruct the latent features of the original image in the latent space. This easily leads to unstable cross-modal alignment, and the resulting images still suffer from deficiencies in semantic consistency and visual fidelity.

[0007] Therefore, it is necessary to provide an EEG-to-image reconstruction method that can effectively model EEG frequency characteristics and achieve robust cross-modal semantic alignment through latent space generative constraints, so as to improve the semantic accuracy and perceptual quality of the reconstruction results. Summary of the Invention

[0008] To address the shortcomings of existing EEG-to-image decoding and reconstruction methods in practical applications, such as insufficient frequency domain neurodynamic modeling capabilities, weak cross-modal semantic alignment, and low semantic consistency and visual fidelity of generated images, this invention proposes an EEG image reconstruction method based on frequency guidance and bidirectional diffusion. This invention aims to construct an EEG image decoding framework that combines discriminative semantic alignment and generative reconstruction capabilities by explicitly modeling the frequency domain dynamic features of EEG signals closely related to visual cognition and introducing bidirectional diffusion generation and cyclic consistency constraints at the latent space level. This enables high-semantic-consistency image retrieval and high-fidelity image reconstruction based on non-invasive EEG signals.

[0009] To address the aforementioned problems in the prior art, this invention provides a frequency-guided and bidirectional diffusion-based EEG image reconstruction method. The inventors used the internationally publicly available EEG-image pairing dataset THINGS-EEG to verify the effectiveness of the method. After preprocessing, the dataset is divided into training and testing sets. The training set is used as input during model training, and the testing set is used as input during model inference. The final output is a reconstructed visual image, and quantitative and qualitative evaluations of image semantic consistency and perceptual quality are performed.

[0010] A frequency-guided and bidirectional diffusion-based EEG image reconstruction method according to an embodiment of the present invention includes the following steps:

[0011] Step 1: Obtain the internationally publicly available EEG-image pairing dataset THINGS-EEG dataset, preprocess the EEG data in the dataset, and divide it into training set data and test set data. The training set data is used for model training, and the test set data is used for model evaluation.

[0012] Step 2: Construct a bidirectional diffusion-guided frequency semantic alignment model structure using the PyTorch deep learning framework;

[0013] Step 3: Input the training set data established in Step 1 into the model, perform end-to-end training on the model by jointly optimizing the cross-modal contrastive learning loss and the bidirectional diffusion generation loss, and save the best training model.

[0014] Step S4: Input the EEG data from the preprocessed test set data in S1 into the model saved in S3 to obtain the corresponding EEG latent space features. Calculate the cosine similarity with the image latent space features to retrieve the corresponding image. Reconstruct the corresponding image using the EEG-to-image latent space diffusion generator and the pre-trained SDXL model.

[0015] In step 1, the EEG data undergoes systematic preprocessing to ensure the stability and effectiveness of subsequent feature extraction and model training. Specifically, firstly, based on the electrode arrangement of the adopted EEG acquisition system, the raw EEG signals are channel-selected, removing channels with low relevance to visual tasks or poor signal quality. Then, bandpass filtering is applied to the retained EEG signals, with the filtering frequency band set to 1–100 Hz to retain neural oscillation components closely related to human visual perception and cognitive activities, while suppressing DC drift and high-frequency noise interference. After filtering, the EEG signals are downsampled to reduce data dimensionality and computational complexity without losing key information. Furthermore, non-brain-derived artifact signals such as electrooculography (EOG) and electromyography (EMG) are removed using methods such as independent component analysis or threshold discrimination. After the above processing is completed, the continuous EEG signal is segmented using a sliding window strategy, dividing the time series signal into signal segments of fixed length. Preferably, the window length is set to 250 ms, and the windows do not overlap or slide according to a preset step size. Each EEG signal segment is associated with a visual stimulus image presented in the experiment, thereby forming EEG-image pairing samples for cross-modal learning.

[0016] In step 2, a bidirectional diffusion-guided frequency semantic alignment model structure is constructed based on the PyTorch deep learning framework. This model consists of three parts: a frequency-spatiotemporal dynamic encoder, a bidirectional latent space diffusion generator, and a pre-trained image generation model, which are sequentially connected. The frequency-spatiotemporal dynamic encoder receives the preprocessed EEG signal segments from step 1 and uses a frequency-aware mechanism to jointly model the temporal, spatial, and frequency domain information in the EEG signal, extracting discriminative and robust latent space feature representations. The bidirectional latent space diffusion generator uses the aforementioned EEG latent space features and image latent space features as conditional inputs to establish a bidirectional generative mapping relationship between the EEG latent space and the image latent space, achieving cross-modal semantic consistency constraints. The pre-trained image generation model receives the image latent space features output by the bidirectional latent space diffusion generator and decodes them into high-resolution pixel-level visual images. Through this structural design, the semantic information of the EEG signal can be fully aligned with the visual representation in the latent space and ultimately transformed into a perceptible image output.

[0017] In step 3, the model constructed in step 2 is jointly trained end-to-end using the training set data constructed in step 1. During training, a multi-objective loss function is introduced to optimize the model. On one hand, the InfoNCE contrastive loss function is used to constrain the distance between the EEG latent space features output by the frequency-spatiotemporal dynamic encoder and the image latent space features output by the pre-trained CLIP image encoder in the shared semantic space, making semantically consistent EEG-image pairs closer in the embedding space, while maintaining sufficient discriminative power for unmatched sample pairs. On the other hand, the denoising diffusion error is calculated in the bidirectional latent space diffusion generator for both the EEG-to-image direction and the image-to-EEG direction, and the L2 loss function is used to measure the difference between predicted noise and real noise, thereby guiding the model to learn a stable cross-modal generation mapping relationship. The above loss terms are jointly optimized with weights, and the model parameters are iteratively updated using the backpropagation algorithm and stochastic gradient descent strategy.

[0018] Furthermore, in step 3, the bidirectional latent space diffusion generator is specifically composed of two structurally symmetrical U-Net-type diffusion modules. The EEG-to-image latent space diffusion generator uses EEG latent space features as conditional input and reconstructs the corresponding image latent space features through a stepwise reverse diffusion process. The image-to-EEG latent space diffusion generator uses image latent space features as conditional input and reconstructs the corresponding EEG latent space features through a stepwise reverse diffusion process. The two diffusion modules together constitute a generative cyclic consistency constraint, which ensures that the EEG latent space features and image latent space features maintain consistency at the generation level, thereby significantly enhancing the stability and generalization ability of cross-modal semantic alignment.

[0019] In step 4, the EEG data from the preprocessed test set data in S1 is input into the model stored in S3 to obtain the corresponding EEG latent space features. By calculating the cosine similarity with the image latent space features, the corresponding image is retrieved, and the image shallow space features are generated through the EEG to image latent space diffusion generator. Finally, the image latent space features are introduced into the pre-trained SDXL image generation model through the IP-Adapter conditional injection mechanism to decode and obtain the final pixel-level reconstructed visual image.

[0020] Meanwhile, in step 4, to comprehensively evaluate model performance, the preprocessed test set data is input into the model to obtain the corresponding reconstructed image sequence, and the reconstruction effect is evaluated from multiple dimensions. Specifically, the model's cross-modal semantic alignment capability under zero-shot conditions is evaluated by Top-k image retrieval accuracy; Fréchet InceptionDistance (FID) is used to measure the difference between the reconstructed image and the original stimulus image in the high-dimensional feature distribution to evaluate the perceptual quality of the generated image; CLIP Score is used to measure the similarity between the reconstructed image and the original stimulus image in the semantic embedding space to reflect the level of semantic consistency. Through the above multi-index joint evaluation, the comprehensive performance advantages of the method of the present invention in both image retrieval and image reconstruction tasks are fully verified.

[0021] The main advantages of the frequency-guided and bidirectional diffusion-based EEG image reconstruction method proposed in this invention include:

[0022] 1. The frequency-spatiotemporal dynamic encoder designed in this invention introduces a frequency-guided Mamba (frequency-guided step) architecture to explicitly model neural oscillation patterns in EEG data and adaptively enhances the Beta and Gamma frequency bands related to visual perception, effectively improving the discriminative power and robustness of EEG representation.

[0023] 2. The bidirectional latent space diffusion generator designed in this invention achieves strong cross-modal alignment far exceeding traditional contrastive learning by constructing a bidirectional generative cyclic consistency constraint between EEG and image latent space, significantly improving the semantic accuracy and visual fidelity of reconstructed images, and achieving state-of-the-art performance on multiple public datasets. Attached Figure Description

[0024] Figure 1 This is a flowchart of an electroencephalogram (EEG) image reconstruction method based on frequency guidance and bidirectional diffusion according to an embodiment of the present invention.

[0025] Figure 2 This is a general framework diagram of an EEG image reconstruction method based on frequency guidance and bidirectional diffusion.

[0026] Figure 3 This is an internal structure diagram of a frequency-temporal dynamic encoder. Part a is the core state matrix structure of the frequency steering module, part b is the selective scanning mechanism, and part c illustrates the specific flow of data after passing through the frequency steering module.

[0027] Figure 4 The image retrieval diagram according to an embodiment of the present invention demonstrates the excellent retrieval capability of a frequency-spatiotemporal dynamic encoder for images corresponding to EEG.

[0028] Figure 5The image reconstruction diagram provided is based on an embodiment of the present invention, demonstrating the excellent image reconstruction capabilities of a bidirectional latent space diffusion generator.

[0029] Figure 6 Top-1 and Top-5 accuracies (%) for zero-sample retrieval within the THINGS-EEG dataset.

[0030] Figure 7 Top-1 and Top-5 accuracy (%) for zero-sample retrieval across subjects on the THINGS-EEG dataset. Detailed Implementation

[0031] In an embodiment of the present invention, an EEG image reconstruction method based on frequency guidance and bidirectional diffusion is proposed. By designing a frequency-spatiotemporal dynamic coding module, a cross-modal semantic alignment module, and a bidirectional latent space diffusion generator module, joint modeling of frequency domain, time domain, and spatial topological features in EEG signals is achieved. A generative cyclic consistency constraint between EEG and image latent space is constructed using the bidirectional latent space diffusion generator mechanism. Model training and parameter optimization are completed through a cross-modal contrastive learning and consistency joint training strategy, ultimately realizing image semantic retrieval and high-fidelity image reconstruction tasks based on EEG signals.

[0032] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, illustrates a frequency-guided and bidirectional diffusion-based electroencephalogram (EEG) image reconstruction method proposed in this invention.

[0033] According to an embodiment of the present invention, the overall process of the frequency-guided and bidirectional diffusion-based EEG image reconstruction method is as follows: Figure 1 As shown, it includes the following steps:

[0034] Step S1: Preprocess the publicly available EEG-image pairing dataset and construct training and test sets for subsequent model training, image retrieval, and image reconstruction performance evaluation.

[0035] The EEG-image pairing dataset is the internationally publicly available THINGS-EEG dataset, which contains multi-channel EEG signals and corresponding visual stimulus images collected from 10 healthy subjects while they viewed natural scene images. Each subject viewed multiple categories of natural image stimuli during the experiment, with each image presented for approximately 1000 ms, and corresponding EEG signals were collected simultaneously. To comprehensively evaluate the model's intra-individual modeling ability and cross-individual generalization ability, this embodiment simultaneously employs single-subject evaluation strategies and cross-subject evaluation strategies to validate the method.

[0036] Step S1.1: Perform channel selection, including: selecting the THINGS-EEG public EEG-image paired dataset as the input data source, retaining 64 standard 10–20 system electrode channels (such as FP1, F3, C3, P3, O1, FP2, F4, C4, P4, O2, etc.), and removing damaged, drifting, and / or redundant channels to ensure signal quality and spatial coverage integrity;

[0037] Step S1.2: Perform bandpass filtering and artifact removal, including: applying a 1–100 Hz bandpass filter to the original continuous EEG signal to preserve the neural oscillation frequency band closely related to visual perception;

[0038] Step S1.3: Perform artifact removal, including: using an automatic artifact detection algorithm to identify and remove non-neurogenic interference such as eye movements, blinking, electromyography, and electrocardiogram;

[0039] Step S1.4: Perform downsampling processing, including downsampling the original high sampling rate signal to 250 Hz to reduce computational overhead while maintaining sufficient temporal resolution to capture key neural events;

[0040] Step S1.5: Perform data segmentation, including: segmenting continuous EEG data using a fixed time window according to the visual stimulus presentation protocol; in THINGS-EEG, each image is presented for 1000 ms, corresponding to 250 sampling points, and each window of the fixed time window generates a 63×250 EEG matrix and establishes a one-to-one mapping with its corresponding natural image;

[0041] Step S1.6: Construct the training and test sets, including:

[0042] Single subject division strategy: Construct corresponding training and test sets for each subject. Divide the brain signal data collected from the subject in multiple visual stimulation experiments according to a preset ratio. One part is used as the training set and the other part is used as the test set to evaluate the model's learning ability and image reconstruction performance of visual semantic representation under the same subject conditions.

[0043] Cross-subject splitting strategy: A strict cross-subject splitting method is adopted, and the data of N-1 subjects out of all N subjects are merged to construct the training set, and the data of the remaining 1 subject who did not participate in the training is used as the test set to verify the model's generalization ability on unseen individuals.

[0044] Under any of the above partitioning strategies, all images are embedded using a frozen CLIP-ViT / L-14 model to extract d=768-dimensional image embeddings. This serves as a semantic supervision signal; this partitioning method also supports a unified evaluation framework for both image retrieval and image reconstruction tasks.

[0045] Step S2: Construct an EEG-to-image reconstruction model structure based on frequency-guided semantic alignment and bidirectional diffusion guidance using the PyTorch deep learning framework, including:

[0046] Step S2.1: Frequency-Spatiotemporal Dynamic Coding Step, used to extract frequency-aware deep spatiotemporal features from the raw EEG data, including:

[0047] In the frequency-guided state-space structure modeling step, Mamba is used as the basic framework, which translates the state transition matrix of the state-space model SSM. The structure is a block diagonal form, with each block being a 2×2 sub-block. Defined as:

[0048] (1)

[0049] Where the damping factor To ensure system stability, adaptive angular frequency It is given by the following formula:

[0050] (2)

[0051] in, For adaptive angular frequency, For activation function, Based on the fundamental frequency parameter, This is a learnable logarithmic frequency bias term used to dynamically adjust the model's response intensity to different frequency bands during training. This enables the model to adaptively enhance its modeling ability for neural oscillation frequency bands closely related to visual semantics and cognitive tasks, including Beta and Gamma, where Beta is 13–30 Hz and Gamma is 30–60 Hz.

[0052] After discretizing the continuous-time state-space model, each sub-block Corresponding discrete form Expressed as:

[0053] (3)

[0054] Among them, damping factor , For adaptive angular frequency, The time step parameter is a learnable parameter used to adapt to EEG signals with different sampling rates and time scales. This discretization form, while maintaining the rotation-decay characteristics of the continuous system, facilitates efficient parallel computation on a GPU, and then... ,

[0055] In the frequency-guided state-space structure modeling step, a selective scan mechanism is used to update the state and model the features of the time series of multi-channel EEG signals. The hidden state update and output calculation process follows the following relationship:

[0056] (4)，

[0057] in:

[0058] t represents the discrete time step.

[0059] This represents the original or intermediate characteristics of the c-th EEG electrode channel input at time t.

[0060] This represents the hidden state vector of the c-th EEG channel at time t, used to characterize the neurodynamic features of that channel in the frequency and time dimensions.

[0061] This represents the output feature of the corresponding state-space model.

[0062] , , , These are the learnable state transition matrix, input mapping matrix, and output mapping matrix, respectively. Their parameters are adaptively updated through backpropagation during model training.

[0063] In one specific embodiment, for each EEG channel, its hidden state sequence is determined within a preset time window (corresponding to the duration of a single visual stimulus presentation). Perform time-dimensional aggregation operations, including but not limited to final time-state selection, time-averaged pooling, or weighted aggregation operations, to obtain a fixed-dimensional feature vector for that channel. ,

[0064] The channel-level feature vectors corresponding to all EEG channels are concatenated according to the channel dimension to form the final output feature matrix of the frequency-guided state-space structure modeling step: ,

[0065] Among them, among them, This represents the feature vector obtained by aggregating the c-th EEG channel within a preset time window; symbol The matrix transpose operation is represented by R, where R represents the set of real numbers, N represents the number of EEG electrode channels, and d represents the potential feature dimension for each channel. The feature matrix... While maintaining linear time complexity, it can stably characterize the frequency-time dynamics information in long-term EEG sequences.

[0066] The neural graph extraction structure step is used to model the spatial topological relationships of EEG electrodes, and the frequency-directed state-space structure modeling step outputs the features. Considered as graph node attributes, in the adjacency matrix defined by the international 10-20 standard electrode placement system. Perform graph convolution operation above:

[0067] (5)

[0068] in, This is the degree matrix corresponding to the adjacency matrix. It is an adjacency matrix. The input features are the frequency-guided state-space structure steps, and the final output features are the input features. The spatial topological features are the output of the neural graph structure extraction step. For the first Layer graph convolution weight parameters, for Activation functions, through the structural extraction steps of neural maps, can effectively capture co-activation patterns across brain regions, enhancing the spatial discriminative ability of EEG representations.

[0069] The spatiotemporal feature extraction structure step, after fusing the frequency dynamic features output from the frequency-guided state-space structure step and the spatial topological features output from the neural graph structure extraction step, further refines the local spatiotemporal features through the spatiotemporal feature extraction step. This module borrows from the EEGNetV4 network structure, employing a depthwise separable convolution stack to reduce parameter size and improve feature representation efficiency. It also fuses temporal and spatial convolution operations. Finally, it uses a projection multilayer perceptron (MLP) to... The fused features are mapped to a fixed-dimensional latent space, and the output is... Dimensional EEG latent space characteristics : ,in Represents the real number field. Represents the dimension of the latent space, and the features of that latent space. This is used as a unified conditional representation for subsequent cross-modal alignment and generation modules.

[0070] Step S22): Perform bidirectional latent space diffusion generation to establish bidirectional generative cyclic consistency constraints between the EEG and the image latent space, including:

[0071] Perform an EEG-to-image latent space diffusion generation step, in order to As a condition, through Time-step back-diffusion process reconstructs latent space features of an image from Gaussian noise. ;

[0072] Perform an image-to-EEG latent space diffusion generation step, in order to As a condition, the latent space features of EEG are reconstructed from Gaussian noise through a T-time step back diffusion process. ;

[0073] Both diffusion steps described above employ an MLP-based U-Net network structure containing five basic modules, along with a noise prediction network during the diffusion process. The inputs include: noisy latent vectors, conditional embeddings, and time steps. In this process, conditional embedding is injected into the intermediate layers of the network through a Feature Linear Modulation (FiLM) layer to dynamically adjust the network activation distribution, thereby achieving cross-modal conditional control.

[0074] The frequency-guided and bidirectional diffusion-based EEG image reconstruction method also includes a pre-trained image generation model, used to integrate the latent space features of the aligned image. Decoded into pixel-level images, this model uses the SDXL diffusion model as the base generator and, through the IP-Adapter mechanism, decodes the latent space features of the image generated by the bidirectional latent space diffusion generator. This cross-attention condition is input into the image generation network, thereby achieving precise control of the generation result by EEG semantics while maintaining high image quality.

[0075] Decoding is performed to pixel-level images. This model uses cross-attention conditions as input to the image generation network, thereby achieving precise control of the generated results by EEG semantics while maintaining high image quality. Step S3: Input the training set data established in step S1 into the model, and perform end-to-end training on the model by jointly optimizing the cross-modal contrastive learning loss and the bidirectional diffusion generation loss to generate the final trained model. The specific training process is as follows... Figure 2 The upper part includes:

[0076] Step S3.1: Model training is performed by jointly optimizing multiple loss functions to simultaneously achieve cross-modal semantic alignment and consistency in latent space generation. The training process specifically includes the following:

[0077] Step S3.1.1: Design of Joint Loss Function

[0078] Comparative learning loss The contrastive learning loss use The loss form, used to constrain the distance relationship between paired EEG embeddings and image embeddings in a shared semantic latent space, is defined as follows:

[0079] (6)

[0080] in:

[0081] : Comparative learning loss;

[0082] EEG latent space features output by a frequency-temporal dynamic encoder.

[0083] Image latent space features output by the image encoder.

[0084] Cosine similarity function

[0085] A learnable temperature parameter used to adjust the smoothness of the similarity distribution.

[0086] By minimizing this loss function, the distance between truly paired EEG-image embeddings can be effectively reduced, while the similarity of unpaired samples in the latent space can be increased, thereby enhancing the discriminative ability of cross-modal semantic alignment and providing stable semantic constraints for subsequent generation tasks.

[0087] EEG to Image Latent Space Diffusion Reconstruction Loss The EEG-to-image latent space diffusion reconstruction loss The ability of a model to reconstruct latent space features of an image under EEG conditions is defined as:

[0088] (7)

[0089] in:

[0090] EEG-to-Image Latent Reconstruction Loss is used to constrain the model's ability to reconstruct latent space features of an image under EEG conditions.

[0091] Expected value: The mathematical expectation, representing the average expected value of all values ​​taken by the random variable within the parentheses.

[0092] : Represents the latent features of the real image.

[0093] The distribution of the original image data, used for sampling to generate z.

[0094] Gaussian noise follows a standard normal distribution N(0,I).

[0095] The diffusion timestep, which is a random sampling method, has a distribution of u(1,T), meaning it samples uniformly between 1 and T.

[0096] The noise predictor learned by the model attempts to predict the added noise based on the features and conditional signals after the noise has been added.

[0097] EEG semantic conditions serve as an important input for guiding model generation.

[0098] and The noise schedule parameter in the diffusion model is used to control the original features at time step t. and noise The mixing ratio,

[0099] The square of the L2 norm is used to calculate the mean square error (MSE) between the predicted noise and the actual noise.

[0100] This loss, by minimizing the noise prediction error, prompts the model to learn to progressively denoise and recover the latent vector features of the image given the EEG semantics, thereby ensuring the stability and consistency of the EEG-to-image generation process.

[0101] Image to EEG latent space diffusion reconstruction loss Image to EEG latent space diffusion reconstruction loss Used to constrain the model to inversely reconstruct EEG latent space features under image conditions, it is defined as:

[0102] (8)

[0103] in:

[0104] Image-to-Image Latent Reconstruction Loss is used to constrain the model's ability to reconstruct latent space features of an image under EEG conditions.

[0105] Mathematical expectation. Represents the average expected value of all values ​​taken by the random variable within the parentheses.

[0106] : Represents the latent features of the real image.

[0107] The distribution of the original image data, used for sampling to generate z.

[0108] Gaussian noise follows a standard normal distribution N(0,I).

[0109] The diffusion timestep, which is a random sampling method, has a distribution of u(1,T), meaning it samples uniformly between 1 and T.

[0110] The noise predictor learned by the model attempts to predict the added noise based on the features and conditional signals after the noise has been added.

[0111] EEG semantic conditions. This serves as an important input for guiding model generation.

[0112] and The noise schedule parameter in the diffusion model is used to control the original features at time step t. and noise The mixing ratio,

[0113] The square of the L2 norm is used to calculate the mean square error (MSE) between the predicted noise and the actual noise.

[0114] This loss term is structurally symmetrical to the EEG-to-image latent space diffusion reconstruction loss, and is used to form bidirectional generation constraints at the latent space level. This avoids overfitting of the model only on unidirectional generation tasks, thereby improving the overall consistency and robustness of cross-modal mapping.

[0115] Total Loss Function: Combining the above loss terms, the total loss function for model training is defined as follows:

[0116] (9)

[0117] Among them, the weighting coefficient This is used to balance the relative contributions between the discriminative semantic alignment objective and the generative latent modeling objective.

[0118] Step S3.1.2: Training parameters and optimization strategy

[0119] The model was trained using the AdamW optimizer, with the weight decay factor set to 1×10⁻⁶. −2 To alleviate overfitting, the learning rate of the frequency-spatiotemporal dynamic encoder was set to 1×10⁻⁶. −4 The learning rate of the bidirectional latent space diffusion generator is set to 5×10. −5 The batch size is set to 128, the maximum number of training epochs is 1000, and a cosine annealing learning rate scheduling strategy is used during training, combined with an early stopping mechanism to avoid invalid iterations.

[0120] Step S3.1.3: Joint Training Process

[0121] The joint training process specifically includes the following steps:

[0122] A batch of paired EEG images is input into the complete model built in S2, without freezing any module parameters, and the total loss function is calculated synchronously via forward propagation. ,

[0123] Total losses Backpropagation is performed to synchronously update all the following trainable parameters: the frequency-guided module, neural graph structure extractor, spatiotemporal convolutional layer, and multilayer perceptron projection in the frequency-spatiotemporal dynamic encoder; the U-Net weights and FiLM conditional injection parameters in the EEG-to-image latent space diffusion generator; and the U-Net weights and FiLM conditional injection parameters in the image-to-EEG latent space diffusion generator.

[0124] This joint update mechanism avoids suboptimal convergence caused by staged training, and directly couples the semantic alignment objective and the generation consistency objective at the gradient level, accelerating model convergence and improving final performance.

[0125] Step S3.2: Model Validation. The model performance is evaluated using the following metrics to verify its effectiveness in cross-modal semantic alignment and consistency in latent space generation:

[0126] After each training epoch, the following three core evaluation metrics are calculated on the validation set, and training is terminated accordingly:

[0127] Top-k accuracy in image retrieval (used to measure semantic alignment capability): Assume the validation set contains M EEG-image paired samples. For the i-th EEG sample, its corresponding EEG embedding is... The CLIP embedding set of all candidate images is Calculate the cosine similarity:

[0128] (10)

[0129] in:

[0130] The cosine similarity score between the i-th EEG sample and the j-th candidate image.

[0131] : The EEG embedding vector corresponding to the i-th EEG sample,

[0132] : CLIP embedding of the j-th candidate image.

[0133] T (superscript): Transpose of a matrix or vector, used here to calculate the dot product of vectors.

[0134] ∥…∥: The L2 norm (modulus) of a vector, used to normalize the vector.

[0135] For each If the image index is truly paired Appears at the top of the similarity ranking If the position is specified, the retrieval is considered successful. Top-k accuracy is defined as follows:

[0136] (11)

[0137] in:

[0138] Top-k accuracy represents the probability of finding the correct pairing sample among the top k most similar results.

[0139] M: Total number of paired EEG-image samples in the validation set

[0140] : Indicator function. Its value is 1 if the condition within the parentheses is true, and 0 otherwise.

[0141] : The similarity score between the i-th EEG sample and its corresponding real paired image.

[0142] :express Set of similarity scores for all candidate images The rankings are arranged in descending order.

[0143] Fréchet Inception Distance (FID) (used to measure image reconstruction fidelity): Let... For model reconstruction Zhang Image For the corresponding real stimulus images, both are input into the pre-trained Inception-v3 network (up to the pool3 layer) to extract the feature mean and covariance:

[0144] (12)

[0145] , (13)

[0146] Then FID is defined as:

[0147] , (14)

[0148] in:

[0149] The image reconstructed by the model.

[0150] : The corresponding real stimulus image

[0151] The mean and covariance of real image features.

[0152] : The mean and covariance of reconstructed image features,

[0153] The trace of a matrix (the sum of its diagonal elements).

[0154] The smaller the value, the more realistic the reconstructed image.

[0155] The smaller the value, the closer the reconstructed image distribution is to the true image distribution.

[0156] CLIP Score (used to measure the consistency between the reconstructed image and the original semantics): For each pair of reconstructed images Compared with the original stimulus image Extract the normalized embedding from the CLIP image encoder output respectively. and Then the CLIP Score is:

[0157] , (15)

[0158] Reconstructing images The normalized embedded vector transpose

[0159] Original stimulus image The normalized embedding vector,

[0160] N: Number of test samples

[0161] The range of values ​​is A larger value indicates higher semantic consistency.

[0162] If the Top-k value does not improve and the FID decreases by less than 0.5 within 5 consecutive epochs, training is terminated, and the Top-k value on the validation set is saved. ( The model parameter with the highest overall score (balance coefficient) is selected; otherwise, proceed to step S31.

[0163] Step S3.3: Save the optimal model from step S3.2.

[0164] Step S3.4: Load the best model saved in step S3.3 for downstream task inference.

[0165] Through the above end-to-end joint training process, this invention effectively integrates discriminative semantic alignment and generative latent modeling, significantly improving the retrieval accuracy and reconstruction fidelity of the EEG decoding system.

[0166] The frequency represents the cosine similarity function of the image output by the image encoder; is a learnable temperature parameter used to adjust the smoothness of the similarity distribution. Step S4: Input the EEG data from the preprocessed test set data in S1 into the model saved in S3 to obtain the corresponding EEG latent space features. Calculate the cosine similarity with the image latent space features to retrieve the corresponding image. Reconstruct the corresponding image using the EEG-to-image latent space diffusion generator and the pre-trained SDXL model. The specific process is as follows: Figure 2 As shown in the lower part:

[0167] Step S4.1: When performing the image retrieval task, i.e., evaluating semantic alignment capability:

[0168] The image retrieval task is used to verify whether the model can map EEG signals to latent spatial locations consistent with their corresponding visual semantics, thereby achieving visual semantic retrieval based on EEG signals. The execution process includes the following steps:

[0169] Step S4.1.1: EEG Latent Vector Feature Extraction

[0170] All EEG samples in the test set are sequentially input into the trained frequency-spatiotemporal dynamic encoder. Forward inference is performed on each EEG sample to extract its corresponding EEG latent vector features, forming an EEG latent vector feature set. , where M represents the total number of paired EEG-image samples in the test set.

[0171] Step S4.1.2: Cross-modal similarity calculation

[0172] For each EEG latent vector feature in the test set Calculate its latent vector feature set with all candidate images in the test set. The cosine similarity between the two is used to measure the degree of matching between the EEG and the image in the shared semantic space.

[0173] Step S4.1.3: Sorting and Hit Determination

[0174] For each EEG data, all candidate images are sorted in descending order based on the calculated similarity value to form an image candidate list. If the stimulus image that is actually paired with the EEG data is within the top k positions of the sorting result (where k=1,5,10), then the sample is determined to have been successfully retrieved in the corresponding Top-k retrieval task.

[0175] Step S4.1.4: Retrieval Performance Statistics

[0176] The retrieval success rate of the model across all samples in the test set is calculated, and the average retrieval accuracy of Top-1, Top-5, and Top-10 is calculated and reported as a quantitative evaluation index of the model's cross-modal semantic alignment capability.

[0177] Experimental results show (see) Figure 6 In the within-subjects setting, the method of this invention achieved a Top-1 accuracy of 37.96% and a Top-5 accuracy of 70.67% in the 200-way task, significantly outperforming VE-SDN (37.20% / 69.90%) and MB2C (28.45% / 60.37%). In the more challenging 50-way task, it further improved to a Top-1 accuracy of 53.06% and a Top-5 accuracy of 87.09%, surpassing ATM (49.30% / 84.05%) and MB2C (50.47% / 84.20%). In the cross-subjects setting (see...), the method of this invention achieved a Top-1 accuracy of 37.96% and a Top-5 accuracy of 70.67% in the 200-way task, significantly outperforming VE-SDN (37.20% / 69.90%) and MB2C (50.47% / 84.20%). Figure 7Our method achieves 13.05% Top-1 accuracy and 34.55% Top-5 accuracy in the 200-way task, outperforming UBP (12.40% / 33.40%) and MB2C (11.90% / 32.03%). In the 50-way task, it achieves 27.20% Top-1 accuracy and 65.05% Top-5 accuracy, significantly outperforming ATM (21.60% / 53.10%) and MB2C (25.33% / 62.67%). Notably, without requiring additional supervision signals, our method achieves 38.0% Top-1 retrieval accuracy on THINGS-EEG, fully validating the effectiveness of the proposed frequency-guided semantic alignment mechanism in constructing a highly discriminative latent space.

[0178] Step S4.2: Image reconstruction task (for evaluating generation fidelity):

[0179] The image reconstruction task is used to verify whether the model can generate images with high visual fidelity and semantic content consistent with the original stimulus, given EEG data.

[0180] Step S4.2.1: EEG Latent Vector Feature Generation

[0181] For a single test EEG dataset, the EEG data is first input into the trained frequency-spatiotemporal dynamic encoder to extract the corresponding EEG latent vector features. This serves as a conditional semantic representation for the subsequent generation process.

[0182] Step S4.2.2: EEG latent vector feature generation using conditionally guided image latent vector feature diffusion.

[0183] The EEG latent vector features The EEG is input to the image latent space diffusion generator, which uses it as a conditional embedding. Starting from a standard Gaussian noise vector, a reverse diffusion process of T=1000 steps is performed. A linear variance scheduling strategy is employed during the diffusion process to progressively remove noise and generate stable image latent space features. This process ensures that the generated latent vectors are consistent with the real image embeddings in both statistical distribution and semantic structure.

[0184] Step S4.2.3: Pixel Space Image Decoding

[0185] latent space features of the generated image The IP-Adapter mechanism injects the data into the pre-trained SDXL diffusion model as a cross-attention condition, guiding SDXL to complete image generation in pixel space and finally outputting a reconstructed image with a resolution of 512×512.

[0186] Step S4.2.4: Reconstruction Performance Evaluation

[0187] Fréchet Inception Distance (FID): This measures the distribution distance between the reconstructed image and the real stimulus image in the Inception-v3 feature space. The smaller the value, the closer the generated image is to the real image in terms of overall distribution. The method of this invention achieves an FID of 19.3 on the THINGS-EEG dataset, which is more than 20 points higher than the MB2C method (FID≈40), significantly improving reconstruction fidelity.

[0188] CLIP Score: The score assesses the consistency between the generated image and the original visual semantics by calculating the cosine similarity between the reconstructed image and the corresponding original stimulus image in the CLIP latent space. The higher the score, the better the semantic matching.

[0189] Human subjective evaluation (optional): Through user research, human evaluators score the recognizability, object category consistency, and overall semantic matching of the reconstructed image as a supplementary verification to the objective indicators.

[0190] As shown in Table 1, in the quantitative reconstruction evaluation on THINGS-EEG, the proposed method is the best in all metrics: IS = 11.81 (MB2C: 10.19), FID = 146.33 (MB2C: 163.94), KID = 0.025 (MB2C: 0.027), SSIM = 0.350 (ATM: 0.345; MB2C: 0.333), PCC = 0.193 (MB2C: 0.188), indicating that the present invention significantly improves the perceptual quality, structural fidelity, and semantic consistency of the generated image.

[0191] In summary, this invention, through the joint optimization of frequency-spatiotemporal dynamic coding and diffusion generation mechanism, significantly outperforms existing technologies in both the two core tasks of zero-sample semantic retrieval and high-fidelity image reconstruction. It not only excels in individual adaptation scenarios but also possesses excellent cross-individual generalization capabilities, fully demonstrating its practical value and technological advancement in the fields of brain-computer interfaces, neural decoding, and human-machine collaborative perception.

[0192] Table 1. Quantitative assessment of EEG-to-image reconstruction quality on the THINGS-EEG dataset.

[0193] method IS ↑ FID ↓ KID ↓ SSIM ↑ PCC ↑ ATM - - - 0.345 0.160 MB2C 10.19 163.94 0.027 0.333 0.188 Ours 11.81 146.33 0.025 0.350 0.193

Claims

1. A modeling method for a bidirectional diffusion-guided frequency semantic alignment model based on a deep learning framework for EEG image reconstruction, characterized in that... include: A) Model building steps, including: A1) Frequency-spatiotemporal dynamic coding step, extracting frequency-aware deep spatiotemporal features from the raw EEG data, including: A11) performs frequency-oriented state-space structure modeling, using Mamba as the basic framework, which transforms the state transition matrix of the state-space SSM. The structure is a block diagonal form, with each block being a 2×2 sub-block. Defined as: Where the damping factor To ensure system stability, adaptive angular frequency It is given by the following formula: in, For adaptive angular frequency, For activation function, Based on the fundamental frequency parameter, This is a learnable logarithmic frequency bias term used to dynamically adjust the response intensity of the frequency-spatiotemporal dynamic encoding step to different frequency bands during training. This enables adaptive enhancement of the modeling ability for neural oscillation frequency bands, including Beta and Gamma, which are closely related to visual semantics and cognitive tasks. After discretizing the continuous-time state space, each sub-block Corresponding discrete form Expressed as: Among them, damping factor , For adaptive angular frequency, The time step parameter is a learnable parameter used to adapt to EEG signals with different sampling rates and time scales. This discretization form, while maintaining the rotation-decay characteristics of the continuous system, facilitates efficient parallel computation on a GPU, and then obtains... , Among them, a selective scanning mechanism is used to update the state and model the features of the time series of multi-channel EEG signals. The hidden state update and output calculation process follows the following relationship: in: t represents the discrete time step. This represents the original or intermediate characteristics of the c-th EEG electrode channel input at time t. This represents the hidden state vector of the c-th EEG channel at time t, characterizing the neurodynamic features of this channel in the frequency and time dimensions. This represents the output feature of the corresponding state-space model. , , , These are the learnable state transition matrix, input mapping matrix, and output mapping matrix, respectively. Their parameters are adaptively updated through backpropagation during model training. The channel-level feature vectors corresponding to all EEG channels are concatenated according to the channel dimension to form the final output feature matrix: , Among them, among them, This represents the feature vector obtained by aggregating the c-th EEG channel within a preset time window; symbol Let R denote the matrix transpose operation, R represent the set of real numbers, N represent the number of EEG electrode channels, and d represent the latent feature dimension corresponding to each channel. While maintaining linear time complexity, it can stably characterize the frequency-time dynamics information in long-term EEG sequences. A12) The neural map structure extraction step is used to model the spatial topological relationships of EEG electrodes, and the features output from step A11 are used to extract the neural map structure. Considered as graph node attributes, in the adjacency matrix defined by the international 10-20 standard electrode placement system. Perform graph convolution operation above: in, This is the degree matrix corresponding to the adjacency matrix. It is an adjacency matrix. The spatial topological features output in step A12, For the first Layer graph convolution weight parameters, for Activation function A13) Spatiotemporal feature extraction structure step, based on the output of step A11 The output of step A12 After fusion, further refined modeling of local spatiotemporal features is performed through spatiotemporal feature extraction operations. This involves employing the EEGNetV4 network structure, using depthwise separable convolution stacks to reduce parameter size and improve feature representation efficiency, and fusing temporal and spatial convolution operations. Finally, a projection multilayer perceptron is used. The fusion operation maps the fused features obtained to a fixed-dimensional latent space, and outputs... Dimensional EEG latent space characteristics : ,in Represents the real number field. Represents the dimension of the latent space, and the features of that latent space. It is used as a unified conditional representation for subsequent cross-modal alignment and generation modules. A2) Perform bidirectional latent space diffusion generation, establishing bidirectional generative cyclic consistency constraints between the EEG and the image latent space, including: A21) performs EEG to image latent space diffusion generation, in order to As a condition, through Time-step back-diffusion process reconstructs latent space features of an image from Gaussian noise. ; A22) performs image-to-EEG latent space diffusion generation, to As a condition, the latent space features of EEG are reconstructed from Gaussian noise through a T-time step back diffusion process. ; Both diffusion operations A21 and A22 employ an MLP-based U-Net network structure containing 5 basic modules, with a noise prediction network during the diffusion process. The inputs include: noisy latent vectors, conditional embeddings, and time steps. In this process, conditional embedding is injected into the middle layer of the network through the feature linear modulation layer, i.e., the FiLM layer, to dynamically adjust the network activation distribution, thereby achieving cross-modal conditional control.

2. The modeling method according to claim 1, characterized in that... Further includes: Step B) Training step: Input the pre-established training set data into the frequency semantic alignment model established in step A, and perform end-to-end training on the model by jointly optimizing the cross-modal contrastive learning loss and the bidirectional diffusion generation loss to generate the final trained model, including: Step B1) Model training is performed by jointly optimizing multiple loss functions to simultaneously achieve cross-modal semantic alignment and consistency in latent space generation, including: Step B11) Determine the joint loss function, including: B111) Determine the contrastive learning loss The contrastive learning loss use The loss form, used to constrain the distance relationship between paired EEG latent space features and image latent space features in a shared semantic latent space, is defined as follows: in: : Comparative learning loss, EEG latent space features output by a frequency-temporal dynamic encoder. Image latent space features output by the image encoder. :: Cosine similarity function A learnable temperature parameter used to adjust the smoothness of the similarity distribution. Through the By minimizing and narrowing the distance between truly paired EEG-image embeddings, while simultaneously widening the similarity of unpaired samples in the latent space, the discriminative power of cross-modal semantic alignment is enhanced, providing stable semantic constraints for subsequent generation tasks. B112) Determine the EEG to image latent space diffusion reconstruction loss : The ability of a frequency semantic alignment model to reconstruct latent space features of an image under EEG conditions is defined as follows: in: EEG to image latent space diffusion reconstruction loss, Expected value: The mathematical expectation, representing the average expected value of all values ​​taken by the random variable within the parentheses. : Represents the latent space features of a real image, The distribution of the original image data is used for sampling to generate latent space features. Gaussian noise The diffusion time step of random sampling has a distribution of u(1,T), meaning that sampling is performed uniformly between 1 and T. The noise predictor learned by the frequency semantic alignment model is used to predict the added noise based on the features and conditional signals after the noise has been added. EEG semantic conditions and These are the noise scheduling parameters in EEG-to-image latent space diffusion generation and image-to-EEG latent space diffusion generation, respectively, used to control the latent space features at time step t. and Gaussian noise The mixing ratio, The square of the L2 norm is used to calculate the mean square error (MSE) between the predicted noise and the actual noise. loss By minimizing noise prediction error, the frequency semantic alignment model learns to progressively denoise and recover latent vector features of the image given EEG semantics, thereby ensuring the stability and consistency of the EEG-to-image generation process. B113) Determine the image-to-EEG latent space diffusion reconstruction loss : The method used to constrain the frequency semantic alignment model for inverse reconstruction of EEG latent space features under image conditions is defined as follows: in: Image-to-EEG latent space diffusion reconstruction loss is used to constrain the ability of the frequency semantic alignment model to reconstruct image latent space features under EEG conditions. Expected value: The mathematical expectation, representing the average expected value of all values ​​taken by the random variable within the parentheses. : Represents the latent space features of a real image, The distribution of the original image data is used for sampling to generate latent space features. Gaussian noise The diffusion time step of random sampling has a distribution of u(1,T), meaning that sampling is performed uniformly between 1 and T. The noise predictor learned by the frequency semantic alignment model is used to predict the added noise based on the features and conditional signals after the noise has been added. EEG semantic conditions and These are the noise scheduling parameters in EEG-to-image latent space diffusion generation and image-to-EEG latent space diffusion generation, respectively, used to control the latent space features at time step t. and Gaussian noise The mixing ratio, The square of the L2 norm is used to calculate the mean square error (MSE) between the predicted noise and the actual noise. Loss Items EEG to image latent space diffusion reconstruction loss It is structurally symmetrical and forms bidirectional generative constraints at the latent space level, avoiding overfitting of the frequency semantic alignment model only on unidirectional generation tasks. B114) Determine the total loss function The sum of the above loss items is defined as follows: Among them, the weighting coefficient This is used to balance the relative contributions between the discriminative semantic alignment objective and the generative latent modeling objective. Step B12) Training parameters and optimization strategy, The model was trained using the AdamW optimizer, with the weight decay factor set to 1×10⁻⁶. −2 This is used to alleviate the overfitting problem. The learning rate of the frequency-spatiotemporal dynamic encoder is set to 1×10. −4 The learning rate of the bidirectional latent space diffusion generator is set to 5×10. −5 The batch size is set to 128, the maximum number of training epochs is 1000, and a cosine annealing learning rate scheduling strategy is used during training, combined with an early stopping mechanism to avoid invalid iterations. Step B13) Perform joint training, including: A batch of paired EEG images is input into the established frequency-semantic alignment model. Without freezing any module parameters, the total loss function is calculated through synchronous forward propagation. , Total losses Backpropagation is performed to jointly update all the following trainable parameters: the frequency-guided state-space structure modeling step, neural graph structure extraction step, spatiotemporal feature extraction structure step, and multilayer perceptron (MLP) in the frequency-spatiotemporal dynamic coding step; the U-Net weights and FiLM conditional injection parameters in the EEG-to-image latent space diffusion generation step; and the U-Net weights and FiLM conditional injection parameters in the image-to-EEG latent space diffusion generation step. This joint training is used to avoid suboptimal convergence, directly coupling the semantic alignment objective and the generative consistency objective at the gradient level, accelerating model convergence and improving final performance.

3. The modeling method according to claim 2, characterized in that... Step B) further includes: Step B2): Evaluate the performance of the frequency semantic alignment model using the following metrics: After each training round, the following three core evaluation metrics are determined on the validation set, and training is terminated accordingly: B21) Determine the Top-k accuracy of image retrieval: Suppose the validation set contains M EEG-image paired samples, and for the i-th EEG sample, its corresponding EEG embedding is... The CLIP embedding set of all candidate images is Calculate the cosine similarity: in: The cosine similarity score between the i-th EEG sample and the j-th candidate image. : The EEG embedding vector corresponding to the i-th EEG sample, : CLIP embedding of the j-th candidate image. T (superscript): Transpose of a matrix or vector, used here to calculate the dot product of vectors. ∥…∥: The L2 norm (modulus) of a vector, used to normalize the vector. For each If the image index is truly paired Appears at the top of the similarity ranking If the position is specified, the retrieval is considered successful. Top-k accuracy is defined as follows: in: Top-k accuracy represents the probability of finding the correct pairing sample among the top k most similar results. M: Total number of paired EEG-image samples in the validation set : Indicator function. Its value is 1 if the condition within the parentheses is true, and 0 otherwise. The similarity score between the i-th EEG sample and its corresponding real-world paired image. :express Set of similarity scores for all candidate images The rankings are arranged in descending order. B22) Determine FID: Let For model reconstruction Zhang Image For the corresponding real stimulus images, both are input into the pre-trained Inception-v3 network, which is then fed to the pool3 layer to extract the feature mean and covariance: Then FID is defined as: in: The image reconstructed by the model. : The corresponding real stimulus image The mean and covariance of real image features. : The mean and covariance of reconstructed image features, The trace of a matrix (the sum of its diagonal elements). The smaller the value, the more realistic the reconstructed image; that is, the smaller the value, the closer the distribution of the reconstructed image is to the distribution of the real image. B23) Determine the CLIP Score used to measure the consistency between the reconstructed image and the original semantics: for each pair of reconstructed images Compared with the original stimulus image By extracting the normalized embeddings #imgpt135# and #imgpt136# from the CLIP image encoder output, the CLIP Score is determined as follows: #imgpt138#: Normalized embedding vector transpose of reconstructed image #imgpt139# #imgpt140#: Normalized embedding vector of the original stimulus image #imgpt141# N: Number of test samples The value range of #imgpt142# is #imgpt143#, and the larger the value, the higher the semantic consistency. If Top-1 does not improve and FID decreases by less than 0.5 within 5 consecutive rounds, training is terminated, and the model parameters with the highest overall score on the validation set (#imgpt144#, where #imgpt145# is the balance coefficient) are saved; otherwise, step B1 is executed. Step B3): Save the optimal model parameters from step B2.

4. The modeling method according to claim 2, characterized in that... Step A further includes: The pre-trained frequency semantic alignment model includes converting the aligned EEG latent space features (#imgpt146#) into image latent space features (#imgpt147#) through EEG-to-image latent space diffusion, and then decoding them into pixel-level images, where: The SDXL diffusion model is used for basic generation operations. Furthermore, through the IP-Adapter mechanism, the latent space features of the image generated by the bidirectional latent space diffusion generator (#imgpt148#) are used as cross-attention conditions and input into the image generation network, thereby achieving precise control of the generation results by EEG semantics while maintaining high image quality.

5. The modeling method according to claim 2, characterized in that... The training set data was created through the following operations: Step C1): Preprocess the publicly available EEG-image pairing dataset and construct training and test sets for subsequent model training, image retrieval, and image reconstruction performance evaluation, including: Step C11): Channel selection is performed, including: selecting the THINGS-EEG public EEG-image paired dataset as the input data source, retaining 64 standard 10–20 system electrode channels, and removing damaged, drifting, and / or redundant channels to ensure signal quality and spatial coverage integrity. Step C12): Perform bandpass filtering and artifact removal, including applying a 1–100 Hz bandpass filter to the original continuous EEG signal to preserve the neural oscillation frequency bands closely related to visual perception. Step C13): Perform artifact removal, including: using an automatic artifact detection algorithm to identify and remove non-neurogenic interference such as eye movements, blinks, electromyography, and electrocardiograms. Step C14): Perform downsampling processing, including downsampling the original high-sampling-rate signal to 250 Hz to reduce computational overhead while maintaining sufficient temporal resolution to capture key neural events. Step C15): Perform data segmentation, including: segmenting continuous EEG data using fixed time windows according to the visual stimulus presentation protocol; in THINGS-EEG, each image is presented for 1000 ms, corresponding to 250 sampling points; each window of the fixed time window generates a 63×250 EEG matrix, and establishes a one-to-one mapping with its corresponding natural image. Step C16): Construct the training and test sets, including: A single-subject partitioning strategy was implemented: a separate training set and a test set were constructed for each subject. Brain signal data collected from the subject in multiple visual stimulation experiments were divided according to a preset ratio, with one part serving as the training set and the other as the test set. This was to evaluate the model's learning ability and image reconstruction performance for visual semantic representations under the same subject conditions. A cross-subject splitting strategy was implemented: a strict cross-subject splitting method was adopted, merging the data of N-1 subjects out of all N subjects to construct the training set, and using the data of the remaining subject who did not participate in the training as the test set, in order to verify the model's generalization ability on unseen individuals. In any of the above partitioning strategies, all images are embedded with d=768-dimensional image embedding #imgpt149# using the frozen CLIP-ViT / L-14 model as a semantic supervision signal; this partitioning method also supports a unified evaluation framework for image retrieval and image reconstruction tasks.

6. A method for reconstructing electroencephalogram images based on frequency guidance and bidirectional diffusion, characterized in that... Includes the following steps: Step S1: Preprocess the pre-acquired publicly available EEG and image pairing dataset by dividing the original data into training and test sets. The training set contains EEG and image pairing samples required for model learning, while the test set contains EEG and image pairing samples required for model testing, used to evaluate the model's image retrieval and image reconstruction capabilities. Step S2: Execute the modeling method for the bidirectional diffusion-guided frequency semantic alignment model according to any one of claims 2-4. Step S3: The EEG data from the preprocessed test set in S1 is input into the model stored in S2 to obtain the corresponding EEG latent space features. By calculating the cosine similarity with the latent space features of the image, the corresponding image is retrieved, and the corresponding image is reconstructed using the EEG-to-image latent space diffusion generator and the pre-trained SDXL model. Step S3 includes: S31) Perform image retrieval, including: retrieving the corresponding image by calculating the cosine similarity with the latent space features of the image. S32) Image generation includes: in the frequency-spatiotemporal dynamic coding step, using EEG data from the test set as input to obtain EEG latent space features; through the EEG-to-image latent space diffusion generation step, using the image latent space features as noise-free target signals and the EEG latent space features as conditions, generating corresponding image latent space features; using a pre-trained image generator to restore the image latent space features into pixel-level images, thereby finally obtaining the reconstructed image corresponding to the EEG data in the test set, realizing visual image reconstruction based on EEG data.

7. A computer-readable storage medium storing a computer-executable program that enables a processor to perform the method according to any one of claims 1-6.

Images