Electroencephalogram signal decoding and visualization method and device based on pre-training diffusion model, equipment, medium and product

Abstract

The invention discloses an electroencephalogram signal decoding and visualization method and device based on a pre-training diffusion model, equipment, a medium and a product, and relates to electroencephalogram signals in the field of biometers.The method comprises the steps that an experiment normal form is designed, and original electroencephalogram signals are obtained; processing the original electroencephalogram signal to obtain a processed electroencephalogram signal; obtaining text description from the visual stimulation image according to a language-image pre-training BLIP model; obtaining text embedding from the text description according to a language-image pre-training CLIP model; constructing and training an EEG encoder, and mapping the electroencephalogram signal to a text embedding space to obtain electroencephalogram embedding; the pre-training diffusion model Stable Diffusion is subjected to fine adjustment, and an adjusted pre-training diffusion model is obtained; and reconstructing electroencephalogram embedding according to the adjusted pre-training diffusion model to obtain an image with consistent semantics, so that the problem that fine-grained semantic information is difficult to decode from electroencephalogram data is solved, and the EEG is reconstructed into the image with consistent visual stimulation semantics.

Description

Technical Field

[0001] The present application relates to electroencephalogram signals in the field of biometrics, and particularly to a method, device, equipment, medium and product for decoding and visualizing electroencephalogram signals based on a pre-trained diffusion model. Background Art

[0002] The human brain has always been regarded as an extremely complex intelligent system, and researchers have conducted a large number of studies on it. In this field, decoding the brain's response to visual stimuli has become a popular research topic with broad application prospects. However, due to the complexity of the human brain system, decoding human brain activities remains a huge challenge.

[0003] The most widely used brain imaging method is electroencephalography (EEG), mainly because of its low cost and high temporal resolution. EEG is an objective response of neural activities in the brain. When an individual faces different visual stimuli, the electroencephalogram signals can capture the changes in the brain activity patterns of the individual when receiving different visual stimuli. Therefore, the research on decoding and visualizing the human brain activities induced by visual stimuli of electroencephalogram signals has attracted much attention. However, there are the following challenges: 1) The amount of electroencephalogram data is very limited, contains a large amount of noise, and has significant individual differences, making it difficult to learn meaningful information from it. 2) There are huge differences in the data spaces of EEG and images, so it is difficult to directly map EEG to images. Summary of the Invention

[0004] The purpose of the present application is to provide a method, device, equipment, medium and product for decoding and visualizing electroencephalogram signals based on a pre-trained diffusion model, which solves the problems of difficult to decode fine-grained semantic information from electroencephalogram data and difficult to reconstruct EEG into an image consistent with the semantic of visual stimuli.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In the first aspect, the present application provides a method for decoding and visualizing electroencephalogram signals based on a pre-trained diffusion model, including:

[0007] Design an experimental paradigm to obtain original electroencephalogram signals.

[0008] Process the original electroencephalogram signals to obtain processed electroencephalogram signals; the processed electroencephalogram signals constitute an electroencephalogram signal dataset.

[0009] Divide the electroencephalogram signal dataset into a training set and a test set; the electroencephalogram signal dataset includes visual stimulus images.

[0010] Obtain a text description from the visual stimulus images according to the first language pre-training model.

[0011] Obtain a text embedding from the text description according to the second-language pre-trained model.

[0012] Construct and train an EEG encoder to map the processed EEG signals into the text embedding space to obtain EEG embeddings.

[0013] Based on the EEG embeddings, fine-tune the pre-trained diffusion model according to the training set to obtain an adjusted pre-trained diffusion model.

[0014] Process the EEG embeddings according to the adjusted pre-trained diffusion model to obtain semantically consistent images.

[0015] In a second aspect, the present application provides an EEG signal decoding and visualization device based on a pre-trained diffusion model, including:

[0016] An original EEG signal acquisition module for designing an experimental paradigm and acquiring original EEG signals.

[0017] A processed EEG signal determination module for processing the original EEG signals to obtain processed EEG signals; the processed EEG signals constitute an EEG signal data set.

[0018] An EEG signal data set partitioning module for partitioning the EEG signal data set into a training set and a test set; the EEG signal data set includes visual stimulus images.

[0019] A text description determination module for obtaining a text description from the visual stimulus images according to the first-language pre-trained model.

[0020] A text embedding determination module for obtaining a text embedding from the text description according to the second-language pre-trained model.

[0021] An encoder and EEG embedding determination module for constructing and training an EEG encoder to map the processed EEG signals into the text embedding space to obtain EEG embeddings.

[0022] A pre-trained diffusion model adjustment module for fine-tuning the pre-trained diffusion model according to the training set based on the EEG embeddings to obtain an adjusted pre-trained diffusion model.

[0023] A semantically consistent image determination module for processing the EEG embeddings according to the adjusted pre-trained diffusion model to obtain semantically consistent images.

[0024] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the steps of the electroencephalogram signal decoding and visualization method based on a pre-trained diffusion model described in any one of the above.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the electroencephalogram signal decoding and visualization method based on a pre-trained diffusion model described in any one of the above.

[0026] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the electroencephalogram signal decoding and visualization method based on a pre-trained diffusion model described in any one of the above.

[0027] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0028] The present application provides an electroencephalogram signal decoding and visualization method, device, equipment, medium and product based on a pre-trained diffusion model. Design an experimental paradigm to obtain the original electroencephalogram signal; process the original electroencephalogram signal, and effectively improve the quality of the electroencephalogram signal through filtering and denoising to obtain the processed electroencephalogram signal; divide the processed electroencephalogram signal into a training set and a test set; obtain a text description from a visual stimulus image according to the language-image pre-trained BLIP model; tokenize and process the text description according to the tokenizer of the language-image pre-trained CLIP model, and convert the processed text input into a text embedding through the CLIP model text encoder. The language-image pre-trained CLIP model is a cross-modal model that maps images and texts to the same embedding space through contrastive learning, and is pre-trained with a large amount of image-text, enabling the model to understand the relationship between text and image; train the EEG encoder according to the mean square error loss and cosine similarity loss between the EEG embedding and the text embedding to obtain a trained EEG encoder; fine-tune the pre-trained diffusion model StableDiffusion based on the training set to obtain an adjusted pre-trained diffusion model; input the electroencephalogram data in the test set into the trained EEG encoder to obtain an electroencephalogram embedding aligned with the text embedding, and then input the electroencephalogram embedding into the adjusted pre-trained diffusion model to obtain a visualized image, realizing the reconstruction of electroencephalogram data into an image. Description of the Drawings

[0029] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0030] Figure 1 It is a schematic flowchart of a method for electroencephalogram signal decoding and visualization based on a pre-trained diffusion model in an embodiment of the present application;

[0031] Figure 2 It is a schematic diagram of the working process of a method for electroencephalogram signal decoding and visualization based on a pre-trained diffusion model provided by an embodiment of the present application;

[0032] Figure 3 It is a schematic diagram of the EEG encoder network structure in an embodiment of the present application;

[0033] Figure 4 It is a schematic diagram of the network structure of the pre-trained diffusion model Stable Diffusion in an embodiment of the present application;

[0034] Figure 5 It is a schematic diagram for comparing visualization results for the ImageNet-EEG-40 dataset provided by an embodiment of the present application;

[0035] Figure 6 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0037] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0038] In recent years, significant progress has been made in the research of cross-modal models, especially in the fields of image and text understanding. By jointly learning the representations of images and texts, these models can demonstrate powerful capabilities in multi-modal tasks, such as the Contrastive Language-Image Pre-training (CLIP) and the Bootstrapping Language-Image Pre-training (BLIP). CLIP maps images and texts to the same embedding space through contrastive learning, thus achieving the ability of zero-shot learning and being widely applied to tasks such as image classification, retrieval, and image-text matching. BLIP, on the other hand, enhances the interaction ability between images and texts by combining self-supervised learning and generative models, and is particularly outstanding in tasks such as image caption generation and visual question answering. With the continuous progress of these models, they have shown great potential in the understanding and generation of multi-modal data. In addition, recently, diffusion models have achieved remarkable success in image generation tasks. Diffusion models include a forward process and a reverse process. In the forward process, the input image is gradually noisy to be transformed into Gaussian noise. In the reverse process, by gradually denoising, the Gaussian noise can be almost perfectly inverted to the original image. Some studies have shown that diffusion models have surpassed Generative Adversarial Networks (GAN) and Variational Autoencoder (VAE) in image generation.

[0039] Based on the above challenges and existing research, this application will propose a new method for decoding and visualizing human brain activities from electroencephalogram (EEG) signals. This method uses the pre-trained BLIP and CLIP to extract rich text embeddings from images, then aligns the EEG signals with the text embeddings through a convolutional neural network, and finally reconstructs the aligned EEG embeddings into images using the pre-trained text-to-image diffusion model (StableDiffusion), which is the pre-trained diffusion model.

[0040] As Figure 2 shown, this application designs an EEG signal decoding and visualization method based on the pre-trained diffusion model, mainly including two parts: 1) Combining the pre-trained BLIP and CLIP models, an EEG encoder is designed to map EEG signals to the text embedding space; 2) Using the pre-trained text-to-image diffusion model Stable Diffusion to reconstruct the aligned EEG embeddings into images with the same semantics as the EEG signals.

[0041] The technical solutions adopted in this application include the following steps:

[0042] Data acquisition: Design an image stimulation paradigm based on the EEG paradigm design platform Openvibe and the scripting language Python. The subjects watch images of different categories, and the EEG signals of each segment and the category labels of the images are recorded.

[0043] Data processing: Filter and denoise the collected raw EEG data, and slice it into time slices of 0.5 seconds; divide all the data into a training set and a test set.

[0044] Image description and text embedding generation: Use the BLIP model to generate text descriptions from images, and then use the CLIP model to generate text embeddings from the text descriptions.

[0045] Construction and training of the EEG encoder: Construct an EEG encoder to map EEG signals to the text embedding space, thereby generating EEG embeddings; use the training set to train the model, and update the model parameters using the gradient descent algorithm.

[0046] Fine-tuning the pre-trained Stable Diffusion model: Fine-tune the pre-trained Stable Diffusion with the EEG embeddings and images in the training set, and update the model parameters.

[0047] Testing and result evaluation: Generate images from EEG embeddings using the fine-tuned Stable Diffusion, and use the Inception Score to verify the effectiveness of this method.

[0048] As Figure 1 shown, the embodiments of this application provide a method for decoding and visualizing EEG signals based on a pre-trained diffusion model, specifically including:

[0049] S1: Design an experimental paradigm to obtain raw EEG signals.

[0050] S2: Process the raw EEG signals to obtain processed EEG signals; the processed EEG signals constitute an EEG signal dataset.

[0051] S3: Divide the EEG signal dataset into a training set and a test set; the EEG signal dataset includes visual stimulation images.

[0052] S4: Obtain text descriptions from the visual stimulation images according to the first language pre-trained model.

[0053] S5: Obtain text embeddings from the text descriptions according to the second language pre-trained model.

[0054] S6: Construct and train an EEG encoder to map the processed EEG signals into the text embedding space to obtain EEG embeddings.

[0055] S7: Based on the EEG embeddings, fine-tune the pre-trained diffusion model according to the training set to obtain an adjusted pre-trained diffusion model.

[0056] S8: Process the EEG embeddings according to the adjusted pre-trained diffusion model to obtain semantically consistent images.

[0057] This application discloses an EEG signal decoding and visualization method based on a pre-trained diffusion model. This application first obtains data for preprocessing, slicing, and dividing the data set; then combines the pre-trained BLIP and CLIP models to generate text embeddings from images; then constructs and trains an EEG encoder to map EEG signals into the text embedding space; finally, fine-tunes the pre-trained Stable Diffusion model to generate images from EEG embeddings, and uses the Inception Score to verify the effectiveness of this method. This application makes full use of pre-trained multi-modal models, effectively aligns EEG signals with external stimuli (images and texts), can extract fine-grained information from EEG, and achieves good results on the ImageNet-EEG-40 data set.

[0058] Further, in an exemplary embodiment, step S1 can be replaced by the following steps.

[0059] S101: Design a picture stimulation paradigm based on the EEG paradigm design platform Openvibe and the scripting language Python.

[0060] Design a picture stimulation paradigm based on the EEG paradigm design platform Openvibe and the scripting language Python. The stimulation images come from 4 easily recognizable categories (cats, dogs, flowers, and pandas) of the ImageNet data set. Each category contains 50 different images, for a total of 200 images. Each paradigm image is switched every 0.5 seconds, and each stimulation image is displayed on the screen for 0.4 seconds, and then a blank image is displayed for 0.1 seconds to eliminate the influence of the current stimulation image on the next visual stimulation. Each image will appear 4 times. Continuous display of images of the same category can effectively strengthen the visual stimulation of the same category. When the image interface disappears and the end prompt appears, the task ends.

[0061] S102: Stimulate the subject according to the picture stimulation paradigm to obtain the original EEG signals.

[0062] The subjects watched images of different categories, and the electroencephalogram (EEG) signals of each segment and the category labels of the images were recorded. There were 26 subjects, with normal or corrected-to-normal vision. The EEG signal acquisition device used was EMOTIVE, which included 32 electrodes and had a sampling rate of 128 Hz.

[0063] Further, in an exemplary embodiment, step S2 can be replaced by the following steps.

[0064] S201: Filter the original EEG signal and remove the electrooculogram (EOG) artifacts from the original EEG signal according to the independent component analysis method to obtain the processed EEG signal.

[0065] The collected original EEG data was filtered with a band-pass filter of 1 Hz - 70 Hz, then independent component analysis was used to remove the EOG artifacts, and it was sliced into time slices of 0.5 seconds each to obtain the processed EEG signal. The processed EEG signal was randomly divided into a training set and a test set at a ratio of 4:1.

[0066] Further, in an exemplary embodiment, step S5 can be replaced by the following steps.

[0067] S501: Encode the visual stimulus image according to the first-language pre-trained model to obtain a text description.

[0068] The first-language pre-trained model is the language-image pre-trained BLIP model. BLIP is a cross-modal method that combines self-supervised learning and generative models, which improves the interaction ability between images and texts and can provide more accurate text descriptions for image content. The pre-trained BLIP model was used to encode the images and generate relevant text descriptions.

[0069] S502: Tokenize and process the text description according to the tokenizer of the second-language pre-trained model to obtain the processed text input.

[0070] S503: Convert the processed text input into a text embedding according to the text encoder of the second-language pre-trained model.

[0071] The second - language pre - trained model is the language - image pre - trained CLIP model. CLIP is a cross - modal model that maps images and text to the same embedding space through contrastive learning. Its core idea is to pre - train through a large number of image - text pairs so that the model can understand the relationship between text and images. Load the pre - trained CLIP model, use the CLIP tokenizer to tokenize and process the text descriptions generated in S401, and use the CLIP text encoder to convert the processed text input into text embeddings. To match the input requirements of the Stable Diffusion model, map the text to a fixed - dimension embedding space.

[0072] S504: Align the EEG data to the text embedding space using the EEG encoder to obtain EEG embeddings.

[0073] S505: Use the formula to obtain the difference between the EEG embedding and the text embedding; where, L mse (eeg, text) is the mean - square error loss; eeg i is the EEG embedding of the i - th sample; text i is the text embedding of the i - th sample; n is the total number of samples;

[0074] S506: Use the formula to calculate the similarity between the EEG embedding and the text embedding.

[0075] S507: Use the formula L s = L mse (eeg, text)+k·L cos to obtain the total loss between the EEG embedding and the text embedding;

[0076] where, L s is the total loss between the EEG embedding and the text embedding; L mse (eeg, text) is the mean - square error loss; eeg is the EEG embedding extracted by the EEG encoder; text is the text embedding; k is the weight of the cosine similarity loss; L cos (eeg, text) is the cosine similarity loss, which is used to measure the similarity between the EEG embedding and the text embedding.

[0077] As Figure 3 shown in the EEG encoder network structure diagram, the EEG encoder takes EEG data as input and outputs EEG embeddings aligned to the text embedding space. The working process is as follows:

[0078] a) First, input the EEG data into the Temporal Block. It consists of two convolutional layers with a kernel size of (5, 1) and a stride of (2, 1).

[0079] b) Then, send the generated tensor into the Spatial Block. It consists of two convolutional layers with a kernel size of (1, 5) and a stride of (1, 2).

[0080] c) Then, process it through the Temporal - Spatial Block to obtain the EEG embedding. It consists of a convolutional layer with a kernel size of (5, 5) and a stride of (2, 2) and an average pooling layer.

[0081] d) Finally, adjust the dimension of the EEG embedding to 77×768 through a fully - connected layer to obtain the final output.

[0082] BatchNorm normalization and ReLU activation are adopted after each convolutional layer.

[0083] According to the total loss function L s Calculate the loss, and use the gradient - descent algorithm to train the EEG encoder to obtain a trained EEG encoder. Use the Adam optimizer with an initial learning rate of 0.001, a batch size of 64, 100 training epochs, and the dimension of the generated EEG embedding is 77×768.

[0084] Furthermore, in an exemplary embodiment, step S7 can be replaced by the following steps.

[0085] S701: Based on the EEG embedding and image pairs of the training set, use the formula to fine - tune the pre - trained diffusion model to obtain an adjusted pre - trained diffusion model; where, L SD is the SD loss function; is the expectation; ∈ θ is the denoising function implemented by U - Net; x0 is the original image; T is the total number of time steps; t is a time step uniformly distributed between 1 and T; x t is the noisy image at time step t; ∈ is the standard Gaussian noise; τ θ (y) is the intermediate representation projected by the EEG embedding y through the encoder τ θ ; is the square of the L2 norm.

[0086] After obtaining effective EEG embedding from the EEG signal, the pre-trained Stable Diffusion model is used to generate images. Stable Diffusion is a potential diffusion model that adds a cross-attention mechanism to the traditional diffusion model. The network structure diagram of the Stable Diffusion model is shown in the figure below. Figure 4 As shown. EEG embedding y is passed through the encoder τ θ Projection to Intermediate Representation Then through an implementation The cross attention layer maps it to the middle layer of U-Net, where Q is the query matrix, K is the keyword matrix, V is the value matrix, and d is the dimension of the query matrix and keyword matrix

[0087]

[0088] Among them, i represents the i-th attention head in the multi-head attention mechanism, z t represents the denoising state at time step t, is the intermediate representation of U-Net, is the value matrix of the i-th head, is the query matrix of the ith head, is the keyword matrix of the i-th head, which are projection matrices with learnable parameters. The pre-trained diffusion model is the StableDiffusion model.

[0089] The EEG embedding is input into the adjusted pre-trained diffusion model to generate a visualized image, and the Inception Score of the visualized image is calculated. The EEG data in the test set is input into the trained EEG encoder to obtain the EEG embedding aligned with the text embedding, and then the EEG embedding is input into the fine-tuned Stable Diffusion, the number of inference steps is set to 100, the guidance strength is set to 10, the image is generated, and the Inception Score of the generated image is calculated. The Inception Score can measure the quality and diversity of the generated image. The higher the score, the better the performance of the generated model.

[0090] The experimental results are shown in Table 1. Figure 5 As shown in FIG. 1 , the visualization result comparison of the present application for the ImageNet-EEG-40 dataset is shown. The original image is the image viewed by the subject, the text description is generated from the original image, and the generated image is generated from the EEG signal when the subject views the image. The original image can be as shown in FIG. Figure 5The shown scenes of a panda bear in the grass, a pizza oven, a large white plane, and a butterfly sitting on a leaf, corresponding to the text descriptions "a panda bear in the grass", "a pizza oven", "a large white plane", and "a butterfly sitting on a leaf", generate corresponding images. Table 1 and Figure 5 It can be proved that this application has achieved relatively good results in terms of the Inception Score metric and visualization.

[0091] Table 1 Comparison of Experimental Results

[0092] Method Inception Score BCVAE 1.62 DCVAE 1.98 DM-RE2I 12.55 NeuroDM 15.04 This application 16.93

[0093] The embodiment of this application provides an electroencephalogram (EEG) signal decoding and visualization device based on a pre-trained diffusion model, specifically including:

[0094] An original EEG signal acquisition module, used to design an experimental paradigm and acquire the original EEG signal.

[0095] A processed EEG signal determination module, used to process the original EEG signal to obtain a processed EEG signal; the processed EEG signals constitute an EEG signal dataset.

[0096] A processed EEG signal division module, used to divide the EEG signal dataset into a training set and a test set; the EEG signal dataset includes visual stimulus images.

[0097] A text description determination module, used to obtain a text description from the visual stimulus images according to a first-language pre-trained model.

[0098] A text embedding determination module, used to obtain a text embedding from the text description according to a second-language pre-trained model.

[0099] An encoder and EEG embedding determination module, used to construct and train an EEG encoder to map the processed EEG signal into a text embedding space to obtain an EEG embedding.

[0100] A pre-trained diffusion model adjustment module, used to fine-tune the pre-trained diffusion model based on the EEG embedding according to the training set to obtain an adjusted pre-trained diffusion model.

[0101] A semantically consistent image determination module, used to process the EEG embedding according to the adjusted pre-trained diffusion model to obtain a semantically consistent image.

[0102] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as shown in Figure 6 the following figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a video tag processing method.

[0103] An embodiment of the present application provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for decoding and visualizing electroencephalogram signals based on a pre-trained diffusion model described in any one of the above.

[0104] An embodiment of the present application provides a program product, including a computer program. When the computer program is executed by a processor, it implements the method for decoding and visualizing electroencephalogram signals based on a pre-trained diffusion model described in any one of the above.

[0105] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0106] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for decoding and visualizing EEG signals based on a pre-trained diffusion model, characterized in that: The EEG signal decoding and visualization method based on the pre-trained diffusion model includes: Design experimental paradigms and obtain raw EEG signals; Processing the original EEG signal to obtain a processed EEG signal; the processed EEG signal constitutes an EEG signal data set; Dividing the EEG signal data set into a training set and a test set; the EEG signal data set includes visual stimulation images; Obtaining a text description from the visual stimulus image according to a first language pre-trained model; Obtaining a text embedding from the text description according to a second language pre-trained model; Constructing and training an EEG encoder to map the processed EEG signal to a text embedding space to obtain an EEG embedding; Based on the EEG embedding, fine-tuning the pre-trained diffusion model according to the training set to obtain an adjusted pre-trained diffusion model; The EEG embedding is processed according to the adjusted pre-trained diffusion model to obtain a semantically consistent image.

2. The method for decoding and visualizing EEG signals based on a pre-trained diffusion model according to claim 1, characterized in that: Design the experimental paradigm and obtain the original EEG signals, including: Design the picture stimulation paradigm based on the EEG paradigm design platform Openvibe and the scripting language Python; The subject is stimulated according to the picture stimulation paradigm to obtain raw EEG signals.

3. According to the method for decoding and visualizing EEG signals based on a pre-trained diffusion model in claim 1, the original EEG signal is processed to obtain a processed EEG signal, specifically comprising: The original EEG signal is filtered, and the electrooculographic artifacts of the original EEG signal are removed according to an independent component analysis method to obtain a processed EEG signal.

4. The method for decoding and visualizing EEG signals based on a pre-trained diffusion model according to claim 1, characterized in that: Obtaining text embedding from the text description according to the second language pre-training model, specifically including: encoding the visual stimulus image according to a first language pre-trained model to obtain a text description; Obtaining text embedding from the text description according to the second language pre-training model, specifically including: Segmenting and processing the text description according to the tokenizer of the second language pre-trained model to obtain a processed text input; The processed text input is converted into a text embedding according to a text encoder of the second language pre-trained model.

5. The method for decoding and visualizing EEG signals based on a pre-trained diffusion model according to claim 1, characterized in that: After obtaining text embedding from the text description according to the second language pre-training model, the method further includes: Aligning the EEG data to the text embedding space using an EEG encoder to obtain an EEG embedding; Using the formula Obtaining a difference between the EEG embedding and the text embedding; Among them, L mse (eeg,text) is the mean square error loss; eeg i is the EEG embedding of the i-th sample; text i is the text embedding of the i-th sample; n is the total number of samples; Using the formula Calculating the similarity between the EEG embedding and the text embedding; Using the formula L s =L mse (eeg,text)+k·L cos (eeg,text) gets the total loss of the EEG embedding and the text embedding; Among them, L s is the total loss of EEG embedding and text embedding; L mse (eeg, text) is the mean square error loss; eeg is the EEG embedding extracted by the EEG encoder; text is the text embedding; k is the weight of the cosine similarity loss; L cos (eeg,text) is the cosine similarity loss, which is used to measure the similarity between EEG embedding and text embedding.

6. The method for decoding and visualizing EEG signals based on a pre-trained diffusion model according to claim 5, characterized in that: Based on the EEG embedding, the pre-trained diffusion model is fine-tuned according to the training set to obtain an adjusted pre-trained diffusion model, specifically including: Based on the EEG embedding and image pairs of the training set, the formula The pre-trained diffusion model is fine-tuned to obtain an adjusted pre-trained diffusion model; wherein L SD is the SD loss function; is the expectation;∈ θ is the denoising function implemented by U-Net; x0 is the original image; T is the total number of time steps; t is the time step uniformly distributed between 1 and T; x t is the noise image at time step t; ∈ is standard Gaussian noise; τ θ (y) is the EEG embedding y through the encoder τ θ The intermediate representation projected to, is the square of the L2 norm.

7. An electroencephalogram signal decoding and visualization device based on a pre-trained diffusion model, characterized in that: The EEG signal decoding and visualization device based on the pre-trained diffusion model includes: The original EEG signal acquisition module is used to calculate the experimental paradigm and obtain the original EEG signal; A processed EEG signal determination module is used to process the original EEG signal to obtain a processed EEG signal; the processed EEG signal constitutes an EEG signal data set; An EEG signal data set division module, used for dividing the EEG signal data set into a training set and a test set; the EEG signal data set includes a visual stimulation image; a text description determination module, configured to obtain a text description from the visual stimulus image according to a first language pre-trained model; A text embedding determination module, configured to obtain a text embedding from the text description according to a second language pre-trained model; An encoder and EEG embedding determination module, used to construct and train an EEG encoder, map the processed EEG signal to a text embedding space, and obtain an EEG embedding; A pre-trained diffusion model adjustment module, used for fine-tuning the pre-trained diffusion model based on the EEG embedding and the training set to obtain an adjusted pre-trained diffusion model; The semantically consistent image determination module is used to process the EEG embedding according to the adjusted pre-trained diffusion model to obtain a semantically consistent image.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for decoding and visualizing EEG signals based on a pre-trained diffusion model as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for decoding and visualizing EEG signals based on a pre-trained diffusion model described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for decoding and visualizing EEG signals based on a pre-trained diffusion model described in any one of claims 1 to 6 is implemented.

Images