Electroencephalogram-based image generation method and device, computer device, and storage medium

Abstract

The application relates to the technical field of deep learning, and discloses an electroencephalogram-based image generation method and device, computer equipment and a storage medium, the method comprising the following steps: acquiring an electroencephalogram signal of a subject recalling a target image; inputting the electroencephalogram signal into an electroencephalogram encoder, generating the target image according to the electroencephalogram signal, and obtaining an imagined anchor point image; displaying the imagined anchor point image to the subject, re-acquiring an electroencephalogram signal of the subject watching the imagined anchor point image, and continuously generating a new imagined anchor point image; calculating a diversity value of the imagined anchor point image until the diversity value is reduced to a preset threshold value, and then determining that an image with the highest similarity in the current batch of imagined anchor point images is the target image. By repeatedly acquiring the electroencephalogram signal and the imagined anchor point image, the finally generated imagined anchor point image and the target image are closer, and the image generation accuracy is increased.

Description

Technical Field

[0001] This invention relates to the field of deep learning, and more particularly to an image generation method, apparatus, computer device, and storage medium based on electroencephalography (EEG). Background Technology

[0002] In existing deep learning technologies, there are two main methods for retrieving and reconstructing faces from memory: neural signal-based methods and drawing / textual description-based methods. Neural signal-based methods primarily utilize functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) signals. These methods typically use visual stimuli to induce neural signals generated when subjects imagine images, then use similarity metrics to retrieve the original image that best matches the imagined image, or use image reconstruction algorithms to reconstruct the imagined image. Drawing / textual description-based methods, on the other hand, utilize the subject's drawing or textual description of the imagined face to retrieve or reconstruct the corresponding image. In the current technological context, diffusion-generative models are often used for image reconstruction. Summary of the Invention

[0003] In a first aspect, this application provides an image generation method based on electroencephalography (EEG), comprising:

[0004] Acquire electroencephalogram (EEG) signals of subjects as they recall target images;

[0005] The EEG signal is input into the EEG encoder, and the target image is generated based on the EEG signal to obtain the imagination anchor point image;

[0006] The imagined anchor image is displayed to the subject, the EEG signals of the subject when viewing the imagined anchor image are reacquired, and new imagined anchor images are generated.

[0007] The diversity value of the imagined anchor images is calculated until the diversity value decreases to a preset threshold. Then, the image with the highest similarity among the imagined anchor images in the current batch is determined as the target image.

[0008] Furthermore, the step of inputting the EEG signal into the EEG encoder and generating the target image based on the EEG signal to obtain the imagined anchor point image includes:

[0009] The brainwave encoder converts the brainwave signal into an embedded brainwave signal;

[0010] Based on the EEG signal embedding, a search is performed in a given image database to determine the cosine similarity between the image embeddings of all images to be searched and the EEG signal embeddings.

[0011] The image with the highest cosine similarity to be retrieved is output as the imagined anchor image.

[0012] Furthermore, the step of inputting the EEG signal into the EEG encoder and generating the target image based on the EEG signal to obtain the imagined anchor point image includes:

[0013] The brainwave encoder converts the brainwave signal into an embedded brainwave signal;

[0014] The EEG signal is embedded into the cross-attention layer of a pre-trained diffusion generation model. The diffusion generation model reconstructs the image based on the embedded EEG signal, and the output image is an imagined anchor image.

[0015] Furthermore, the step of inputting the EEG signal into the EEG encoder and generating the target image based on the EEG signal to obtain the imagined anchor point image includes:

[0016] The EEG encoder is used to reduce the dimensionality of the EEG signal into an embedded EEG signal;

[0017] Obtain the text description of the target image, input the text description into a pre-trained text encoder, and obtain the text description embedding of the text description;

[0018] The text description embedding and the EEG signal embedding are jointly input into the cross-attention layer of the diffusion generation model. The diffusion generation model reconstructs the image based on the EEG signal embedding and the text description embedding, and the output image is the imagination anchor image.

[0019] Furthermore, the training method for the EEG encoder includes:

[0020] The brain signals of the subjects when they observe or recall preset images are acquired, and the brain signal embedding in the low-dimensional multimodal latent space is obtained in the pre-trained brain encoder through mask learning.

[0021] Obtain the standard description data of the preset image and obtain the standard description data embedding;

[0022] The pre-trained EEG encoder is enhanced by narrowing the distance between the standard descriptive data embedding and the EEG signal embedding in the low-dimensional multimodal latent space.

[0023] Furthermore, calculating the diversity value of the imagined anchor point image includes:

[0024] Calculate the cosine similarity between each imagined anchor point image, and then calculate the diversity value based on all the cosine similarities;

[0025] The expression for calculating the diversity value is:

[0026]

[0027] In the formula, d(x) i ,…,x j ) represents the diversity value, n represents the number of the imagined anchor images, X represents the imagined anchor images, i and j represent the sequence numbers of the imagined anchor images, and S represents the diversity value. CLIP (x i ,x j (x) is the imagined anchor point image. i and x j Cosine similarity between them.

[0028] Furthermore, the step of enhancing the pre-trained EEG encoder by narrowing the distance between the standard descriptive data embedding and the EEG signal embedding in the low-dimensional multimodal latent space includes:

[0029] Calculate the loss value of the EEG signal embedding and the standard descriptive data embedding, and modify the parameters of the pre-trained EEG encoder until the loss value is less than the expected value.

[0030] Secondly, this application also provides an image generation device based on electroencephalography (EEG), comprising:

[0031] The signal acquisition module is used to acquire the electroencephalogram (EEG) signals of the subject when recalling the target image;

[0032] An image generation module is used to input the EEG signal into the EEG encoder and generate the target image based on the EEG signal to obtain the imagination anchor point image;

[0033] An iterative module is used to display the image of the imagined anchor point to the subject, reacquire the EEG signals of the subject when viewing the image of the imagined anchor point, and continue to generate new images of the imagined anchor point.

[0034] The comparison module is used to calculate the diversity value of the imagined anchor images. When the diversity value decreases to a preset threshold, the image with the highest similarity in the current batch of imagined anchor images is determined as the target image.

[0035] Thirdly, this application also provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and the computer program executes the EEG-based image generation method when it is run on the processor.

[0036] Fourthly, this application also provides a readable storage medium storing a computer program that executes the EEG-based image generation method when run on a processor.

[0037] This invention discloses an image generation method, apparatus, computer device, and storage medium based on electroencephalography (EEG). The method includes: acquiring EEG signals when a subject recalls a target image; inputting the EEG signals into an EEG encoder, generating the target image based on the EEG signals to obtain an image of imagination anchors; displaying the image of imagination anchors to the subject, acquiring the EEG signals of the subject again when viewing the image of imagination anchors, and continuing to generate new image of imagination anchors; calculating the diversity value of the image of imagination anchors until the diversity value decreases to a preset threshold, then determining the image with the highest similarity among the current batch of image of imagination anchors as the target image. By repeatedly acquiring EEG signals and image of imagination anchors, the final generated image of imagination anchors is made closer to the target image, increasing the accuracy of image generation. It not only utilizes the subject's verbal descriptions but also effectively captures and analyzes their EEG signals, further improving the accuracy of image reconstruction. Attached Figure Description

[0038] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope of protection of the present invention. In the various drawings, similar components are numbered similarly.

[0039] Figure 1 This paper illustrates a flowchart of an image generation method based on electroencephalography (EEG) according to an embodiment of this application.

[0040] Figure 2 This paper illustrates a flowchart of a brainwave encoder training method according to an embodiment of this application.

[0041] Figure 3 A schematic diagram of an imaginary anchor point according to an embodiment of this application is shown;

[0042] Figure 4 A schematic diagram of an image generation device based on electroencephalography (EEG) according to an embodiment of this application is shown. Detailed Implementation

[0043] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0044] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0045] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0046] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0047] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0048] The technical solution of this application is applied to obtain the picture imagined by the subject based on the electroencephalogram (EEG) signal. The EEG signal is input into the EEG encoder, and the target image is generated according to the EEG signal to obtain the imagined anchor image. Then the subject is allowed to view the imagined anchor image to obtain a new EEG signal. The imagined anchor image is then generated again. Finally, when the diversity of the generated images is reduced to a certain extent, the generated image is confirmed to be the target image.

[0049] The technical solution of this application will be described below with specific embodiments.

[0050] Example 1

[0051] like Figure 1 As shown, the image generation method based on electroencephalography (EEG) in this embodiment includes:

[0052] Step S100: Acquire the electroencephalogram (EEG) signals of the subject when recalling the target image;

[0053] In this embodiment, it is necessary to acquire the EEG signals of the subject when recalling the target image, such as recalling the appearance of a person or the image of a specific object, and then acquire the EEG signals of the subject when imagining these images.

[0054] After acquiring EEG signals, some preprocessing operations are performed on the EEG signals, including filtering, denoising, and normalization, in order to reduce noise in the EEG signals and improve the signal-to-noise ratio.

[0055] Step S200: Input the EEG signal into the EEG encoder, and generate the target image based on the EEG signal to obtain the imagined anchor point image.

[0056] Then, the EEG signals are input into the EEG encoder, which is a model that maps the EEG signals to a low-dimensional multimodal latent space. This low-dimensional multimodal latent space can be the CLIP (Contrastive Language-Image Pre-Training) image / text space, or it can use dimensionality reduction algorithms such as t-SNE and UMAP to visualize or map the EEG data.

[0057] Based on the data obtained after processing by the EEG encoder as EEG signal embedding, this step further explains the training process of the EEG encoder in order to illustrate the effect of the above dimensionality reduction operation.

[0058] like Figure 2 As shown, the training method for the EEG encoder includes:

[0059] Step S210: Obtain the EEG signals when the subject observes or recalls a preset image, and obtain the standard descriptive data embedding of the EEG signals in the low-dimensional multimodal latent space through mask learning in the pre-trained EEG encoder.

[0060] During training, just like in step S100, the subject's EEG signals are collected. However, the EEG signals collected at this time are those when the subject observes or recalls a preset image. For example, a specific image is shown to the subject, and the subject imagines what the image looks like in his / her mind. The subject's EEG signals are collected at this time.

[0061] A pre-trained EEG encoder can be obtained using mask learning as the initial EEG encoder. At this point, the EEG encoder can process the EEG signal to obtain the EEG signal embedding. However, the obtained EEG signal embedding is only an initial value, not the final value. Only after enhancing the current EEG encoder through training can the ideal EEG signal embedding be obtained.

[0062] Step S220: Obtain the standard description data of the preset image and obtain the standard description data embedding;

[0063] It's understandable that an image itself contains image data that can describe it. In the field of image recognition, trained models identify the types of objects in an image based on this image data. Similarly, for speech recognition models, target images can be generated or retrieved based on a certain amount of textual description. These technologies are already quite mature, so this image data or textual description is standard descriptive data that can be directly obtained. Likewise, based on existing models, the embeddings of this image data or textual description in the aforementioned low-dimensional multimodal latent space can be directly obtained. These embeddings are standard descriptive data embeddings.

[0064] For example, an image can be processed using the BLIP (Graphical Multimodal Image-Text Interaction) model to obtain image and text embeddings. Thus, the aforementioned EEG signal embedding and standard data embedding are mapped to the same space.

[0065] Step S230: Enhance the training of the pre-trained EEG encoder by narrowing the distance between the standard description data embedding and the EEG signal embedding in the low-dimensional multimodal latent space.

[0066] In this context, standard data embeddings represent the standard features of the preset image. EEG signal embeddings should theoretically also represent the features of the image, but they will obviously differ from these standard data embeddings. Minimizing this difference indicates a higher similarity between the EEG signal embedding and the standard input embedding. In this low-dimensional multimodal latent space, these data points each have their own coordinates, so cosine similarity or Euclidean distance can be used to simply measure the similarity between two sets of data.

[0067] Therefore, the similarity between two data sets can be determined by calculating a loss value, for example, when the standard data embeddings mentioned above consist of both image embeddings and text embeddings. The expression for this loss value is:

[0068] L=(L CLIP (h e ,h i )+L CLIP (h e ,h i )) / 2

[0069] In the formula, L is the loss value, L CLIP (a,b)=CrossEntropy(∈·a·b T Let [0,1,…,n] be the cosine similarity between data a and b, where ∈ is a scaling factor, and h is the cosine similarity between data a and b. e For embedding of EEG signals, h i For image embedding, ht Embedded text.

[0070] If the standard data embedding consists of only one image embedding or text embedding, then the above formula only needs to calculate one cosine similarity.

[0071] The EEG encoder is adjusted and trained based on the aforementioned loss value for enhanced training. Finally, when the loss value is less than the preset threshold, it indicates that the EEG encoder training is complete.

[0072] As can be seen from the training process described above, after the EEG encoder processes the EEG signals, the resulting EEG signal embeddings are data that are sufficiently similar to known image embeddings or text embeddings. Therefore, for these EEG signal embeddings, a recognition model such as a diffusion generation model can be used for reconstruction, or a search can be performed in a given database to obtain the corresponding imagined anchor images.

[0073] Specifically, the retrieval method refers to searching for corresponding images in a given image database. During the retrieval, the EEG encoder converts the EEG signals into embedded EEG signals.

[0074] The image is retrieved from a given image database based on the embedded EEG signals. The cosine similarity between the image embeddings of all images to be retrieved and the embedded EEG signals is determined. The image with the highest cosine similarity is output as the imagination anchor image, thereby obtaining the imagination anchor image.

[0075] For example, if there are 100 images in the database, 100 sets of cosine similarity can be obtained by comparison. The top three images with the highest cosine similarity are selected as the imaginary anchor images.

[0076] The reconstruction method involves obtaining an embedded EEG signal, which is then input into the cross-attention layer of a pre-trained diffusion generation model. The model reconstructs the image based on the embedded EEG signal, outputting an image representing the imagined anchor point. Alternatively, textual descriptions provided by the subject can also be used as input, and reconstruction can be performed using both textual and EEG information.

[0077] It is understandable that the above-mentioned retrieval and reconstruction methods can be used individually or in combination to obtain multiple imagined anchor point images.

[0078] Step S300: Display the image of the imagined anchor point to the subject, reacquire the EEG signal of the subject when viewing the image of the imagined anchor point, and continue to generate new images of the imagined anchor point.

[0079] These imagery anchor images are displayed to the subject. When the subject views these images, it can enhance the subject's imagination of the target image, making the subject's imagination of the target image more concrete. Then, the EEG signals generated by the subject at this time are acquired, and the above step S200 is performed again to obtain new imagery anchor images.

[0080] It is understandable that when the subject initially imagines the target image, the resulting EEG signal will inevitably differ from the EEG signal obtained by directly viewing the target image. However, the similarity between the target image and the ideal EEG signal in the low-dimensional latent space will be very close. After generating the imagination anchor image using step S200, the EEG signal obtained by the subject after viewing the imagination anchor will be closer to the ideal EEG signal. At this time, the generated anchor image will also be closer to the target image. This process is repeated until the EEG signal converges to the ideal EEG signal, and the generated anchor image also converges to the target image. In this way, the target image can be obtained in the end.

[0081] Step S400: Calculate the diversity value of the imagined anchor image until the diversity value decreases to a preset threshold, then determine the image with the highest similarity among the imagined anchor images in the current batch as the target image.

[0082] According to the method in step S300, multiple imaginary anchor images can be generated each time. As the number of imaginary anchor image generation increases, the multiple imaginary anchor images generated will become more and more similar. For example, if four imaginary anchor images are generated each time, the three images that are obviously different at the beginning will become more and more similar after a certain number of rounds of imaginary anchor image generation.

[0083] like Figure 4 As shown, this image compares the imagined anchor points with the actual imagined images of the subjects in each round. The subject on the left in the image continuously imagines a target face image, and after three rounds of generating imagined anchor points, the face image generated in the third round is very close to the imagined target image, and the four images are also very similar to each other. Therefore, the diversity value between imagined anchor points can be used to determine whether the target image can be considered to have been generated.

[0084] Specifically, the diversity value can be calculated by calculating the cosine similarity between each imagined anchor point image and then calculating the diversity value based on all the cosine similarities.

[0085] The expression for calculating the diversity value is:

[0086]

[0087] In the formula, d(x) i ,…,x j) represents the diversity value, n represents the number of the imagined anchor images, X represents the imagined anchor images, i and j represent the sequence numbers of the imagined anchor images, and S represents the diversity value. CLIP (x i ,x j (x) is the imagined anchor point image. i and x j Cosine similarity between them.

[0088] When the diversity value is less than a certain level, it can be considered that these generated imaginary anchor images are all the same picture, and the differences can be ignored. Therefore, it can be considered that convergence has been completed, and the EEG signals emitted by the subject are already ideal EEG signals. At this time, the retrieved or reconstructed image is also the target image.

[0089] This embodiment's EEG-based image generation method iteratively optimizes and refines the final decoded facial image, making it closer to the subject's actual recollection. Furthermore, it innovatively proposes an interactive EEG-based face retrieval and reconstruction method by combining the unique connection between facial imagination and EEG signals. This method not only utilizes the subject's verbal descriptions but also effectively captures and analyzes their EEG signals, thereby accurately retrieving original images matching the imagined face from a large-scale database. This technology provides a novel approach for more intuitive and accurate face retrieval and reconstruction, overcoming the limitations of traditional face retrieval techniques. By employing a multimodal latent space combined with EEG signals, this invention ensures precise alignment of EEG signals, images, and textual descriptions in a common latent space. This structural design not only improves the consistency and correlation between EEG signals and images but also provides powerful conditions for facial image reconstruction, resulting in higher-quality facial images that more closely resemble the imagined face.

[0090] Example 2

[0091] like Figure 4 As shown, this application also provides an image generation device based on electroencephalography (EEG), comprising:

[0092] Signal acquisition module 10 is used to acquire the electroencephalogram (EEG) signals of the subject when recalling the target image;

[0093] The image generation module 20 is used to input the EEG signal into the EEG encoder and generate the target image based on the EEG signal to obtain the imagination anchor point image;

[0094] The iteration module 30 is used to display the image of the imagination anchor to the subject, reacquire the EEG signal of the subject when viewing the image of the imagination anchor, and continue to generate new images of the imagination anchor.

[0095] The comparison module 40 is used to calculate the diversity value of the imagined anchor image until the diversity value decreases to a preset threshold, then the image with the highest similarity in the current batch of imagined anchor images is determined as the target image.

[0096] This application also provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and the computer program executes the electroencephalogram-based image generation method when it is run on the processor.

[0097] This application also provides a readable storage medium storing a computer program that, when run on a processor, executes the described EEG-based image generation method.

[0098] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, as an alternative implementation, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0099] In addition, the functional modules or units in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0100] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating images based on electroencephalography (EEG), characterized in that, include: Acquire electroencephalogram (EEG) signals of subjects as they recall target images; The EEG signal is input into the EEG encoder, and the target image is generated based on the EEG signal to obtain the imagination anchor point image; The imagined anchor image is displayed to the subject, the EEG signals of the subject when viewing the imagined anchor image are reacquired, and new imagined anchor images are generated. Calculate the diversity value of the imagined anchor images until the diversity value decreases to a preset threshold, then determine the image with the highest similarity among the imagined anchor images in the current batch as the target image; The training method for the EEG encoder includes: The brain signals of the subject when observing or recalling a preset image are acquired, and a pre-trained brain encoder is obtained through mask learning to obtain the brain signal embedding of the brain signals in a low-dimensional multimodal latent space. Obtain the standard description data of the preset image and obtain the standard description data embedding; The pre-trained EEG encoder is enhanced by narrowing the distance between the standard description data embedding and the EEG signal embedding in the low-dimensional multimodal latent space. The calculation of the diversity value of the imagined anchor point image includes: Calculate the cosine similarity between each imagined anchor point image, and then calculate the diversity value based on all the cosine similarities; The expression for calculating the diversity value is as follows: ； In the formula, Let n be the diversity value, n be the number of the imagined anchor images, X be the imagined anchor images, and i and j be the sequence numbers of the imagined anchor images. For imaginary anchor point images Cosine similarity between them.

2. The image generation method based on electroencephalography (EEG) according to claim 1, characterized in that, The step of inputting the EEG signal into the EEG encoder and generating the target image based on the EEG signal to obtain the imagined anchor point image includes: The brainwave encoder converts the brainwave signal into an embedded brainwave signal; The cosine similarity between the image embeddings of all images to be retrieved and the image embeddings of the EEG signals is determined by searching a preset image database based on the EEG signal embeddings. The image with the highest cosine similarity to be retrieved is output as the imagined anchor image.

3. The image generation method based on electroencephalography (EEG) according to claim 1, characterized in that, The step of inputting the EEG signal into the EEG encoder and generating the target image based on the EEG signal to obtain the imagined anchor point image includes: The brainwave encoder converts the brainwave signal into an embedded brainwave signal; The EEG signal is embedded into the cross-attention layer of a pre-trained diffusion generation model. The diffusion generation model reconstructs the image based on the embedded EEG signal, and the output image is an imagined anchor image.

4. The image generation method based on electroencephalography (EEG) according to claim 1, characterized in that, The step of inputting the EEG signal into the EEG encoder and generating the target image based on the EEG signal to obtain the imagined anchor point image includes: The EEG encoder is used to reduce the dimensionality of the EEG signal into an embedded EEG signal; Obtain the text description of the target image, input the text description into a pre-trained text encoder, and obtain the text description embedding of the text description; The text description and the EEG signal are both input into the cross-attention layer of the diffusion generation model. The diffusion generation model reconstructs the image based on the EEG signal embedding and the text description embedding, and the output image is the imagination anchor image.

5. The image generation method based on electroencephalography (EEG) according to claim 1, characterized in that, The step of enhancing the pre-trained EEG encoder by narrowing the distance between the standard descriptive data embedding and the EEG signal embedding in the low-dimensional multimodal latent space includes: Calculate the loss value of the EEG signal embedding and the standard descriptive data embedding, and modify the parameters of the pre-trained EEG encoder until the loss value is less than the expected value.

6. An image generation device based on electroencephalography (EEG), characterized in that, include: The signal acquisition module is used to acquire the electroencephalogram (EEG) signals of the subject when recalling the target image; An image generation module is used to input the EEG signal into the EEG encoder and generate the target image based on the EEG signal to obtain the imagination anchor point image; An iterative module is used to display the image of the imagined anchor point to the subject, reacquire the EEG signals of the subject when viewing the image of the imagined anchor point, and continue to generate new images of the imagined anchor point. The comparison module is used to calculate the diversity value of the imagined anchor images. When the diversity value decreases to a preset threshold, the image with the highest similarity in the current batch of imagined anchor images is determined as the target image. The training method for the EEG encoder includes: The brain signals of the subject when observing or recalling a preset image are acquired, and a pre-trained brain encoder is obtained through mask learning to obtain the brain signal embedding of the brain signals in a low-dimensional multimodal latent space. Obtain the standard description data of the preset image and obtain the standard description data embedding; The pre-trained EEG encoder is enhanced by narrowing the distance between the standard description data embedding and the EEG signal embedding in the low-dimensional multimodal latent space. The calculation of the diversity value of the imagined anchor point image includes: Calculate the cosine similarity between each imagined anchor point image, and then calculate the diversity value based on all the cosine similarities; The expression for calculating the diversity value is as follows: ； In the formula, Let n be the diversity value, n be the number of the imagined anchor images, X be the imagined anchor images, and i and j be the sequence numbers of the imagined anchor images. For imaginary anchor point images Cosine similarity between them.

7. A computer device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when run on the processor, executes the EEG-based image generation method according to any one of claims 1 to 5.

8. A readable storage medium, characterized in that, It stores a computer program that, when run on a processor, executes the EEG-based image generation method according to any one of claims 1 to 5.

Images