Image generation method and device for eliminating bad information in text map, and medium

Abstract

The invention provides an image generation method and device for eliminating bad information in a text picture, equipment and a medium. The method comprises the following steps: acquiring a data set containing an image triple of a bad picture, a picture subjected to thick mosaic processing on the bad picture and a normal picture; the method comprises the following steps: constructing an SD model containing a self-attention layer, training the SD model by using a data set of an image triple, converting the image triple into an implicit vector triple by an encoder, circularly adding noise into the implicit vector triple through a DDPM noise scheduler, denoising the implicit vector triple full of the noise by the self-attention layer, and outputting the denoising result. Calculating a total loss function according to an output result of the self-attention layer; and inputting the text information into the trained SD model to generate an image for eliminating bad information, so as to solve the problems that external defense measures face an undergeneralization problem, and internal defense measures are difficult to process vocabularies which are seemingly unrelated but have sexual hints, and the vocabularies are difficult to enumerate.

Description

Technical Field

[0001] The present invention relates to the technical field of text-generated image technology, and in particular to an image generation method, device, equipment and medium for eliminating bad information in a text-generated image. Background Art

[0002] With the booming development of artificial intelligence technology, text-generated image models have gradually become a research hotspot. In this era of data explosion, massive amounts of text data and image data provide sufficient resources for model training. The continuous integration of computer vision and natural language processing has led to the birth of text-generated image models. Researchers hope to establish effective model architectures such as Stable Diffusion (SD), MidJourney, DALL·E, etc. to enable computers to understand the semantic information in text like humans and convert it into vivid image content. However, such models can also be used for unethical or illegal behaviors such as the production of false content and the dissemination of misleading information.

[0003] In the existing technology, solutions are proposed from external and internal perspectives, specifically:

[0004] First, external defense measures use security filters to detect inappropriate text input or image output when generating images. This method is easy to deploy, but it can be easily removed at the code level for open source models. In addition, filters can also be used to review text-image pairing data that is not suitable for the workplace (Not-Safe-For-Work, NSFW) and retrain Stable Diffusion 2.1 (SD-V2.1) from scratch, but it may take up to 200,000 hours.

[0005] Second, internal defenses employ modifications to the text-to-image model itself. Existing internal defenses are designed to guide the model to detect sexually relevant words and are therefore text-dependent. Such defenses require predefined workplace unsuitable concepts to avoid potentially unsafe areas, or fine-tuning model parameters to suppress inappropriate text.

[0006] Therefore, it is urgent to propose an image generation method, device, equipment and medium for eliminating bad information in text-generated images, so as to solve the technical problems that the external defense measures in the text-to-image model in the prior art face under-generalization problems, are insufficient in filtering adversarial cues and explicit images, the internal defense measures have difficulties in processing seemingly irrelevant but sexually suggestive words, the words are difficult to enumerate, and visually explicit expressions may be produced in the text-to-image model. Summary of the invention

[0007] In order to overcome the problems existing in the related art, the present disclosure provides an image generation method, device, equipment and medium for eliminating bad information in a text-to-image model, so as to solve the technical problems that the external defense measures in the text-to-image model in the related art face the problem of under-generalization, there are deficiencies in filtering adversarial prompts and explicit images, the internal defense measures have difficulties in processing seemingly irrelevant but sexually suggestive words, the words are difficult to enumerate, and visually explicit expressions may be produced in the text-to-image model.

[0008] One or more embodiments of the present specification provide a method for generating an image by eliminating bad information in a cultural image, comprising the following steps:

[0009] Obtain a data set of image triplets, each triplet comprising three images: a bad picture, a picture after thick mosaic processing of the bad picture, and a normal picture;

[0010] Constructing an SD model including a self-attention layer, and training the SD model using the dataset of the image triples, wherein the encoder of the SD model converts the image triples into latent vector triples, and adds noise to the latent vector triples through a DDPM noise scheduler to obtain noisy latent vector triples, and the self-attention layer denoises the noisy latent vector triples to obtain denoised latent vectors, and calculates the total loss function according to the output result of the self-attention layer, and performs back-propagation to perform optimization operations to obtain a trained SD model;

[0011] The text information is input into the trained SD model to generate an image with the bad information eliminated.

[0012] Preferably, the self-attention layer denoises the noisy latent vector triplet to obtain the denoised latent vector, which specifically includes the following steps:

[0013] Generate the attention matrix W using the self-attention mechanism K ,W V and W Q , converting the noisy latent vector triplet into two vectors through an attention matrix, and converting the intermediate latent vector of the noisy latent vector triplet into a matrix through the attention matrix;

[0014] The matrix Q is multiplied by the vector K and processed by the softmax function to obtain a self-attention matrix;

[0015] The attention matrix M is multiplied by the vector V to obtain the denoised latent vector O=MV.

[0016] Preferably, the total loss function includes the minimum loss function L m And minimize the loss function Lp , specifically expressed as:

[0017]

[0018] Among them, λ m Represents the size of the hyperparameter coefficient that retains the ability to generate benign images, λ p Represents the size of the hyperparameter coefficient of the bad picture that needs to be erased, Represents the denoising UNet model's estimate of the noise component in the bad picture, represents the latent vector of the nude image after T times of denoising, represents the latent vector of the mosaic image after T times of denoising, represents the noise added by the noise scheduler in this process, Represents the denoising UNet model's estimate of the noise component in a benign image.

[0019] Preferably, the adding noise to the latent vector triplet by cyclically using a DDPM noise scheduler specifically comprises the following steps:

[0020] The same noise is added to the bad latent vectors and the mosaicked bad latent vectors by the DDPM noise scheduler.

[0021] One or more embodiments of this specification provide an image generation device for eliminating bad information in a cultural image, including a data acquisition module, a model training module and an image generation module;

[0022] The data acquisition module is used to acquire a data set of image triplets, each triplet containing three images: a bad picture, a picture after thick mosaic processing of the bad picture, and a normal picture;

[0023] The model training module includes a model construction unit, which is used to construct an SD model including a self-attention layer, and train the SD model using the data set of the image triples; a noise adding unit, which is used for the encoder of the SD model to convert the image triples into latent vector triples, and add noise to the latent vector triples through a DDPM noise scheduler to obtain noisy latent vector triples; a denoising unit, which is used for the self-attention layer to denoise the noisy latent vector triples to obtain denoised latent vectors; and an optimization unit, which is used to calculate the total loss function according to the output result of the self-attention layer, and perform back-propagation to perform optimization operations to obtain a trained SD model;

[0024] The image generation module is used to input text information into the trained SD model to generate an image that eliminates bad information.

[0025] Preferably, the denoising unit is specifically configured as follows:

[0026] Used to generate the attention matrix W using the self-attention mechanism K ,W V and W Q , the noisy latent vector triplet is passed through the attention matrix W K ,W V Transformed into two vectors K, V, through the attention matrix W Q Convert the intermediate latent vector of the noisy latent vector triplet into a matrix Q;

[0027] The matrix Q is multiplied by the vector K and processed by the softmax function to obtain a self-attention matrix M;

[0028] The attention matrix M is multiplied by the vector V to obtain the denoised latent vector O=MV.

[0029] Preferably, the total loss function includes the minimum loss function L m And minimize the loss function L p , specifically expressed as:

[0030]

[0031] Among them, λ m Represents the size of the hyperparameter coefficient that retains the ability to generate benign images, λ p Represents the size of the hyperparameter coefficient of the bad picture that needs to be erased, Represents the denoising UNet model's estimate of the noise component in the bad picture, represents the latent vector of the nude image after T times of denoising, represents the latent vector of the mosaic image after T times of denoising, represents the noise added by the noise scheduler in this process, Represents the denoising UNet model's estimate of the noise component in a benign image.

[0032] Preferably, the noise adding unit is specifically configured as follows:

[0033] The same noise is added to the bad latent vector and the bad latent vector after mosaic processing by the DDPM noise scheduler.

[0034] One or more embodiments of the present specification provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image generation method for eliminating bad information in the above-mentioned Vincent map when executing the computer program.

[0035] One or more embodiments of the present specification provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of the image generation method for eliminating bad information in the above-mentioned Vincent diagram are implemented.

[0036] The present disclosure provides an image generation method, device, equipment and medium for eliminating bad information in a cultural image. The advantages are that, by acquiring a data set of image triplets including bad pictures, pictures after thick mosaic processing of the bad pictures, and normal pictures, the model is trained only by image data without the need for text data guidance, thereby cutting off the connection between the model's explicit visual representation and conceptual sex-related prompt words; constructing an SD model including a self-attention layer, and training the SD model using the data set of the image triplets, wherein the encoder of the SD model converts the image triplets into latent vector triplets, and cyclically adds the latent vector triplets to the latent vector triplets through a DDPM noise scheduler. Noise is obtained, and a latent vector triplet full of noise is obtained. The self-attention layer denoises the latent vector triplet full of noise to obtain a denoised latent vector. The total loss function is calculated according to the output result of the self-attention layer, and back-propagation is performed to perform optimization operations to obtain a trained SD model. The quality and semantics of the image generated by the model are adjusted through the unconditional denoising diffusion of the self-attention layer. The text information is input into the trained SD model to generate an image in which bad information is eliminated in the text map, so that the model can prevent the generation of unsafe images even under the guidance of adversarial prompts, and does not affect the quality of normal image generation, which is conducive to the subsequent targeted security enhancement of text-to-image applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 A schematic flow chart of an image generation method for eliminating bad information in a cultural image provided by one or more embodiments of this specification;

[0039] Figure 2 A schematic diagram of a SD model training method provided for one or more embodiments of this specification;

[0040] Figure 3 A flowchart of an image generation method for eliminating bad information in a cultural image provided by one or more embodiments of this specification;

[0041] Figure 4 A comparison chart of the removal rate of bad images of a method for generating an image to eliminate bad information in a cultural image provided by one or more embodiments of this specification and other methods;

[0042] Figure 5 A structural diagram of an image generation device for eliminating bad information in a cultural image provided by one or more embodiments of this specification;

[0043] Figure 6 A schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present invention.

[0045] The present invention is described in detail below in conjunction with specific implementation methods and the accompanying drawings.

[0046] Method Embodiment

[0047] According to an embodiment of the present invention, a method for generating an image by eliminating bad information in a cultural image is provided. Figure 1 FIG. 1 is a flow chart of an image generation method for eliminating bad information in a Vincent diagram provided by the present embodiment. The image generation method for eliminating bad information in a Vincent diagram according to the present embodiment includes the following steps:

[0048] Step S110: Obtain an image triplet including about 100 bad pictures, pictures after thick mosaic processing of the bad pictures, and normal pictures <x n ,x m ,x b >a data set, wherein a mosaic neural network is used to blur the bad picture to obtain a thick mosaic processed picture of the bad picture, and a normal picture is randomly sampled from a Human Detection Dataset.

[0049] Step S120: construct a SD model including a self-attention layer, using the image triplet <x n ,x m ,x b>The dataset is used to train the SD model, such as Figure 2 As shown, it is a schematic diagram of the SD model training method provided in this embodiment, wherein the encoder of the SD model converts the image triples <x n ,x m ,x b >Converted into latent vector triples The superscripts n, m, and b represent bad pictures, pictures after thick mosaic processing of the bad pictures, and normal pictures, respectively. The subscripts represent the time steps, which are [0, T]. The noise is added to the latent vector triples through the DDPM (Denoising Diffusion Probabilistic Models) noise scheduler loop. Get the noisy latent vector triplet The DDPM noise scheduler adds the same noise to the bad latent vector and the mosaic-processed bad latent vector Under the input of blank text information, the self-attention layer of the U-Net structure in the SD model pays attention to the noisy latent vector triplet To denoise, from its attention matrix W Q ,W K ,W V The connection between the relevant text and the bad visual representation is gradually removed to obtain the denoised latent vector, as shown in Figure 3 As shown, it is a workflow diagram of an image generation method for eliminating bad information in a cultural image provided by this embodiment.

[0050] At the same time, the total loss function is calculated according to the output result of the self-attention layer, the Adam optimizer is used for gradient descent optimization, and back propagation is performed to update the parameters related to the self-attention layer in the U-Net structure.

[0051] Step S130: input the text information into the trained SD model to generate an image with the bad information eliminated.

[0052] The method provided in this embodiment obtains a data set of image triplets including bad pictures, pictures after thick mosaic processing of the bad pictures, and normal pictures, and trains the model only through image data without the guidance of text data, thereby cutting off the connection between the model's explicit visual representation and conceptual sex-related prompt words; constructs an SD model including a self-attention layer, and trains the SD model using the data set of the image triplets, wherein the encoder of the SD model converts the image triplets into latent vector triplets, and adds noise to the latent vector triplets cyclically through a DDPM noise scheduler to obtain a latent vector triplets full of noise. The self-attention layer denoises the noisy latent vector triplet to obtain the denoised latent vector, calculates the total loss function according to the output result of the self-attention layer, performs back-propagation optimization operation to obtain a trained SD model, and adjusts the quality and semantics of the image generated by the model through the unconditional denoising diffusion of the self-attention layer; inputs the text information into the trained SD model to generate an image that eliminates bad images, so that the model can prevent the generation of unsafe images even under the guidance of adversarial prompts, and does not affect the quality of normal image generation, which is conducive to the subsequent targeted security enhancement of text-to-image applications.

[0053] In one embodiment, the classifier-independent image generation guidance is extensively used, which includes a conditional denoising diffusion process ∈ U (z t ,c,t) and an unconditional denoising diffusion process ∈ U (z t ,t). The noise predicted at step t It can be calculated by the following formula:

[0054]

[0055] Among them, when the guidance scale parameter η>1 (generally set to 7.5), the predicted noise is more inclined to the conditional score and gradually deviates from the unconditional score. After this iterative denoising process is completed, z0 can be converted to the image space through the pre-trained decoder

[0056] The conditional denoising diffusion process in the denoising process is implemented by the cross-attention layer, while the unconditional denoising diffusion process is implemented by the self-attention layer. Specifically, the self-attention layer denoises the noisy latent vector triplet to obtain the denoised latent vector, which specifically includes the following steps:

[0057] Generate the attention matrix W using the self-attention mechanism K ,W V and W Q , the noisy latent vector triplet Through the attention matrix W K ,W V Transformed into two vectors K, V, through the attention matrix W Q The noisy latent vector triplet The intermediate latent vector is transformed into the matrix Q.

[0058] The matrix Q is multiplied by the vector K and processed by the softmax function to obtain the self-attention matrix M.

[0059] The attention matrix M is multiplied by the vector V to obtain the denoised latent vector O=MV, which contains important information retained during the denoising process.

[0060] The method provided in this embodiment can understand visual information more comprehensively, cut off the connection between explicit visual representation and conceptual sex-related prompt words, and ultimately effectively prevent the generation of bad images.

[0061] In one embodiment, the optimization goal of the SD model is to gradually denoise the latent vector of the noisy bad picture by U-Net until it becomes a noise-free picture after thick mosaic processing of the bad picture, that is, the latent vector At the same time, the latent vector of the normal, no-naked image full of noise is transformed into the latent vector of the normal, no-naked image without noise, that is, the latent vector Therefore, the total loss function is designed to include the minimum loss function L m And minimize the loss function L p , minimize the loss function L m The term enables the self-attention layer to remove the representation of bad pictures, such as mapping them to latent vectors processed by thick mosaics, while minimizing the loss function L p The term can ensure that these layers can maintain the representation quality of normal images and avoid parameter deviation, which is specifically expressed as:

[0062]

[0063] Among them, λ m Represents the size of the hyperparameter coefficient that retains the ability to generate benign images, λ p Represents the size of the hyperparameter coefficient of the bad picture that needs to be erased, Represents the denoising UNet model's estimate of the noise component in the bad picture, represents the latent vector of the nude image after T times of denoising, represents the latent vector of the mosaic image after T times of denoising, represents the noise added by the noise scheduler in this process, Represents the denoising UNet model's estimate of the noise component in a benign image.

[0064] Furthermore, noise Obeying the normal distribution, each time it is added to the original latent vector Noise are all predefined by the DDPM noise scheduler. In other words, Represents the sum of the noise added in the noise adding process, that is: Similarly, represents the total noise predicted by the U-Net U* after adjustment in the self-attention layer. It is found through experiments that the scale parameter is set to λ m ∶0.1,λ p ∶0.9 can make the model best block bad pictures and retain normal images. Figure 4 As shown, it is a comparison chart of the removal rate of bad pictures of an image generation method for eliminating bad information in a cultural image provided by this embodiment and other methods.

[0065] The method provided in this embodiment enables the model to best prevent bad images and retain normal images by setting a loss function that enables the self-attention layer to remove the representation of bad images and maintain the representation quality of normal images and avoid parameter shift.

[0066] Device Embodiment

[0067] According to an embodiment of the present invention, a device for generating an image in which bad information is eliminated from a cultural image is provided. Figure 5 As shown, it is a structural diagram of the image generation device for eliminating bad information in a Vincent diagram provided in this embodiment. According to the image generation device for eliminating bad information in a Vincent diagram according to an embodiment of the present invention, it includes a data acquisition module, a model training module and an image generation module.

[0068] The data acquisition module 510 is used to acquire a data set of image triplets including a bad picture, a picture after thick mosaic processing of the bad picture, and a normal picture.

[0069] The model training module 520 includes a model construction unit 521 for constructing an SD model including a self-attention layer, and using the data set of the image triples to train the SD model; a noise adding unit 522 for the encoder of the SD model to convert the image triples into latent vector triples, and to add noise to the latent vector triples through a DDPM noise scheduler to obtain noisy latent vector triples. Specifically, the same noise is added to the bad latent vectors and the bad latent vectors after mosaic processing through the DDPM noise scheduler; a denoising unit 523 for the self-attention layer to denoise the noisy latent vector triples to obtain the denoised latent vectors; and an optimization unit 524 for calculating the total loss function according to the output result of the self-attention layer, and performing back-propagation to perform optimization operations to obtain a trained SD model.

[0070] The image generation module 530 is used to input text information into the trained SD model to generate an image with the bad information eliminated.

[0071] The device provided in this embodiment obtains a data set of image triplets including bad pictures, pictures after thick mosaic processing of the bad pictures, and normal pictures through a data acquisition module 510, trains the model only through image data without the need for text data guidance, and cuts off the connection between the model's explicit visual representation and conceptual sex-related prompt words; constructs an SD model including a self-attention layer through a model training module 520, and trains the SD model using the data set of the image triplets, wherein the encoder of the SD model converts the image triplets into latent vector triplets, and adds noise to the latent vector triplets through a DDPM noise scheduler to obtain a noise-filled latent vector triplets. The self-attention layer denoises the noisy latent vector triplet to obtain the denoised latent vector, calculates the total loss function according to the output result of the self-attention layer, and performs back-propagation to perform optimization operations to obtain a trained SD model, and adjusts the quality and semantics of the image generated by the model through the unconditional denoising diffusion of the self-attention layer; inputs the text information into the trained SD model through the image generation module 530 to generate an image with bad information eliminated, so that the model can prevent the generation of unsafe images even under the guidance of adversarial prompts, and does not affect the quality of normal image generation, which is conducive to the subsequent targeted security enhancement of text-to-image applications.

[0072] In one embodiment, the denoising unit 523 is specifically configured as follows:

[0073] Used to generate the attention matrix W using the self-attention mechanism K ,W V and W Q , the noisy latent vector triplet is passed through the attention matrix WK ,W V Transformed into two vectors K, V, through the attention matrix W Q The intermediate latent vector of the noisy latent vector triplet is converted into a matrix Q.

[0074] The matrix Q is multiplied by the vector K and processed by the softmax function to obtain the self-attention matrix M.

[0075] The attention matrix M is multiplied by the vector V to obtain the denoised latent vector O=MV.

[0076] The device provided in this embodiment can understand visual information more comprehensively, cut off the connection between explicit visual representation and conceptual sex-related prompt words, and ultimately effectively prevent the generation of bad images.

[0077] In one embodiment, the total loss function includes the minimum loss function L m And minimize the loss function L p , specifically expressed as:

[0078]

[0079] Among them, λ m Represents the size of the hyperparameter coefficient that retains the ability to generate benign images, λ p Represents the size of the hyperparameter coefficient of the bad picture that needs to be erased, Represents the denoising UNet model's estimate of the noise component in the bad picture, represents the latent vector of the nude image after T times of denoising, represents the latent vector of the mosaic image after T times of denoising, represents the noise added by the noise scheduler in this process, Represents the denoising UNet model's estimate of the noise component in a benign image.

[0080] The device provided in this embodiment enables the model to best achieve bad images by setting a loss function that can enable the self-attention layer to remove the representation of bad images and maintain the representation quality of normal images and avoid parameter deviation. The embodiment of the present invention is a device embodiment corresponding to the above-mentioned method embodiment. The specific operations of the processing steps of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.

[0081] like Figure 5As shown, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image generation method for eliminating bad information in the raw image in the above-mentioned embodiment, or, when executed by a processor, implements the image generation method for eliminating bad information in the raw image in the above-mentioned embodiment.

[0082] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0083] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A method for generating an image to eliminate bad information in a cultural graph, characterized in that: The following steps are involved: Obtain a data set of image triplets including a bad picture, a picture after thick mosaic processing of the bad picture, and a normal picture; Constructing an SD model including a self-attention layer, and training the SD model using the dataset of the image triples, wherein the encoder of the SD model converts the image triples into latent vector triples, and adds noise to the latent vector triples through a DDPM noise scheduler to obtain noisy latent vector triples, and the self-attention layer denoises the noisy latent vector triples to obtain denoised latent vectors, and calculates the total loss function according to the output result of the self-attention layer, and performs back-propagation to perform optimization operations to obtain a trained SD model; The text information is input into the trained SD model to generate an image with the bad information eliminated.

2. The image generation method according to claim 1, characterized in that: The self-attention layer denoises the noisy latent vector triple to obtain a denoised latent vector, specifically comprising the following steps: Generate the attention matrix W using the self-attention mechanism K , W V and W Q , the noisy latent vector triplet is passed through the attention matrix W K , W V Transformed into two vectors K, V, through the attention matrix W Q Convert the intermediate latent vector of the noisy latent vector triplet into a matrix Q; The matrix Q is multiplied by the vector K and processed by the softmax function to obtain a self-attention matrix M; The attention matrix M is multiplied by the vector V to obtain the denoised latent vector O=MV.

3. The image generation method according to claim 1, wherein: The total loss function includes the minimum loss function L m And minimize the loss function L p , specifically expressed as: Among them, λ m Represents the size of the hyperparameter coefficient that retains the ability to generate benign images, λ p Represents the size of the hyperparameter coefficient of the bad picture that needs to be erased, Represents the denoising UNet model's estimate of the noise component in the bad picture, represents the latent vector of the nude image after T times of denoising, Represents the latent vector of the mosaic image after T times of denoising, represents the noise added by the noise scheduler in this process, Represents the denoising UNet model's estimate of the noise component in a benign image.

4. The image generation method according to claim 1, wherein: The adding noise to the latent vector triplet by the DDPM noise scheduler loop specifically includes the following steps: The same noise is added to the bad latent vectors and the mosaicked bad latent vectors by the DDPM noise scheduler.

5. An image generation device for eliminating bad information in a cultural image, characterized in that: Including data acquisition module, model training module and image generation module; The data acquisition module is used to acquire a data set of image triplets including a bad picture, a picture after thick mosaic processing of the bad picture, and a normal picture; The model training module includes a model construction unit, which is used to construct an SD model including a self-attention layer, and train the SD model using the data set of the image triples; a noise adding unit, which is used for the encoder of the SD model to convert the image triples into latent vector triples, and add noise to the latent vector triples through a DDPM noise scheduler to obtain noisy latent vector triples; a denoising unit, which is used for the self-attention layer to denoise the noisy latent vector triples to obtain denoised latent vectors; and an optimization unit, which is used to calculate the total loss function according to the output result of the self-attention layer, and perform back-propagation to perform optimization operations to obtain a trained SD model; The image generation module is used to input text information into the trained SD model to generate an image with good information eliminated.

6. The image generating device according to claim 5, wherein: The denoising unit is specifically configured as follows: Used to generate the attention matrix W using the self-attention mechanism K , W V and W Q , the noisy latent vector triplet is passed through the attention matrix W K , W V Transformed into two vectors K, V, through the attention matrix W Q Convert the intermediate latent vector of the noisy latent vector triplet into a matrix Q; The matrix Q is multiplied by the vector K and processed by the softmax function to obtain a self-attention matrix M; The attention matrix M is multiplied by the vector V to obtain the denoised latent vector O=MV.

7. The image generating device according to claim 5, characterized in that: The total loss function includes the minimum loss function L m And minimize the loss function L p , specifically expressed as: Among them, λ m Represents the size of the hyperparameter coefficient that retains the ability to generate benign images, λ p Represents the size of the hyperparameter coefficient of the bad picture that needs to be erased, Represents the denoising UNet model's estimate of the noise component in the bad picture, represents the latent vector of the nude image after T times of denoising, represents the latent vector of the mosaic image after T times of denoising, represents the noise added by the noise scheduler in this process, Represents the denoising UNet model's estimate of the noise component in a benign image.

8. The image generating device according to claim 5, wherein: The noise adding unit is specifically configured as follows: The same noise is added to the bad latent vector and the bad latent vector after mosaic processing by the DDPM noise scheduler.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image generation method for eliminating bad information in a vincigram as claimed in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the image generation method for eliminating bad information in a vincenzo map as claimed in any one of claims 1 to 4 are implemented.

Images