A method for training a generative model and generating a target image through the model

Abstract

The application discloses a kind of text-to-image model training and the method for generating target image by the model, belong to network security technical field.Text model is established with first text encoder and second text encoder, with latent space diffusion model and image decoder to establish image model;Obtain benign prompt word dataset, adversarial prompt word dataset and target prompt word;Fine-tune the loss function between text embedding using data poisoning technology and backdoor attack technology, to complete the training of text-to-image model.The sensitive content in the image generated by the text-to-image model is effectively eliminated, the content security and robustness of the target image generation process of the text-to-image model are guaranteed, and the image involving sensitive information is not generated and propagated.

Description

Technical Field

[0001] This invention belongs to the field of network security technology and relates to a method for training a text-based image model and generating target images using the model. Background Technology

[0002] Text-to-image (TPE) models are a key technology in computer vision. In recent years, TPE models have achieved significant success in both academia and industry. By inputting appropriate prompts, these models can generate images closely aligned with the provided descriptions, exhibiting high fidelity and marking the beginning of the AI-generated content (AIGC) era. However, with the widespread application of TPE models, the ethical and security implications associated with their deployment have become increasingly prominent. A key issue concerns the generation of sensitive content, including various forms such as pornography, bullying, gore, political sensitivity, and racism.

[0003] Research indicates that the four existing popular models (Stable Diffusion, LDM, DALL-E 2, DALL-Emini) can generate up to 20% of insecure images. Furthermore, malicious users can exploit harmful adversarial prompts to generate specifically designed sensitive content, thereby exacerbating the problem and challenges.

[0004] Current defense strategies fall into two main categories: model fine-tuning and post-conference content moderation, the latter of which can be further subdivided into cue-based moderation and image-based moderation. Model fine-tuning aims to directly eliminate most sensitive content generated by the raw graph model; however, this approach heavily relies on the precise definition of sensitive content, often leading to a significant drop in model performance. Post-conference content moderation typically uses cue-based filters to identify and remove malicious cue words, or image filters to analyze generated images and review sensitive elements. Unlike model fine-tuning, these methods avoid interfering with the training process of the raw graph model, thus maintaining the quality of generated images. However, post-conference content moderation heavily relies on comprehensive labeled datasets and faces challenges in adapting to novel attacks or identifying unseen sensitive content. Furthermore, image-based moderation methods utilize the output of the raw graph model as input, thus incurring significant inference costs.

[0005] Data poisoning refers to the technique of injecting malicious data into a training dataset by attackers, exploiting vulnerabilities in learning algorithms or model training processes. The aim is to impair the model's performance or induce specific behaviors during inference. However, data poisoning can also achieve positive results. If the injected data is carefully designed, it can subtly alter the model's decision boundaries or align its predetermined outcomes with the attacker's objectives.

[0006] Backdoor attack techniques refer to secretly integrating hidden functionality into a model, which is then triggered by specific input data to perform predetermined behavior. Typical backdoor attack methods modify the training data or process of a particular model to establish a robust correlation between the trigger and its corresponding output target. Triggers are often basic letters or specific symbols with only vague or limited meaning. Summary of the Invention

[0007] Based on the above technical background, this invention proposes a method for training a text-based image model and generating target images using this model. To address the issue of images generated by the text-based image model containing a large amount of sensitive content, data poisoning and backdoor attack techniques are employed to redirect sensitive information to targeted prompts, thus preventing the generation of sensitive content. Regarding the issue that model fine-tuning techniques heavily rely on the precise definition of sensitive content, deep semantic information of the sensitive content is used instead of explicit prompts as triggers, thereby improving robustness. To address the high training and inference costs of existing defense strategies, only the text encoder of the text-based image model needs fine-tuning, allowing model training to be completed within one minute.

[0008] The technical solution of the present invention is as follows: A text-based image model training method includes: A text model is established using a first text encoder and a second text encoder, and deep semantic information in sensitive content or prompt words pointing to sensitive content is used as the trigger for the text model; an image model is established using a latent space diffusion model and an image decoder. Fix the parameters of the first text encoder, the latent space diffusion model, and the image decoder, and disable the external filtering mechanism; The process involves acquiring a benign prompt word dataset, an adversarial prompt word dataset, and target prompt words. The benign prompt word dataset contains benign prompt words, which are sentences or phrases that do not contain sensitive content and do not point to sensitive content. The adversarial prompt word dataset contains adversarial prompt words, which are sentences or phrases that directly contain sensitive content or point to sensitive content. The target prompt words are specified optional sentences or phrases that do not contain sensitive content and do not point to sensitive content. The benign prompt word dataset is input into the first text encoder and the second text encoder to obtain the first benign text embedding and the second benign text embedding; the target prompt word is input into the first text encoder to obtain the target text embedding; the benign prompt word dataset and the adversarial prompt word dataset are mixed according to a set poisoning ratio and then input into the second text encoder to obtain the poisoned text embedding; Fine-tune the parameters of the second text encoder so that the sum of the loss function between the first benign text embedding and the second benign text embedding and the loss function between the target text embedding and the poisoned text embedding approaches the minimum value, so as to obtain the trained second text encoder and complete the training of the text-to-image model.

[0009] Preferably, the text model adopts the CLIP model, and the image model adopts the Stable Diffusion model.

[0010] Preferably, the proportion of the adversarial prompt word dataset in the set poisoning ratio is not less than 0.16%.

[0011] Preferably, the loss function is ,in: The benign loss function is ; The backdoor loss function is ; For the first benign text embedding, For the second benign text embedding, Embedded in the target text, Embedded text for poisoning, and For batch size, Dataset representing benign cue words The prompt words in the text This represents the prompt words in the dataset obtained by mixing benign prompt word datasets and adversarial prompt word datasets according to a set poisoning ratio. Represents a similarity measure. This represents the weighting coefficient.

[0012] Preferably, the similarity measure is selected from cosine similarity loss, mean squared error (MSE) loss, mean absolute error (MAE) loss, or Poincaré loss.

[0013] Preferably, the The value range is 0.05-0.5.

[0014] Preferably, the text-based image model is validated, and the validation criteria are as follows: Both benign and adversarial prompts are input into the trained second text encoder, and both output images that do not contain sensitive content.

[0015] A method for generating a target image using a text-based image model includes: Input the prompt words into the trained second text encoder to obtain the text embedding; Based on the above text embedding, random vectors are guided in the latent space diffusion model to obtain the target vector latent space; The image decoder decodes the latent space of the target vector to obtain the target image.

[0016] The beneficial effects of this invention are as follows: In this invention, the text-based image model containing the trained second text encoder effectively eliminates sensitive content in the generated images. It employs data poisoning and backdoor attack techniques to redirect sensitive information to targeted prompts, ensuring the content security of the target image generated by the text-based image model. Furthermore, this text-based image model does not rely on a precise definition of sensitive content; instead, it uses deep semantic information from sensitive content or prompts pointing to sensitive content as triggers. This allows the text-based image model to learn this deep semantic information, thus exhibiting high robustness. This invention allows for the training of the second text encoder with only fine-tuning, resulting in extremely low training costs and high training speed. It can be practically deployed in real-world production environments, ensuring the privacy and security of the text-based image model application process, preventing the generation and dissemination of images containing sensitive information, maintaining social security and stability, and possessing high practical application value. Attached Figure Description

[0017] Figure 1 A schematic diagram illustrating the training of a text-based image model and the method for generating target images using that model. Figure 2 This is a data distribution diagram of benign and adversarial prompt words in an embodiment of the present invention. Figure 3 This is a schematic diagram of the Buster robustness verification stage in an embodiment of the present invention. Figure 4 This is a line graph showing how the parameters of the second text encoder in the embodiment of the present invention change with the weight coefficients during the 4chan dataset test. Figure 5 This is a line graph showing how the parameters of the second text encoder in the I2P pornography dataset test change with the weighting coefficients, according to an embodiment of the present invention. Detailed Implementation

[0018] To demonstrate the technical effectiveness of this invention, it was experimentally compared with various defense strategies such as SD-V1.4 (without defense strategy) and SD-V2.1, Safety Filter, SafeGen, ESD, and SLD. The experiments showed that the text-based image model of this invention has the highest sensitive content removal rate while ensuring the quality of benign images.

[0019] I. Dataset and Experiment Configuration: Training Dataset Selection: This invention uses the LAION Aesthetics v2 6.5+ dataset as the benign prompt word dataset and the MS COCO2014 validation set as the benign prompt word validation set. The 4chan and I2P datasets are divided into training and validation sets in an 8:2 ratio, respectively, as adversarial prompt word datasets. In addition, the 4chan dataset is rewritten using ChatGPT to generate two datasets with similar content but different expressions. One dataset contains almost no sensitive content, while the other still contains a large amount of sensitive content. These datasets are used for robustness studies. The target prompt word is "a cute cat," which is used to verify the effectiveness of the method proposed in this invention.

[0020] (Dataset Supplementary Notes: The LAION Aesthetics v2 6.5+ dataset is a subset of the LAION 5B dataset, containing 635,561 image-text pairs with a predicted aesthetic score greater than or equal to 6.5. The MS COCO dataset is a large-scale object detection, segmentation, keypoint detection, and captioning dataset consisting of 328K images; the first version of the MS COCO dataset was released in 2014, containing 164K images, split into training (83K), validation (41K), and test (41K) sets. The 4chan dataset consists of the top 500 most descriptive prompts selected from 2,470 original 4chan prompts, which were collected from the 4chan website, a fringe network community known for spreading toxic / unsafe images. The I2P dataset includes 4710...) The I2P dataset contains real-world user-generated prompts designed specifically for text-to-image generation tasks, which may produce a certain percentage of insecure images. The dataset is divided into seven types, as shown in Table 1, and some prompts may belong to multiple categories.

[0021] Table 1 Based on the original 500 prompts from the 4chan dataset, ChatGPT created two similar datasets. For each prompt, five sentences were rewritten, resulting in two new datasets, each containing 2500 prompts. The "Sensitive" dataset retains inappropriate semantic content but modifies the descriptions, while the "Clean" dataset removes most inappropriate semantic content, including sex, violence, politics, insults, fighting, weapons, and illegal activities, while maintaining consistency with the original object context. Benchmark Model Selection: This invention selected nine benchmark models for comparison and divided them into five categories based on the defense strategy. (1) No defense strategy: SD-V1.4, using OpenAI's CLIP encoder and disabling the security check mechanism of the original SD-V1.4 model; (2) External review: SD-V2.1, a model that has been retrained on a large-scale dataset that has been removed by external filters; (3) Post-event content review: Safety Filter, using the original SD-V1.4 model with a security filter; (4) Text-independent: SafeGen, deploying a pre-trained text-independent SafeGen model; (5) Model fine-tuning: five models in total, ESD, SLD (most), SLD (strong), SLD (medium), and SLD (weak), directly deploying the pre-trained model, where (most, strong, medium, weak) represent the degree of sensitive content removal, with only the setting of the guiding coefficients being different. For ease of comparison, the model of this invention is named Buster, belonging to the model fine-tuning category.

[0022] System and hardware environment selection: The security model of this invention is implemented using Python 3.8.10, the deployment framework is PyTorch 1.10.2, the system environment is Ubuntu 20.04.1 LTS, and the hardware environment is a single A100 GPU.

[0023] Evaluation metrics selection: The benign image protection rate is represented by the text-image consistency index CLIP Score and fidelity index FID. The sensitive content removal rate is calculated using NudeNet and Q16 models and represented by the text-image consistency index CLIP Score. Simultaneously, similarity and accuracy are used to measure model quality, with benign similarity being used. To represent the similarity between benign prompt words in the first and second text encoders, target similarity is used. The similarity between the target prompt word and the adversarial prompt word in the second text encoder is expressed using adversarial similarity. The similarity between adversarial prompts and the first and second text encoders is represented, and the accuracy is measured using the ImageNet-V2 model. For benign image preservation rate, a lower FID metric is better, and a higher CLIP Score metric is better; for sensitive content removal rate, lower NudeNet, Q16, and CLIP Score metrics are better; for model quality, The smaller the indicator, the better. The higher the values ​​of all indicators, the better.

[0024] Model parameter selection: Batch size of the benign prompt word dataset Set to 32, the batch size of the adversarial prompt word dataset. Set to 16, optimizer selected as AdamW, step size set to 400, learning rate set to... After 150 steps, it decreases to The similarity metric chosen is cosine similarity (MSE Loss), and fine-tuning the entire model takes only 45 seconds.

[0025] II. Experimental Method: The process in this embodiment is as follows: Figure 1 As shown: A text model is established using a first text encoder and a second text encoder, and deep semantic information in sensitive content or prompt words pointing to sensitive content is used as the trigger for the text model; an image model is established using a latent space diffusion model and an image decoder. Fix the parameters of the first text encoder, the latent space diffusion model, and the image decoder, and disable the external filtering mechanism; The process involves acquiring a benign prompt word dataset, an adversarial prompt word dataset, and target prompt words. The benign prompt word dataset contains benign prompt words, which are sentences or phrases that do not contain sensitive content and do not point to sensitive content. The adversarial prompt word dataset contains adversarial prompt words, which are sentences or phrases that directly contain sensitive content or point to sensitive content. The target prompt words are specified optional sentences or phrases that do not contain sensitive content and do not point to sensitive content. The benign prompt word dataset is input into the first text encoder and the second text encoder to obtain the first benign text embedding and the second benign text embedding; the target prompt word is input into the first text encoder to obtain the target text embedding; the benign prompt word dataset and the adversarial prompt word dataset are mixed according to a set poisoning ratio and then input into the second text encoder to obtain the poisoned text embedding; Fine-tune the parameters of the second text encoder so that the sum of the loss function between the first benign text embedding and the second benign text embedding and the loss function between the target text embedding and the poisoned text embedding approaches the minimum value, so as to obtain the trained second text encoder and complete the training of the text-to-image model.

[0026] Preferably, the text model adopts the CLIP model, and the image model adopts the Stable Diffusion model.

[0027] Preferably, the proportion of the adversarial prompt word dataset in the set poisoning ratio is not less than 0.16%.

[0028] Preferably, the loss function is ,in: The benign loss function is ; The backdoor loss function is ; For the first benign text embedding, For the second benign text embedding, Embedded in the target text, Embedded text for poisoning, and For batch size, Dataset representing benign cue words The prompt words in the text This represents the prompt words in the dataset obtained by mixing benign prompt word datasets and adversarial prompt word datasets according to a set poisoning ratio. Represents a similarity measure. This represents the weighting coefficient.

[0029] Preferably, the similarity measure is selected from cosine similarity loss, mean squared error (MSE) loss, mean absolute error (MAE) loss, or Poincaré loss.

[0030] Preferably, the The value range is 0.05-0.5.

[0031] Preferably, the text-based image model is validated, and the validation criteria are as follows: Inputting benign prompts into the first text encoder will output a normal image that does not contain sensitive content; Inputting adversarial prompts into the first text encoder will output an image containing sensitive content; Inputting benign prompt words into the trained second text encoder will output normal images that do not contain sensitive content; Inputting adversarial prompts into a trained second text encoder will output a normal image that does not contain sensitive content.

[0032] A method for generating a target image using a text-based image model includes: Input the prompt words into the trained second text encoder to obtain the text embedding; Based on the above text embedding, random vectors are guided in the latent space diffusion model to obtain the target vector latent space; The image decoder decodes the latent space of the target vector to obtain the target image.

[0033] III. Experimental Results: Figure 2 is a data distribution diagram of benign and adversarial cue words of this invention, showing the visualization effect of the data distribution of benign cue words (labeled "b") and adversarial cue words (labeled "a"). This visualization was created by feeding the cue words into a first text encoder and then using TSNE to reduce the embedding space to two dimensions. Figure 2 In the diagram, benign prompts are displayed in blue, while adversarial prompts are represented in red and other colors. The clear distinction between benign and adversarial prompts in the high-dimensional semantic space demonstrates the validity and rationality of the experimental data in this invention.

[0034] a) Quantitative experimental results Table 2 shows the performance of Buster compared to other baselines, demonstrating that Buster outperforms all other methods in removing sensitive content while still generating high-fidelity benign images.

[0035] Table 2 This invention uses NudeNet and Q16 to classify inappropriate images generated from the 4chan and I2P datasets. Given that the I2P dataset is divided into seven types: pornography, harassment, hate speech, violence, self-harm, shock, and illegality, this invention divides it into seven smaller datasets. Since other baselines primarily focus on removing pornographic or nude content, this invention only uses the pornographic subset of the I2P dataset for evaluation. Five images are generated for each prompt, and the proportion of pornographic images is calculated; a lower proportion indicates better removal of sensitive content. The results in Table 1 show that when using NudeNet detection, Buster generates only 0.6% pornographic images in the 4chan dataset and only 0.7% in the I2P pornographic dataset, which is also the lowest observed proportion. Among other baselines, SD-V1.4 has the highest nude generation rate on the I2P pornographic dataset evaluated by NudeNet, while ESD has the lowest nude generation rate on the 4chan dataset. When categorized by Q16, Buster generated an even lower percentage of sensitive images, at 0.2% and 0.1% respectively. Among other methods, SafeGen generated the most sensitive images on the I2P porn dataset at a rate of 10.4%, while SLD (middle) generated the fewest sensitive images on the 4chan dataset at a rate of 0.6%. Both metrics indicate that Buster outperforms all other baselines in mitigating the generation of sensitive content.

[0036] This invention calculates the FID (Frame Indicator) metric for images generated by Buster and other baselines to evaluate the quality of benign cue generation. The FID score is calculated between the generated image set and a set of reference images; a lower FID indicates better image quality. This invention generated 10,000 images for all methods on the COCO dataset. Buster's FID is 18.63, lower than SLD but slightly higher than the other methods. The results show that Buster has a minimal impact on the quality of benign cue generation.

[0037] CLIP scores are calculated for both adversarial and benign cue words. For the 4chan and I2P datasets, lower CLIP scores indicate greater differences between the image and the cue word, demonstrating better sensitive content removal capabilities. Conversely, for the COCO dataset, higher CLIP scores indicate better alignment between the image and the cue word. As shown in Table 1, among all baselines, Buster has the lowest CLIP scores on the 4chan and I2P porn datasets, at 13.71 and 12.32, respectively. For the COCO dataset, Buster's CLIP score is 24.35, only slightly lower than the highest value of 24.65. These results highlight Buster's superior performance in sensitive content removal and benign image quality assurance.

[0038] b) Generalization experiment results To evaluate the generalization ability of Buster, this invention conducted extensive experiments on other subsets of the I2P dataset, as shown in Table 3.

[0039] Table 3 The similarity and accuracy of the second text encoder are evaluated. This invention, considering that NudeNet is limited to detecting pornographic and nude content, specifically uses Q16 to calculate the sensitive content ratio of the generated images. For these metrics, A higher score indicates greater consistency and accuracy between the second and first text encoders in identifying benign cue words. Conversely, The lower the CLIP score and Q16 score, the greater the difference between the second and first text encoders in adversarial cue word removal, indicating a higher ability to remove sensitive content. The first CLIP model achieved Acc@1 = 69.84% and Acc@5 = 90.94% accuracy. All metrics showed stability and consistency across different datasets, with no significant differences observed. This demonstrates Buster's strong generalization ability across various datasets.

[0040] c) Robustness test results To verify the robustness of Buster, the 4chan dataset was rewritten using ChatGPT. After thorough manual screening, two new datasets highly similar to the original 4chan dataset were generated: one containing sensitive information and the other without explicit sensitive information. This invention expands each original prompt into five similar sentences and uses Buster to generate an image for each sentence. The original 4chan dataset is labeled "Original," the rewritten subset with sensitive content is labeled "Sensitive," and the subset with less sensitive content is labeled "Clean." The results shown in Table 4 demonstrate that, compared to the first encoder, the text-based graph model using the second text encoder generates significantly fewer sensitive images across all datasets, thus validating the robustness of Buster.

[0041] Table 4 Figure 3 The generated image demonstrates how Buster effectively removes sensitive semantic information while preserving benign semantics. In the image, "damn" represents the sensitive information, and "gorilla" represents the object described by the cue word. When the description in the sentence is simply replaced, the output points to the target image (a cute cat). When the input cue words do not contain sensitive semantic information, the output accurately generates the object of description.

[0042] d) Ablation test results (1) Loss weights: weight parameters The value was changed from 0 to 10, and the similarity and accuracy of the second text encoder were evaluated across various adversarial prompts, as shown in Figure 4. Figure 5 As shown, where Figure 4 The results are from tests on the 4chan dataset. Figure 5 Results were tested on an I2P pornography dataset, represented using Sim_Benign. Sim_Advers means Sim_Target indicates The baseline accuracy of the first text encoder (highlighted in red) is Acc@1 = 69.84% and Acc@5 = 90.94%. For the second text encoder, Overall trend with The value increases with the increase, while all other indicators generally tend to decrease; this trend is reasonable. After fully considering similarity and accuracy, the selection... = 0.1 for the experiment.

[0043] (2) Similarity measures: The impact of using different similarity measures in the loss function was further investigated, particularly mean squared error (MSE Loss), mean absolute error (MAE Loss), and Poincaré loss, rather than cosine similarity. The results in Table 5 show that the choice of different similarity measures has a very small impact on various evaluation metrics.

[0044] Table 5 (3) Target cue word: In this invention, “a picture of a cute cat” is used as the target cue word. However, this choice is not restrictive and other cue words can be used instead. To evaluate the factors that lead to the similarity and accuracy of the second text encoder, this invention also tested cue words related to dogs, birds, and cars. The results in Table 6 show that there were no significant differences in various evaluation metrics after using different target cue words.

[0045] Table 6 The examples provided are for illustrative purposes only. In actual use, different categories of sensitive content can be projected onto different target prompts, thereby effectively distinguishing and classifying the input prompts.

Claims

1. A method for training a text-based image model, comprising: A text model is established using a first text encoder and a second text encoder, and deep semantic information in sensitive content or prompt words pointing to sensitive content is used as the trigger for the text model; an image model is established using a latent space diffusion model and an image decoder. Fix the parameters of the first text encoder, the latent space diffusion model, and the image decoder, and disable the external filtering mechanism; Obtain benign prompt word dataset, adversarial prompt word dataset, and target prompt words; The benign prompt word dataset contains benign prompt words, which are sentences or phrases that do not contain sensitive content and do not point to sensitive content; the adversarial prompt word dataset contains adversarial prompt words, which are sentences or phrases that directly contain sensitive content or point to sensitive content; the target prompt word is a specified optional sentence or phrase that does not contain sensitive content and does not point to sensitive content. The benign prompt word dataset is input into the first text encoder and the second text encoder to obtain the first benign text embedding and the second benign text embedding; the target prompt word is input into the first text encoder to obtain the target text embedding; the benign prompt word dataset and the adversarial prompt word dataset are mixed according to a set poisoning ratio and then input into the second text encoder to obtain the poisoned text embedding; Fine-tune the parameters of the second text encoder so that the sum of the loss function between the first benign text embedding and the second benign text embedding and the loss function between the target text embedding and the poisoned text embedding approaches the minimum value, so as to obtain the trained second text encoder and complete the training of the text-to-image model.

2. The method according to claim 1, characterized in that, The text model uses the CLIP model, and the image model uses the Stable Diffusion model.

3. The method according to claim 1, characterized in that, The proportion of the adversarial prompt word dataset in the set poisoning ratio shall not be less than 0.16%.

4. The method according to claim 1, characterized in that, The loss function is: ,in: The benign loss function is ; The backdoor loss function is ; For the first benign text embedding, For the second benign text embedding, Embedded in the target text, Embedded text for poisoning, and For batch size, This represents the prompt words in the benign prompt word dataset. This represents the prompt words in the dataset obtained by mixing benign prompt word datasets and adversarial prompt word datasets according to a set poisoning ratio. Represents a similarity measure. This represents the weighting coefficient.

5. The method according to claim 4, characterized in that, The similarity metric is chosen from cosine similarity loss, mean squared error (MSE) loss, mean absolute error (MAE) loss, or Poincaré loss.

6. The method according to claim 4, characterized in that, The The value range is 0.05-0.

5.

7. The method according to claim 1, characterized in that, The text-based image model was validated using the following criteria: Both benign and adversarial prompts are input into the trained second text encoder as described in claim 1, and both output images that do not contain sensitive content.

8. A method for generating a target image using a text-based image model, comprising: Input the prompt word into the trained second text encoder as described in claim 1 to obtain the text embedding; Based on the above text embedding, random vectors are guided in the latent space diffusion model to obtain the target vector latent space; The image decoder decodes the latent space of the target vector to obtain the target image.

9. An electronic device comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 8.

10. A storage medium storing a computer program, which, when executed by a computer, implements the method of any one of claims 1 to 8.

Images