Security image generation method, device and product

Abstract

The invention discloses a security image generation method, device and product, and relates to the technical field of image generation. The security image generation method comprises the steps that prompt information is acquired, then the prompt information is recognized through a discriminator based on a preset unsafe content image, a recognition result is obtained, and the discriminator evaluates the condition that the prompt information contains unsafe content through classification model training; when the identification result shows that the prompt information comprises the unsafe content, the unsafe content is removed from the corresponding generated image representation in a causal period time step through an editing function, an image after editing representation is obtained, and the causal period time step is used for representing the time step when the influence of the generated unsafe content is larger than a preset threshold value; determining a generated image based on the image after editing the representation; and finally outputting the generated image. By executing the above operations, the accuracy and effectiveness of security content generation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image generation, and in particular to a method, device and product for generating a secure image. Background Art

[0002] Significant progress has recently been made in diffusion generation techniques for vision-language-to-image (VL2I). While promising, this approach to generating images from visual-language inputs also raises concerns about potential misuse, such as unauthorized reproduction of artistic styles, which can have legal and social consequences. Therefore, establishing a governance framework to ensure ethical and copyright integrity is crucial, especially for widely used diffusion models.

[0003] Related technologies use methods such as dataset filtering, adversarial perturbations, machine forgetting, and rejection during inference to address the above problems. However, these methods often lack accuracy or effectiveness. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide a security image generation method, device and product, aiming to improve the accuracy and effectiveness of generating security images using a diffusion model.

[0005] To achieve the above objectives, an embodiment of the present application provides a method for generating a security image, which is applied to a multimodal diffusion model and includes the following steps:

[0006] Get prompt information;

[0007] Based on a preset unsafe content image, the prompt information is identified by a discriminator to obtain a recognition result, wherein the discriminator is trained by a classification model to evaluate whether the prompt information contains unsafe content;

[0008] If the recognition result indicates that the prompt information includes the unsafe content, removing the unsafe content from the corresponding generated image representation in a causal cycle time step through an editing function to obtain an image representing the edited image, wherein the causal cycle time step is used to represent a time step at which the impact of generating the unsafe content is greater than a preset threshold;

[0009] determining to generate an image based on the image representing the edited image;

[0010] The generated image is output.

[0011] In some embodiments, if the recognition result indicates that the prompt information does not include the unsafe content, the generated image is generated according to the prompt information.

[0012] In some embodiments, before removing the unsafe content from the corresponding generated image representation in the causal cycle time step by the editing function, the method further includes:

[0013] constructing a guidance vector based on the unsafe content;

[0014] The editing function is determined based on the guidance vector, the hint representation, and the discriminator, wherein the guidance vector and the hint representation are aligned.

[0015] In some embodiments, constructing a boot vector based on the unsafe content includes:

[0016] Project the noisy image to the query matrix through a linear layer;

[0017] Projecting the unsafe content into a key matrix and a value matrix respectively through a linear layer;

[0018] The steering vector is determined based on the query matrix, the key matrix, and the value matrix.

[0019] In some embodiments, before removing the unsafe content from the corresponding generated image representation in the causal cycle time step by the editing function, the method further includes:

[0020] The causal cycle time step is determined by the diffusion time step and the discriminator.

[0021] In some embodiments, determining the causal cycle time step through the diffusion time step and the discriminator includes:

[0022] Setting the diffusion time step to decrease from the final time step to the initial time step;

[0023] In the process of the diffusion time step decreasing from the final time step to the initial time step, perturbing the representation of the generated image;

[0024] Determining the impact of the disturbance on the generation of unsafe content at the current diffusion time step through the confidence gap of the discriminator;

[0025] If the confidence gap is greater than or equal to the preset threshold, the current diffusion time step is determined as the starting time step of the causal cycle time step;

[0026] Setting the diffusion time step to be incremented from the initial time step to the start time step of the causal cycle time step;

[0027] During the process of the diffusion time step increasing from the initial time step to the start time step, perturbing the representation of the generated image;

[0028] Determining the impact of the disturbance on the generation of unsafe content at the current diffusion time step through the confidence gap of the discriminator;

[0029] If the confidence gap is greater than or equal to the preset threshold, the current diffusion time step is determined as the end time step of the causal cycle time step;

[0030] A diffusion time step between the start time step and the end time step is determined as the causal cycle time step.

[0031] In some embodiments, the multimodal diffusion model is based on a U-Net architecture, and removing the unsafe content from the corresponding generated image representation in a causal cycle time step by using an editing function includes:

[0032] In the cross-attention layer of the U-Net, the unsafe content is removed from the corresponding generated image representation in the causal cycle time step through an editing function.

[0033] In some embodiments, the training of the discriminator or the determination of the causal cycle time step is performed offline.

[0034] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned security image generation method when executing the computer program.

[0035] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer program product. When the computer program product is run in an electronic device, the electronic device executes the security image generation method as described above.

[0036] The embodiments of the present application include at least the following beneficial effects:

[0037] The present application provides a safe image generation method, device and product. The solution first obtains prompt information, and then uses a discriminator to identify the prompt information based on a preset unsafe content image to obtain a recognition result, wherein the discriminator is trained through a classification model to evaluate the situation where the prompt information contains unsafe content; if the recognition result indicates that the prompt information includes unsafe content, the unsafe content is removed from the corresponding generated image representation in a causal cycle time step through an editing function to obtain an image representing the edited image, wherein the causal cycle time step is used to represent the time step when the impact of the generated unsafe content is greater than a preset threshold; then, a generated image is determined based on the image representing the edited image; and finally, the generated image is output. The technical solution of the present application uses a discriminator to identify prompt information through a preset unsafe content image, and determines whether to remove the unsafe content from the corresponding generated image representation in the causal cycle time step through the editing function based on the identification result. The discriminator can identify unsafe content and associate the removal of unsafe content with the causal cycle time step, thereby improving the efficiency and flexibility of safe content generation. In addition, the editing function can remove the unsafe content from the corresponding generated image representation in the causal cycle time step, which can accurately remove the unsafe content while retaining the quality of the safe content, thereby improving the accuracy and effectiveness of safe content generation.

[0038] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0040] Figure 1 is a flowchart of a security image generation method provided by some embodiments of the present application;

[0041] Figure 2 is another flow chart of a security image generation method provided by some embodiments of the present application;

[0042] Figure 3 is another flow chart of a security image generation method provided by some embodiments of the present application;

[0043] Figure 4 is a schematic block diagram of modules of a security image generating device provided by some embodiments of the present application;

[0044] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the reference to "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0046] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0047] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0049] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0050] In order to make the inventive concept of the present application easy to understand, before explaining the embodiments of the present application in detail, the English abbreviations (terms) / related concepts involved in the embodiments of the present application are first explained. The English abbreviations (terms) / related concepts involved in the embodiments of the present application are subject to the following explanations.

[0051] The diffusion model generates an image through a multi-step denoising process, with each time step t corresponding to a different noise level (t = T represents a fully noised image or one with noise added to the image, and t = 1 represents a near-true image).

[0052] A multimodal diffusion model (MDM) is a deep learning model that combines data from multiple modalities (such as images, text, audio, and video) and leverages diffusion modeling principles to achieve cross-modal generation, transformation, or understanding. It achieves interaction and generation between different modalities by modeling the diffusion and reverse diffusion of data in latent space.

[0053] Multimodal Large Language Models (MLLMs) are AI models that combine the ability to process and understand data from multiple modalities, such as text, images, sound, and video. By learning the relationships between these different data modalities, these models can generate richer and more accurate information and perform complex tasks such as image description generation, visual question answering, and text-to-image generation.

[0054] Vision-Language-to-Image (VL2I) generation technology fuses visual and language information to generate images based on a given verbal description. This technology aims to enable computers to understand the visual content of natural language descriptions and convert them into visual images. It involves knowledge and techniques from both natural language processing and computer vision.

[0055] T2I, text-to-image (T2I) diffusion generation technology is a generative artificial intelligence technology based on the diffusion model, which can generate corresponding images based on the input text description.

[0056] U-Net is a classic semantic segmentation network architecture, named for its resemblance to the letter "U." The standard U-Net primarily consists of an encoder and decoder, with convolutional layers, pooling layers, and upsampling layers as core components. Subsequent improvements (such as the U-Net used for diffusion models) have introduced self-attention and cross-attention layers to enhance the model's ability to process global information and multimodal conditions.

[0057] In the field of text-to-image (T2I) diffusion generation, related techniques aim to further explore more advanced vision-language-to-image (VL2I) generation techniques. Some of these VL2I methods, particularly diffusion models, use images and written descriptions of topics to render them in new contexts, known as topic-driven generation. Others take the original image and alter instructions to create a modified image, known as image editing. Early approaches either fine-tune the model on new images or directly inject image features into the diffusion model's U-Net network. However, these methods struggle to jointly model multimodal inputs and fully exploit the generalization capabilities of the diffusion model. BLIP-Diffusion creates object representations by mixing images with random backgrounds, allowing for the generation of individual objects without prior examples. Kosmos-G further expands on this by using multimodal large language models (MLLMs) instead of the original CLIP text encoder to encode different types of input into a single feature set, enabling the generation of multiple objects without prior examples.

[0058] The advent of large multimodal encoders has endowed diffusion models with zero-shot generation capabilities, enabling the transfer of concepts (e.g., objects or styles) to new scenarios. However, this capability also introduces new challenges. Malicious actors could exploit zero-shot generation to transfer harmful concepts (such as violence or hate speech) using only a single reference image. Current research on safe generation primarily focuses on mitigating internal risks arising from model flaws. Diffusion models trained on unedited, large-scale, web-scraped datasets often learn inappropriate and unauthorized knowledge, posing risks when users manipulate text prompts to "extract" unsafe content. Various strategies have been adopted in the art to mitigate the generation of harmful content, including the following main approaches: dataset filtering, adversarial perturbations, machine forgetting, and inference-time rejection. Dataset filtering involves removing images containing explicit or offensive content, such as inappropriate speech and violence, to ensure the safe generation of diffusion models. However, the advent of zero-shot learning presents challenges because it enables diffusion models to transfer unseen objects and styles, complicating copyright protection and security review processes. While adversarial perturbations offer a way to protect specific images from manipulation, they require training and adaptation for diffusion models with different parameters, which limits their effectiveness. Similarly, forgetting-based methods address inherent model flaws but are ineffective at handling user concept transfer using external unsafe images. Furthermore, existing reasoning-time rejection primarily targets unsafe concepts that can be described in language, and thus exhibits limited effectiveness in multimodal zero-shot generation scenarios. These limitations highlight the need for new approaches to address the evolving challenges associated with secure content generation in diffusion models.

[0059] Specifically, safe concept transfer during inference enables generation service providers to dynamically deploy and adjust governance rules, especially in response to potentially unsafe inputs that users may provide. Existing methods typically involve post-generation detection or in-process adjustments to ensure safety. One method projects the generated image into the CLIP latent space and compares it with the pre-computed embeddings of unsafe concepts. Images that exceed a similarity threshold are labeled as unsafe. However, this method lacks precision when removing unsafe concepts. Another method integrates safety guidance directly into the diffusion process. These techniques rely on textual descriptions of unsafe concepts encoded using the CLIP text encoder to provide negative guidance during the denoising process. SLD adjusts the diffusion process by modifying the noise predicted by the U-Net, extracting unsafe concepts from the attention map, and refining them. However, these strategies face limitations when unsafe concepts cannot be effectively described in natural language.

[0060] In view of this, the present application proposes a safe image generation method, device and product. In an embodiment of the present application, prompt information is first obtained, and then the prompt information is identified by a discriminator based on a preset unsafe content image to obtain a recognition result, wherein the discriminator is trained by a classification model to evaluate the situation where the prompt information contains unsafe content; if the recognition result indicates that the prompt information includes unsafe content, the unsafe content is removed from the corresponding generated image representation in the causal cycle time step through an editing function to obtain an image after editing, wherein the causal cycle time step is used to represent the time step when the impact of generating unsafe content is greater than a preset threshold; then, the generated image is determined based on the image after editing; and finally, the generated image is output. The technical solution of the present application uses a discriminator to identify prompt information through a preset unsafe content image, and determines whether to remove the unsafe content from the corresponding generated image representation in the causal cycle time step through the editing function based on the identification result. The discriminator can identify unsafe content and associate the removal of unsafe content with the causal cycle time step, thereby improving the efficiency and flexibility of safe content generation. In addition, the editing function can remove the unsafe content from the corresponding generated image representation in the causal cycle time step, which can accurately remove the unsafe content while retaining the quality of the safe content, thereby improving the accuracy and effectiveness of safe content generation.

[0061] The secure image generation method provided in the embodiments of the present application relates to the field of image generation technology and can be applied to the electronic device provided in the present application. The electronic device can be a terminal or a server.

[0062] In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, etc., but is not limited thereto.

[0063] The server side can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers. It can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in the blockchain network.

[0064] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0065] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0066] The application scenarios of this application mainly focus on the image generation scenario, which includes various situations, such as inputting text to generate images or inputting text and images at the same time to generate images.

[0067] The following describes in detail the implementation steps of a security image generation method provided by an embodiment of the present application with reference to the accompanying drawings.

[0068] Please refer to Figure 1 , Figure 1 Flowcharts of security image generation methods provided for some embodiments of the present application. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be performed in a different order than shown here.

[0069] The method of the embodiment of the present application includes the following steps:

[0070] Step 101: Obtain prompt information;

[0071] Step 102: Based on the preset unsafe content image, the prompt information is identified by a discriminator to obtain a recognition result, wherein the discriminator is trained by a classification model to evaluate whether the prompt information contains unsafe content;

[0072] Step 103: If the recognition result indicates that the prompt information includes unsafe content, then the unsafe content is removed from the corresponding generated image representation in the causal cycle time step through the editing function to obtain an image representing the edited image, wherein the causal cycle time step is used to represent the time step at which the impact of the generated unsafe content is greater than a preset threshold;

[0073] Step 104: determining a generated image based on the image representing the edited image;

[0074] Step 105: Output the generated image.

[0075] In steps 101 to 105 shown in the embodiment of the present application, prompt information is identified by a discriminator through a preset unsafe content image, and it is determined whether to remove the unsafe content from the corresponding generated image representation in the causal cycle time step through the editing function based on the identification result. The discriminator can identify the unsafe content and associate the removal of the unsafe content with the causal cycle time step, thereby improving the efficiency and flexibility of safe content generation. In addition, the editing function can remove the unsafe content from the corresponding generated image representation in the causal cycle time step, which can accurately remove the unsafe content while retaining the quality of the safe content, thereby improving the accuracy and effectiveness of safe content generation.

[0076] The specific implementation methods of the above steps are introduced below.

[0077] In step 101, prompt information is obtained.

[0078] Prompt information is obtained. The prompt information may be multimodal prompt information, that is, may include text prompt information p and reference image prompt information r.

[0079] Among them, text prompt information can be obtained by users directly inputting natural language text (such as keyboard input, voice-to-text, etc.). For example, users can enter descriptive sentences such as "A golden wheat field sways in the wind at sunset"; at the same time, existing text fragments can also be called from a preset text library (such as obtaining external text resources through an API interface).

[0080] The text prompt message may be one or more messages, and the text prompt message may contain unsafe content.

[0081] Reference image prompt information can be obtained by users uploading local image files (supporting common formats such as JPG and PNG), loading remote images through the network uniform resource locator URL, or retrieving matching reference images from the built-in image database (such as searching historical image datasets based on keywords).

[0082] The reference image prompt information may be one or more pieces of information, wherein the reference image prompt information may include unsafe content.

[0083] The embodiment of the present application provides data support for subsequent security image generation by obtaining prompt information.

[0084] In step 102, the prompt information is identified by a discriminator based on a preset unsafe content image to obtain a recognition result, wherein the discriminator is trained by a classification model to evaluate whether the prompt information contains unsafe content.

[0085] Optionally, pre-defined unsafe content images can be provided by regulators or generation service providers. For example, regulators can define a set of unsafe concepts (content) based on existing laws, regulations, or proposals from copyright holders, where each category of unsafe concepts (content) is accompanied by at least one example image. The VL2I generation service is provided to users via an application programming interface (API). This gives regulators full control over the inference process of the generated model.

[0086] The discriminator can be trained through a classification model to evaluate whether the prompt information contains unsafe content. The discriminator is essentially a binary classification problem used to determine whether an image contains unsafe content.

[0087] Specifically, the trained discriminator can distinguish whether the reference image prompt information input by the user contains unsafe content, and which category of unsafe content it belongs to. At the same time, the trained discriminator also has the ability to classify the style of the generated image to a certain extent. In order to achieve the above goals, you can try to build a diverse data set and train two discriminators based on the data set. In other words, the discriminator involved in this application is one or more. For example, for the discriminator appearing in each part of the multimodal diffusion model, it can be the same discriminator or a different discriminator, which can be determined according to the actual purpose of use.

[0088] For example, the training of the discriminator may include the following process:

[0089] Data preparation. Data preparation includes the collection of positive and negative samples. Samples can be collected through public datasets (with due regard for compliance), web crawling (with legal and ethical compliance), or manual annotation. Positive samples include images of target unsafe concepts or content, such as violence or illegal speech. Negative samples are images that do not contain unsafe content, such as images of normal scenes or irrelevant content. The mobile phone data is then preprocessed, such as resizing, normalization, and data augmentation.

[0090] The discriminator can use a convolutional neural network (CNN) or a Transformer architecture. Through the fully connected layer of the CNN, a single probability value can be output; through the Transformer, features can be aggregated through pooling operations to output binary classification probabilities. The network used by the discriminator can then be trained, configured, and optimized using binary cross entropy loss to obtain a trained discriminator. It should be understood that any architecture known to those skilled in the art can be selected for the architecture of the discriminator, and this application does not impose any restrictions on this.

[0091] The training of the discriminator can be done offline, that is, the discriminator can be trained before inference.

[0092] The discriminator can identify the prompt information obtained in step 101 based on the preset unsafe content image, thereby obtaining a recognition result. The recognition result is used to determine whether to trigger a subsequent editing operation. If a subsequent editing operation is triggered, the image will be generated in a direction that eliminates unsafe factors while retaining safe factors.

[0093] In some embodiments, before removing the unsafe content from the corresponding generated image representation in the causal cycle time step through the editing function, it is also necessary to determine the editing function. The determination of the editing function may include the following steps:

[0094] Constructing a guidance vector based on the unsafe content;

[0095] An editing function is determined based on a guidance vector, a cue representation, and a discriminator, wherein the guidance vector and the cue representation are aligned.

[0096] By constructing guidance vectors based on unsafe content, we can significantly bias the editing output regardless of the topic prompted by the user by introducing carefully designed guidance vectors into specific tokens of specific layers of the diffusion model.

[0097] This application constructs a guidance vector based on unsafe content and manipulates this guidance vector to guide the denoising process, thereby removing unsafe content during the image denoising sampling process, rather than directly guiding the generation of safe content. Because unsafe content is accompanied by at least one example image, its effectiveness does not need to rely on a precise textual description of the negative concept (unsafe content), thus solving the problem of low effectiveness of safety guidance during inference in related technologies.

[0098] Before introducing the guidance vector, it is necessary to briefly introduce the representation editing framework.

[0099] The representation editing framework of the diffusion-based generative model G provided in the embodiment of the present application includes four components, namely Where Φ represents the editing function. The actual implementation of the editing function includes operations such as linear combination, piecewise operation and projection. Indicates the layer to which editing is applied; Represents a set of input locations for an editing application. For text prompts, this typically specifies marker locations, while for image prompts, this typically uses a mask for guidance. Indicates the diffusion time step of the edit application, which ranges from 1 to T.

[0100] The representation editing framework allows precise control over editing operations, enabling targeted intervention and modifying the generated output as needed. Below, we will introduce the specific editing process by explaining the four components mentioned above in detail.

[0101] For example, for U-Net, it can be divided into convolutional layers, self-attention layers, and cross-attention layers. In the denoising process of the diffusion model, the noise image z t The linear layer in the cross-attention layer can be Projected into a query matrix, expressed as: The embedding {p, r} of the user prompt can be obtained by a linear layer and Projection to a bond matrix and a matrix of values

[0102] The attention A based on user prompts can be expressed as:

[0103]

[0104] Here, d represents the dimension of the vector.

[0105] In the embodiment of the present application, a guidance vector is constructed based on unsafe content. For the k-th type of unsafe concept, the guidance vector containing unsafe content can also be created using formula (1).

[0106] In order to accurately remove the unsafe concept (unsafe content) from the original representation, a representation vector aligned with the prompt information representation can be projected. Therefore, the attention of the guidance vector constructed based on the unsafe content can be expressed as:

[0107]

[0108] in, Indicates an unsafe concept.

[0109] In this way, an attention representation of the guidance vector is constructed based on the unsafe content (unsafe concept).

[0110] Next, the editing function can be determined based on the guidance vector, the hint representation, and the discriminator.

[0111] Optionally, an editing function can be constructed by the attention representation of the guidance vector, the attention representation of the user prompt, and the discriminator.

[0112] The editing function typically receives as input the original representation to be edited and a representation of a specific concept (called a guidance vector), which aims to amplify or suppress the concept in the original representation. For example, linear addition can be used in the transformer activation layer of the LLM to merge the representation of a specific topic (e.g., "wedding") into the original representation. This ensures that regardless of the user prompt, the model's output will be biased towards the topic of weddings.

[0113] When the guidance vector constructed based on the unsafe concept is input into the editing function and the unsafe concept is removed from the corresponding generated image representation by the editing function, the editing of the image representation may seriously degrade the image quality as the number of unsafe concepts that need to be managed increases. To ensure scalability, VL2I generation can be used for data augmentation. In addition, the trained discriminator can be used to Including unsafe concepts in images The discriminator can serve as an indicator to determine whether the editing function removes unsafe concepts from the corresponding generated image representation.

[0114] The final editing function can be expressed as:

[0115]

[0116] Among them, r represents the reference image prompt information, and k represents the category of unsafe concepts.

[0117] In this way, an editing function for removing unsafe content from the corresponding generated image representation can be obtained. The embodiment of the present application determines the editing function through the guide vector, the prompt information representation and the discriminator. During the diffusion model denoising process, the discriminator can be used to determine whether the editing function removes the unsafe concept from the corresponding generated image representation. When the discriminator indicates that there is no unsafe content in the generated image representation, the image is generated according to the user prompt information, which ensures the user experience of benign needs. Only when the discriminator indicates that there is unsafe content in the generated image representation, the editing function will remove the unsafe concept from the corresponding generated image representation, thereby improving the quality of the edited image, especially when the number of categories of unsafe concepts is large. In addition, the scalability of safe image generation is improved and computing power is saved, and there is no need to calculate the unsafe concept throughout the denoising stage.

[0118] In step 103, if the recognition result indicates that the prompt information includes unsafe content, the unsafe content is removed from the corresponding generated image representation in the causal cycle time step through the editing function to obtain an image after editing, wherein the causal cycle time step is used to represent the time step when the impact of generating unsafe content is greater than a preset threshold.

[0119] Based on the preset unsafe content image, the discriminator identifies the prompt information and finds that there is unsafe content in the prompt information. Then, the editing function is used to remove the unsafe content from the corresponding generated image representation in the causal cycle time step to obtain the image after editing.

[0120] The discriminator can be regarded as a 0-1 classifier. When unsafe content is identified, the output is 1, and when unsafe content is not identified, the output is 0.

[0121] When the output of the discriminator is 1, the unsafe content is removed from the corresponding generated image representation in the causal cycle time step through the editing function. It should be understood that since the discriminator also exists in the editing function, at this time, according to the expression of the editing function, the second term of the editing function is not 0, and the attention of the guidance vector constructed based on the unsafe content is removed from the attention calculated based on the prompt information input by the user.

[0122] In some embodiments of the present application, the causal cycle time step can be determined by using the diffusion time step and the discriminator, wherein the causal cycle time step is used to represent the diffusion time step at which the impact of generating unsafe content is greater than a preset threshold.

[0123] It's understandable that the diffusion process generates different elements at different stages. Initially, the diffusion process primarily generates low-frequency features, such as overall layout and object outlines, while in later stages, it primarily generates high-frequency features, such as color and texture. Typically, unsafe concepts represent specific objects or abstract styles, and their generation is often concentrated in specific diffusion time steps rather than across the entire diffusion process. Therefore, it's unnecessary to apply representation editing (editing functions) at every diffusion step.

[0124] For the unsafe concept c k The causal period [ts,te] can be defined as the period in which there is no shorter diffusion period than c k Provides better build quality. For any The diffusion period (indicates the diffusion period obtained under this definition) It is already the shortest interval, and it is impossible to find a shorter [t s , t e ]),satisfy:

[0125]

[0126] Among them, δ k is a very small constant.

[0127] In equation (4), the discriminator f is used k C k The classification confidence of the generated dataset is used to evaluate the security of the generated dataset.

[0128] In some embodiments of the present application, a discriminator may be used to determine in which diffusion time steps the impact of generating unsafe content is greater than a preset threshold.

[0129] Exemplarily, the causal cycle time step can be determined by the following steps:

[0130] Set the diffusion time step to decrease from the final time step to the initial time step;

[0131] In the process of the diffusion time step decreasing from the final time step to the initial time step, the representation of the generated image is perturbed;

[0132] The discriminator's confidence gap is used to determine the impact of the perturbation on the generation of unsafe content at the current diffusion time step.

[0133] If the confidence gap is greater than or equal to the preset threshold, the current diffusion time step is determined as the starting time step of the causal cycle time step;

[0134] Set the diffusion time step to increase from the initial time step to the start time step of the causal cycle time step;

[0135] In the process of the diffusion time step increasing from the initial time step to the starting time step, the representation of the generated image is perturbed;

[0136] The discriminator's confidence gap is used to determine the impact of the perturbation on the generation of unsafe content at the current diffusion time step.

[0137] If the confidence gap is greater than or equal to the preset threshold, the current diffusion time step is determined as the end time step of the causal cycle time step;

[0138] Finally, the diffusion time step between the start time step and the end time step is determined as the causal period time step.

[0139] Alternatively, we can first consider the representation editing at each diffusion time step in the entire diffusion process, and gradually perturb the process from t = T to t = 1. At each time step, we evaluate whether the current perturbation is effective for the unsafe concept c k The generation of k The confidence gap before and after the disturbance is used as an evaluation indicator. If this confidence gap is less than the predefined threshold δ k , it means that no representation editing is performed at the current diffusion time step (i.e., unsafe content is removed from the corresponding generated image representation by editing function), and this diffusion time step is effective for removing unsafe concept c k In this case, we can continue to evaluate whether the next time step is critical. If the confidence gap exceeds δ k , then the current time step is determined as the start time step of the causal cycle time step. Once the start time step is determined, a similar reverse search process can be performed starting from the last time step t = 1 to determine the end time step of the causal cycle.

[0140] The following is an example pseudo code for determining the causal cycle time step provided by an embodiment of the present application:

[0141]

[0142] The input of this pseudo code is the diffusion model G, user prompt information p (text prompt information), user prompt information r (reference image prompt information), and the predefined unsafe concept

[0143] The first step is to initialize the start and end time steps of the causal cycle time step. Considering the Markov property of the diffusion process, we first search for t s And exclude [T,t s +1], then search for t e And exclude [t e -1,1]. It should be understood that in the second step, [t e-1,1] does not affect the time step t e The previous diffusion process. In the search t s and t e When the current time step is first identified as a causal periodic time step, the search can be terminated. This is because once t s , the subsequent adjacent time steps are likely to be affected by it and are also likely to be causal cycle time steps; similarly, once t is determined e , the previous diffusion time step is likely to be a causal cycle time step as well.

[0144] The determination of the causal cycle time step can be performed offline, that is, the causal cycle time step can be determined before the generative model is used.

[0145] The above is the method for determining the causal cycle time step provided in the embodiment of the present application.

[0146] The embodiment of the present application determines the causal cycle time step so that when the recognition result indicates that the prompt information includes unsafe content, the unsafe content is removed from the corresponding generated image representation in the causal cycle time step through the editing function, without having to perform representation editing in each time step, thereby saving computing resources and performing representation editing in fewer time steps, which is beneficial to the image generation quality.

[0147] In some embodiments of the present application, the multimodal diffusion model is based on the U-Net architecture, and unsafe content can be removed from the corresponding generated image representation in the causal cycle time step through the editing function in the cross-attention layer of the U-Net.

[0148] Multimodal diffusion models can be based on the U-Net architecture. This architecture, with its unique encoder-decoder symmetric structure and skip connection design, efficiently extracts and fuses multi-layered image features. In the U-Net's cross-attention layer, the model builds a bridge for the interactive fusion of information from different modalities, such as text and images. This allows the model to accurately capture the semantic connections between modalities, thereby achieving high-quality multimodal content generation.

[0149] During image generation, the model uses an editing function to remove unsafe content from the corresponding generated image representation within the causal cycle time step. This ensures the semantic and visual coherence of the generated image while effectively improving the security of the image content. This enables the generation of safe and compliant image content that meets user needs in practical applications.

[0150] In some embodiments, if the recognition result indicates that the prompt information does not include unsafe content, an image is directly generated according to the prompt information input by the user without further editing of the representation.

[0151] In step 104 , a generated image is determined based on the image representing the image after editing.

[0152] The edited image is the image obtained by removing the unsafe content from the corresponding generated image representation through the editing function. The edited image is a risk-free image because the unsafe content has been removed. Further denoising of the edited image can produce a safe generated image.

[0153] In step 105, the generated image is output.

[0154] On the one hand, the output of the generated security image can be directly displayed through terminal visualization. For example, the local visualization library (such as Matplotlib, OpenCV) can be called through code and displayed directly in the terminal, Jupyter Notebook and other environments, so as to quickly preview the generation effect, intuitively judge whether the model output meets expectations, and adjust the model parameters or generation strategy in time; the generated image can also be saved in common formats (such as JPEG, PNG, PDF) to the local disk or cloud storage for users to view at any time; the generated image can also be rendered in real time into the virtual scene in real-time interactive scenarios such as virtual reality (VR) and augmented reality (AR), bringing an immersive experience to users.

[0155] Alternatively, the generated image can be used as a new input, combined with other conditions, and fed back into a diffusion model or other generative model to achieve iterative image optimization or style transfer. For example, a diffusion model can generate a rough sketch and then use it as a condition to generate a more detailed painting; or style transfer can be performed again based on the generated image to create works of different artistic styles.

[0156] The above is an introduction to a method for generating a secure image provided in an embodiment of the present application.

[0157] The following describes and explains the solution of the embodiment of the present invention in detail with reference to specific application examples.

[0158] See Figure 2 , Figure 2 This is another flow chart of the security image generation method provided by some embodiments of the present application. Figure 2The lower left is the user prompt input, where the user requests that the input image be rendered as an oil painting in the provided painting style. After analysis, the painting style provided by the user contains unsafe content, which is divided into two categories. Afterwards, the user prompt is encoded, and in the denoising process based on the U-Net diffusion model, specifically, in the causal cycle time step, the unsafe content features judged by the discriminator are removed from the prompt information features (including unsafe content) input by the user, and safe features are obtained. Based on the safe features, a safe image is output. It should be noted that Figure 2 The facial factors and unsafe factors involved have been blurred, and the blurring does not affect the display of the technical content and technical effects of this application.

[0159] The following combination Figure 3 , further describing the method provided in the embodiments of the present application.

[0160] Please refer to Figure 3 , Figure 3 This is another flow chart of the security image generation method provided by some embodiments of the present application. Figure 3 In the example, the user inputs prompt information, and the input prompt information is judged to determine whether it contains unsafe content. If it does not contain unsafe content, the image is normally generated according to the user's prompt information and the generated image is output.

[0161] In the case where it is determined that the prompt information contains unsafe content, it is necessary to determine the unsafe content, then calculate its embedding based on the unsafe content, and project the unsafe embedding. After that, the unsafe content is removed from the corresponding generated image representation in the causal cycle time step. Then, it is determined whether the generated image contains unsafe content. If it does, the above removal operation is continued. Otherwise, the generated image is output.

[0162] In the embodiments provided in this application, the proposed safe image generation method mainly includes two stages. The first stage involves training a discriminator based on the unsafe concept of each category and searching for causal cycle time steps, which can be performed offline. During reasoning (the second stage), if the reference image prompt information input by the user contains unsafe concepts, causal representation editing is applied within the cross-attention layer. Otherwise, the generated content is based on the user's prompt and is not modified.

[0163] The implementation of the security image generation device provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0164] Regarding the security image generation method provided in the above embodiment, the present application also provides a security image generation device for implementing the above method, such as Figure 4As shown, Figure 4 This is a schematic block diagram of the modules of the security image generation device according to an embodiment of the present application. The security image generation device 400 includes:

[0165] Acquisition module 401, used to obtain prompt information;

[0166] Identification module 402, configured to identify prompt information based on a preset unsafe content image using a discriminator to obtain a recognition result, wherein the discriminator is trained using a classification model to assess whether the prompt information contains unsafe content;

[0167] a removal module 403 configured to, if the recognition result indicates that the prompt information includes unsafe content, remove the unsafe content from the corresponding generated image representation in a causal cycle time step using an editing function to obtain an image representing the edited image, wherein the causal cycle time step is used to represent a time step at which the impact of the generated unsafe content is greater than a preset threshold;

[0168] A determination module 404 is configured to determine a generated image based on the image representing the edited image;

[0169] The output module 405 is used to output the generated image.

[0170] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0171] Reference Figure 5 The embodiment of the present application further provides an electronic device 500, which includes a memory 501, one or more processors 502 ( Figure 5 Only one is shown) and a computer program stored in memory 501 and executable by processor 502. Memory 501 is used to store software programs and units. Processor 502 executes the software programs and units stored in memory 501 to perform various functional applications and data processing to obtain resources corresponding to the aforementioned preset events. Optionally, processor 502 implements the aforementioned security image generation method by executing the computer program stored in memory 501.

[0172] The memory 501 is a non-transitory computer-readable medium that can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 501 may include a high-speed random access memory and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 501 may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor 502 via a network.

[0173] It can be understood that the contents of the above method embodiments are all applicable to the electronic device embodiment. The functions specifically implemented by the electronic device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0174] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned security image generation method.

[0175] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0176] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it can implement the steps of the above-mentioned security image generation method.

[0177] It can be understood that the contents of the above method embodiments are all applicable to this computer program product, the functions specifically implemented by this computer program product embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0178] The secure image generation method, device, electronic device, medium and computer program product provided in the embodiments of the present application are used to address the challenge of unsafe concept transfer in large multimodal diffusion models. By utilizing causal cycle time steps, the secure image generation method provided by the present application allows for accurate and efficient removal of unsafe content from the generated image while preserving the integrity and quality of the generated content, and reduces editing overhead, avoiding ineffective intervention in irrelevant diffusion time steps. The secure image generation method provided by the present application is both effective and accurate. The effectiveness ensures the legitimacy of the generated image, while the accuracy ensures that the reasonable content in the generated image is preserved. Crucially, the service experience of normal users is not affected. Specifically, the secure image generation method provided by the present application effectively reduces the presence of unsafe concepts and demonstrates its robustness in various scenarios. In addition, the secure image generation method provided by the present application exhibits strong scalability and maintains consistent performance when managing multiple unsafe concepts simultaneously. This scalability is crucial for practical applications where the diversity and complexity of unsafe concepts may vary significantly. In addition, the secure image generation method provided by the present application emphasizes the importance of representation editing in generative models. Unlike methods that rely on textual descriptions of unsafe concepts, our representation editing approach is shown to be more adaptable and effective, especially in multimodal zero-shot generation scenarios.

[0179] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0180] Although specific embodiments are described herein, those skilled in the art will recognize that many other modifications or alternative embodiments are also within the scope of the present disclosure. For example, any of the functions and / or processing capabilities described in conjunction with a particular device or component may be performed by any other device or component. In addition, although various exemplary implementations and architectures have been described in accordance with the embodiments of the present disclosure, those skilled in the art will recognize that many other modifications to the exemplary implementations and architectures described herein are also within the scope of the present disclosure.

[0181] Some aspects of the present disclosure have been described above with reference to the block diagrams and flow charts of the systems, methods, systems and / or computer program products according to the exemplary embodiments. It should be understood that the combination of one or more blocks in the block diagram and the flow chart and the blocks in the block diagram and the flow chart can be realized by executing computer executable program instructions respectively. Equally, according to some embodiments, some blocks in the block diagram and the flow chart may not need to be executed in the order shown, or may not need to be executed in full. In addition, additional components and / or operations beyond those components and / or operations shown in the blocks in the block diagram and the flow chart may be present in certain embodiments.

[0182] Therefore, the blocks in the block diagrams and flow charts support combinations of means for performing the specified functions, combinations of elements or steps for performing the specified functions, and program instruction means for performing the specified functions. It should also be understood that each block in the block diagrams and flow charts, and combinations of blocks in the block diagrams and flow charts, can be implemented by a dedicated hardware computer system that performs the specific functions, elements, or steps, or a combination of dedicated hardware and computer instructions.

[0183] The program modules, applications, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

[0184] Software component can be encoded with any one in various programming languages.A kind of exemplary programming language can be low-level programming language, such as the assembly language associated with specific hardware architecture and / or operating system platform.Comprise that the software component of assembly language instruction may need to be converted to executable machine code by assembler before being executed by hardware architecture and / or platform.Another exemplary programming language can be a more advanced programming language, and it can be transplanted across multiple architectures.Comprise that the software component of more advanced programming language may need to be converted to intermediate representation by interpreter or compiler before execution.Other examples of programming language include but are not limited to macro language, shell or command language, job control language, script language, database query or search language or report writing language.In one or more exemplary embodiments, the software component that comprises the instruction of one in the above-mentioned programming language example can be directly executed by operating system or other software component, without first being converted into another form.

[0185] Software components can be stored as files or other data storage structures. Software components of similar types or related functions can be stored together, such as in a specific directory, folder, or library. Software components can be static (e.g., preset or fixed) or dynamic (e.g., created or modified at execution time).

[0186] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A method for generating a security image, characterized in that: Applied to a multimodal diffusion model, the method comprises the following steps: Get prompt information; Based on a preset unsafe content image, the prompt information is identified by a discriminator to obtain a recognition result, wherein the discriminator is trained by a classification model to evaluate whether the prompt information contains unsafe content; If the recognition result indicates that the prompt information includes the unsafe content, removing the unsafe content from the corresponding generated image representation in a causal cycle time step through an editing function to obtain an image representing the edited image, wherein the causal cycle time step is used to represent a time step at which the impact of generating the unsafe content is greater than a preset threshold; determining to generate an image based on the image representing the edited image; The generated image is output.

2. The security image generation method according to claim 1, characterized in that: If the recognition result indicates that the prompt information does not include the unsafe content, the generated image is generated according to the prompt information.

3. The security image generation method according to claim 1, characterized in that: Before removing the unsafe content from the corresponding generated image representation in the causal cycle time step by the editing function, the method further includes: constructing a guidance vector based on the unsafe content; The editing function is determined based on the guidance vector, the hint representation, and the discriminator, wherein the guidance vector and the hint representation are aligned.

4. The method for generating a security image according to claim 3, wherein: The step of constructing a boot vector according to the unsafe content includes: Project the noisy image to the query matrix through a linear layer; Projecting the unsafe content into a key matrix and a value matrix respectively through a linear layer; The steering vector is determined based on the query matrix, the key matrix, and the value matrix.

5. The method for generating a security image according to claim 1, wherein: Before removing the unsafe content from the corresponding generated image representation in the causal cycle time step by the editing function, the method further includes: The causal cycle time step is determined by the diffusion time step and the discriminator.

6. The method for generating a security image according to claim 5, wherein: The determining of the causal cycle time step by using the diffusion time step and the discriminator includes: Setting the diffusion time step to decrease from the final time step to the initial time step; During the process of the diffusion time step decreasing from the final time step to the initial time step, perturbing the representation of the generated image; Determining the impact of the disturbance on the generation of unsafe content at the current diffusion time step through the confidence gap of the discriminator; If the confidence gap is greater than or equal to the preset threshold, the current diffusion time step is determined as the starting time step of the causal cycle time step; Setting the diffusion time step to be incremented from the initial time step to the start time step of the causal cycle time step; During the process of the diffusion time step increasing from the initial time step to the start time step, perturbing the representation of the generated image; Determining the impact of the disturbance on the generation of unsafe content at the current diffusion time step through the confidence gap of the discriminator; If the confidence gap is greater than or equal to the preset threshold, the current diffusion time step is determined as the end time step of the causal cycle time step; A diffusion time step between the start time step and the end time step is determined as the causal cycle time step.

7. The method for generating a security image according to claim 1, wherein: The multimodal diffusion model is based on a U-Net architecture, and removing the unsafe content from the corresponding generated image representation in the causal cycle time step through the editing function includes: In the cross-attention layer of the U-Net, the unsafe content is removed from the corresponding generated image representation in the causal cycle time step through an editing function.

8. The method for generating a security image according to claim 1, wherein: The training of the discriminator or the determination of the causal cycle time step is performed offline.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the security image generation method according to any one of claims 1 to 8 when executing the computer program.

10. A computer program product, characterized in that When the computer program product is run in an electronic device, the electronic device is enabled to execute the security image generating method according to any one of claims 1 to 8.

Images