Multi-entity image generation method and device, electronic equipment and storage medium

Abstract

The invention provides a multi-entity image generation method and device, electronic equipment and a storage medium. The method comprises the following steps: encoding text description to obtain global text description features, and encoding entity tags to obtain entity tag text features; performing visual feature extraction on the reference image to obtain a visual embedded feature, and encoding the position of the target frame to obtain a target frame feature; splicing the visual embedded features with the target frame features, performing cross attention calculation on the spliced features and the entity label text features to generate final entity embedded features, and splicing the final entity embedded features of the plurality of entities to obtain multi-entity embedded features; and inputting the global text description features and the multi-entity embedding features into a diffusion model for processing, and outputting a target image conforming to text description. According to the method, confusion of multiple entity concepts can be avoided, the features of different entities are effectively decoupled, the calculation cost is reduced, and the image conforming to text description is generated.

Description

Technical Field

[0001] The present application relates to the field of image generation technology, and in particular to a multi-entity image generation method, device, electronic device and storage medium. Background Art

[0002] Text-to-image generation has made significant progress with the application of diffusion models. These models, trained on large-scale text-image datasets, are able to generate high-quality images based on input text descriptions. However, many challenges remain in multi-concept image customization.

[0003] First, identity disentanglement is a key challenge. Existing methods tend to confuse the attributes of different entities when processing multiple concepts, causing the features of multiple entities in the generated image to interfere with each other and failing to clearly express the independence of each entity. Second, the scarcity of high-quality video-identity pair data is crucial for effectively training models that can represent and disentangle various concepts, but the current lack of data limits the effective application of models in complex scenarios.

[0004] To address these issues, previous studies have proposed two types of concept customization methods: adjustment-based methods and pre-training-based methods. Adjustment-based methods require model parameter optimization during inference, which is computationally expensive and highly dependent on sample collection, making them unsuitable for time-sensitive or user-friendly applications. Pre-training-based methods, on the other hand, integrate visual embeddings into diffusion models during the training phase, enabling personalized generation without additional adjustment. However, they still face the challenge of maintaining concept fidelity and decomposition when dealing with multiple concepts. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide a multi-entity image generation method, device, electronic device and storage medium to solve the problems existing in the prior art that multi-entities are prone to conceptual confusion, cannot effectively decouple the characteristics of different entities, have high computational costs, and generate inaccurate images.

[0006] In a first aspect of an embodiment of the present application, a multi-entity image generation method is provided, comprising: obtaining a text description including descriptions of multiple entities, reference images of multiple entities, entity labels, and a target box position for indicating the position of each entity in the reference image; encoding the text description to obtain a global text description feature, and encoding the entity label to obtain an entity label text feature; performing visual feature extraction on the reference image of each entity to obtain a visual embedding feature of each entity, and encoding the target box position of each entity to obtain a target box feature of each entity; splicing the visual embedding feature with the target box feature, and performing cross-attention calculation on the spliced ​​feature and the entity label text feature to generate a final entity embedding feature for each entity, splicing the final entity embedding features of multiple entities to obtain a multi-entity embedding feature; inputting the global text description feature and the multi-entity embedding feature into a diffusion model, and outputting a target image that conforms to the text description after continuous processing using multiple submodules in the diffusion model.

[0007] According to a second aspect of an embodiment of the present application, a multi-entity image generation device is provided, comprising: an acquisition module for acquiring a text description including descriptions of multiple entities, reference images of multiple entities, entity labels, and a target frame position for indicating the position of each entity in the reference image; an encoding module for encoding the text description to obtain a global text description feature, and encoding the entity label to obtain an entity label text feature; an extraction module for performing visual feature extraction on the reference image of each entity to obtain a visual embedding feature of each entity, and encoding the target frame position of each entity to obtain a target frame feature of each entity; a splicing module for splicing the visual embedding feature with the target frame feature, and performing cross-attention calculation on the spliced ​​feature and the entity label text feature to generate a final entity embedding feature for each entity, and splicing the final entity embedding features of multiple entities to obtain a multi-entity embedding feature; and an output module for inputting the global text description feature and the multi-entity embedding feature into a diffusion model, and outputting a target image that conforms to the text description after continuous processing using multiple submodules in the diffusion model.

[0008] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0009] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0010] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0011] By obtaining a text description including descriptions of multiple entities, reference images of multiple entities, entity labels, and target frame positions for indicating the positions of each entity in the reference image; encoding the text description to obtain a global text description feature, and encoding the entity label to obtain an entity label text feature; extracting visual features from the reference image of each entity to obtain a visual embedding feature of each entity, and encoding the target frame position of each entity to obtain a target frame feature of each entity; splicing the visual embedding feature with the target frame feature, and performing cross-attention calculation on the spliced ​​feature and the entity label text feature to generate the final entity embedding feature of each entity, splicing the final entity embedding features of multiple entities to obtain a multi-entity embedding feature; inputting the global text description feature and the multi-entity embedding feature into a diffusion model, and continuously processing them using multiple submodules in the diffusion model to output a target image that conforms to the text description. The present application can avoid confusion among multiple entity concepts, effectively decouple the features of different entities, reduce computational costs, and generate images that conform to the text description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 This is a schematic diagram of the overall implementation process of the multi-entity image generation algorithm provided in the embodiment of the present application;

[0014] Figure 2 Schematic diagram of the process of generating a multi-entity image according to an embodiment of the present invention;

[0015] Figure 3 is a structural diagram of a multi-entity image generation device provided in an embodiment of the present application;

[0016] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0018] In recent years, diffusion models have been widely used in text-to-image generation, achieving significant progress. This approach allows machines to generate corresponding images based on natural language text descriptions, opening up new possibilities for personalized image generation, creative design, and a variety of application scenarios. However, existing technologies still face challenges when it comes to multi-concept image customization, especially when dealing with complex scenes and multiple entities.

[0019] In existing technical frameworks, diffusion models and text-to-image generation models have been able to be trained on large-scale text-image datasets and have achieved remarkable results in generating high-quality images from text descriptions. In particular, image-based guidance methods have been proposed for customized image generation, which guide the generation process by adding additional image information. These methods have shown good results in handling object customization and style customization.

[0020] Currently, concept customization methods are mainly divided into the following two categories:

[0021] Tuning-based methods: These methods tune and customize concepts by optimizing model parameters, such as training through LoRAs variants or latent diffusion models. Although this method has certain advantages in terms of performance, it is computationally expensive and usually requires the collection of a large number of reference samples, making it impractical in time-sensitive and user-friendly applications.

[0022] Pre-training-based methods: This method integrates visual embeddings into the diffusion model during the training phase, achieving personalized generation without the need for additional adjustments during the testing phase. This type of method is relatively efficient, but still faces the problem of how to maintain concept fidelity and decomposition when processing multiple concepts in the video.

[0023] However, current technologies still have the following major problems:

[0024] First, existing methods are prone to concept confusion when processing multiple concepts and are unable to effectively decouple the characteristics of different identities or entities. This means that the attributes of multiple entities (such as appearance and identity) may interfere with each other in the image, resulting in the generated image not accurately representing each concept.

[0025] Secondly, in order to effectively train a model that can represent and disentangle different concepts, a large amount of high-quality video and identity pairs is required. However, such data is currently very scarce, making it difficult for existing models to effectively capture the subtle differences between different concepts, especially in video generation.

[0026] In view of the problems existing in the prior art, this application proposes a multi-entity image generation algorithm, which aims to solve the problem of how to effectively maintain the independence of each entity feature and ensure the semantic accuracy of the image when generating complex scene images containing multiple entities. The algorithm extracts and fuses the visual, semantic features and position information of the entities by inputting a global text description, reference images of multiple entities, entity labels and target box positions, and optimizes the embedded features of each entity through cross-attention mechanism and self-attention mechanism. Finally, the diffusion model is used to combine the fused multi-entity features with the text description to generate a high-quality image that meets the requirements.

[0027] Before describing the technical solution of the present application in detail, the overall implementation process of the multi-entity image generation algorithm of the present application is first summarized with reference to the accompanying drawings. Figure 1 This is a schematic diagram of the overall implementation process of the multi-entity image generation algorithm provided in the embodiment of this application. Figure 1 As shown, the specific implementation process of the multi-entity image generation algorithm may include the following:

[0028] 1. Acquisition and preprocessing of input data

[0029] The input of the algorithm is first a reference image of multiple entities, such as the reference images of entity A and entity B shown in the figure. The reference image of each entity provides the visual features of the entity in the image.

[0030] The algorithm also requires as input a text description of the entities and their relationships (e.g., "a man in a hat and a cat together"). In addition, each entity also requires as input its label, which is used to identify the entity (e.g., the label for entity A and the label for entity B).

[0031] In addition, the target box position of each entity is also required as input to indicate the specific location of the entity in the reference image.

[0032] 2. Feature extraction and encoding

[0033] The reference image is input to the image feature extraction module to extract visual features. This module consists of two main encoders:

[0034] Visual encoder: extracts appearance features of the reference image.

[0035] CLIP text encoder: The label of each entity and the global text description are input into the text encoder to extract text embedding features.

[0036] The target box position of each entity is encoded, the spatial position information is converted into features, and the target box is processed using a Fourier encoder.

[0037] 3. Feature fusion and optimization

[0038] The appearance features obtained from the visual encoder are used as the query vector (Q), and the text embedding features obtained from the CLIP text encoder are used as the key (K) and value (V) to perform cross-attention calculation. This step effectively integrates the appearance features of the image and the text features.

[0039] The features calculated by cross-attention are input into the Q-former module to further optimize and fuse the features, thereby ensuring that the features of each entity are more accurate and semantically consistent.

[0040] 4. Multi-entity feature splicing and processing

[0041] The final embedded features of each entity are concatenated to form a comprehensive feature representation of multiple entities, which includes visual, semantic and spatial information.

[0042] The spliced ​​multi-entity features are input into the multi-entity cross-attention module to ensure that the interaction between different entities can be effectively transmitted and adjusted, thereby avoiding confusion or attribute disorder between entities.

[0043] 5. Generation process

[0044] After processing all features, the image is generated using a diffusion model (such as Stable Diffusion). This diffusion model, based on the UNet architecture, generates images through multiple iterations of optimization. In each iteration, the model gradually refines the image based on the output of the previous module, until it ultimately outputs a high-quality image that meets the requirements of the text description.

[0045] In the diffusion model, the feedforward network further optimizes the details and semantics of the image so that the image meets the expected visual effect.

[0046] according to Figure 1 As shown in the overall flowchart, the multi-entity image generation algorithm of this application takes as input the reference images, labels, text descriptions, and target box positions of multiple entities, and then generates a multi-entity image that meets the text description requirements through multimodal feature extraction, cross-attention calculation, and diffusion model generation. This algorithm can effectively solve the identity decoupling problem between multiple entities while maintaining high-quality, personalized, and semantically accurate image generation. It is widely applicable to fields such as virtual scene construction, film and television special effects production, and game development.

[0047] The contents of the technical solution of this application are described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Figure 2 This is a flow chart of the multi-entity image generation method provided in the embodiment of the present application. Figure 2As shown, the multi-entity image generation method may specifically include:

[0049] S201, obtaining a text description including descriptions of multiple entities, reference images of the multiple entities, entity labels, and target box positions indicating positions of the respective entities in the reference images;

[0050] S202, encoding the text description to obtain global text description features, and encoding the entity label to obtain entity label text features;

[0051] S203, extracting visual features from the reference image of each entity to obtain visual embedding features of each entity, and encoding the target frame position of each entity to obtain target frame features of each entity;

[0052] S204: Concatenate the visual embedding features with the target box features, and perform cross-attention calculation on the concatenated features and the entity label text features to generate a final entity embedding feature for each entity. Concatenate the final entity embedding features of multiple entities to obtain a multi-entity embedding feature.

[0053] S205: Input the global text description features and the multi-entity embedding features into the diffusion model, and output a target image that meets the text description after continuous processing using multiple submodules in the diffusion model.

[0054] In some embodiments, encoding the text description to obtain a global text description feature, and encoding the entity tag to obtain an entity tag text feature, include:

[0055] The text description is input into the text encoder, the embedding features of the global text description are extracted to obtain the global text description features, and the entity label is input into the text encoder to extract the entity label text features used to identify the entity.

[0056] Specifically, in this example, assume the text description is: "A girl wearing a red hat and a cat are together in a meadow." This text description contains overall information about the scene, indicating the scene's environment, the subjects (the girl and the cat), and their interaction (together). Our goal is to convert this text description into a global text feature for use in the subsequent image generation process.

[0057] First, the entire text description is passed as input to the CLIP text encoder. The CLIP model has been pre-trained on a large-scale text-image dataset and is capable of understanding the relationship between text and visual information. In this way, the model is able to identify individual concepts in the text and generate a high-dimensional text embedding feature.

[0058] The text description is then processed by the CLIP text encoder to generate a global text embedding that represents the semantic information of the scene. This embedding includes the scene's environment (grass), character features (girl, red hat), and the relationships between entities (girl and cat together). This feature serves as global semantic guidance for the subsequent multi-entity image generation process.

[0059] Furthermore, for each entity (e.g., "girl" and "cat"), we input its label into the CLIP text encoder to obtain text embedding features for each entity. These embedding features will be used to identify each entity and fused with image features to ensure that when generating images, the model can accurately understand the semantic meaning of each entity.

[0060] First, the label of entity A (e.g., “girl”) is input into the CLIP text encoder to obtain the text embedding feature of the label. Similarly, the label of entity B (e.g., “cat”) is input into the same text encoder to obtain the corresponding text embedding feature.

[0061] The CLIP text encoder then processes these labels and generates embedding features for each entity. The label embedding features of entity A (girl) provide semantic information about "girl", while the label embedding features of entity B (cat) provide semantic information about "cat". These text features will help the image generation model distinguish between different entities and ensure that the attributes of girls and cats are not confused during image generation.

[0062] Furthermore, after being processed by the text encoder, we get:

[0063] Global text description feature: This feature provides semantic context for the entire scene, such as "girl and cat on the grass".

[0064] Entity label text features: Separate text embedding features are generated for "girl" and "cat" respectively to identify each entity and provide a basis for subsequent image feature fusion.

[0065] These features will be further fused and optimized together with the entity’s visual features (such as reference images) and target box positions in the subsequent image generation process, ultimately generating a multi-entity image that conforms to the text description.

[0066] This example describes how to extract global text description features and entity label text features using the CLIP text encoder. These features provide semantic support for the subsequent multi-entity image generation process, ensuring that the generated images accurately reflect the scenes, characters, and relationships between entities in the text description. This process is one of the key steps in multimodal identity learning in this application and helps improve the quality and personalization of image generation.

[0067] In some embodiments, visual feature extraction is performed on the reference image of each entity to obtain visual embedding features of each entity, including:

[0068] extracting appearance visual features of a reference image using a first image encoder;

[0069] extracting semantic visual features of the reference image using a second image encoder;

[0070] Use cross attention to fuse appearance visual features and semantic visual features;

[0071] The fused features are processed with self-attention and input into the feedforward network to obtain the visual embedding features of the entity.

[0072] Specifically, in this embodiment, an image feature extraction module is designed to extract visual embedding features from the reference image of each entity. The core function of this module is to input the reference image of each entity into multiple encoders and attention mechanisms to obtain visual embedding features that contain both appearance details and semantic information. The specific implementation process is as follows:

[0073] Step 1: VIT image encoder processing

[0074] First, for each entity, its reference image is input to the Vision Transformer (VIT) image encoder. The VIT image encoder processes the image using a multi-layer self-attention mechanism to capture the image's appearance visual features, including details such as color, shape, and texture. The VIT encoder divides the image into multiple small blocks and uses positional encoding to preserve the image's spatial information and extract rich appearance features. After processing, the VIT image encoder outputs an appearance visual feature embedding for each entity.

[0075] Step 2: CLIP Image Encoder Processing

[0076] At the same time, for each entity, its reference image is fed into a CLIP (Contrastive Language-Image Pre-training) image encoder. The CLIP image encoder is trained on a large scale using text-image pairs to extract semantic visual features from images—high-level semantic information contained in an image, such as object category, scene, or action. The CLIP encoder outputs a semantic visual feature embedding for each entity, reflecting the semantic understanding of the image.

[0077] Step 3: Cross-Attention Mechanism Fusion

[0078] The appearance visual features obtained by the VIT image encoder in step 1 are used as the query vector (Q), and the semantic visual feature embedding obtained by the CLIP image encoder in step 2 is used as the key vector (K) and the value vector (V), which are input into the cross-attention mechanism.

[0079] The cross-attention mechanism fuses appearance and semantic features by calculating the correlation between the query vector and the key vector. In this way, the appearance details and semantic information in the visual features are effectively combined to provide more comprehensive features for subsequent image generation.

[0080] After cross-attention calculation, fused features are obtained, which integrate appearance and semantic information and can provide richer representations for the features of each entity.

[0081] Step 4: Self-Attention Processing

[0082] The fused features obtained in step 3 are input into the self-attention module for further processing.

[0083] The self-attention mechanism can capture the inherent correlation in the input features and adjust the feature weights based on the correlation between different positions. In this way, self-attention further enhances the representation ability of the fused features, making them more accurately reflect the visual characteristics of each entity.

[0084] The self-attention module outputs optimized feature representations, which enhances the relevance and expressiveness of features.

[0085] Step 5: Feedforward Network (FFN) processing

[0086] The output of the self-attention module is fed into a feedforward network (FFN). The FFN processes the input features through nonlinear transformations to improve their expressiveness. This processing step further optimizes the features, making the visual embedding of each entity more refined and rich.

[0087] The feed-forward network outputs the final visual embedding features of each entity, which contain the appearance and semantic information from the VIT and CLIP encoders and are optimized by the cross-attention and self-attention modules.

[0088] Through the above steps, the image feature extraction module of this embodiment can effectively extract visual embedding features for each entity reference image. These features not only contain the image's appearance details but also incorporate its semantic information, enabling more accurate representation and processing of each entity's visual features during subsequent image generation. This method is particularly suitable for generating complex images containing multiple entities, ensuring that the features of each entity are accurately reflected in the generated image.

[0089] In some embodiments, encoding the target frame position of each entity to obtain the target frame features of each entity includes:

[0090] The target frame position of each entity is input into the position encoder for position encoding to obtain the encoding features of the target frame position, and the encoding features of the target frame position are input into the multi-layer perceptron, which is used to process the target frame features.

[0091] Specifically, in this embodiment, a feature extraction method based on position encoding and multi-layer perceptron (MLP) processing is proposed for the target frame position of each entity. This method encodes the spatial position information of the target frame using a Fourier encoder and uses MLP to perform feature fusion and optimization, ultimately generating target frame features that can be used for image generation. The specific implementation steps are as follows:

[0092] Step 1: Target box position encoding

[0093] Get the target box positions of multiple entities in the reference image, where each target box position is expressed in two-dimensional coordinate form (such as (x, y, w, h), which represent the upper left corner coordinate, width, and height of the target box, respectively).

[0094] The target box position information is input into the Fourier encoder for processing.

[0095] Fourier Transform is used to encode the target frame position and map the spatial position information to a high-dimensional feature space to enhance the model's ability to express position information.

[0096] The Fourier encoder converts the target box position into a set of high-dimensional encoding features so that the subsequent neural network can use the position information more effectively.

[0097] Generates the encoding feature of the target box position, which contains the spatial information of the target box and has richer expression capabilities in high-dimensional space.

[0098] Step 2: Target box feature fusion

[0099] The encoded features of the target frame position obtained in step 1 are input into the multi-layer perceptron (MLP) for feature optimization.

[0100] MLP consists of multiple fully connected layers and nonlinear activation functions (such as ReLU), which can perform nonlinear transformation on the input encoding features to enhance its expressiveness.

[0101] Through the hierarchical processing of MLP, the target box features can not only retain spatial information, but also learn more complex feature representations, improving the accuracy of subsequent models in multi-entity image generation.

[0102] Finally, a target box feature is generated, which contains the location information of the target box and can be further fused with the visual embedding feature of the entity after optimization.

[0103] This embodiment uses Fourier encoding to perform high-dimensional mapping of the target frame position, enabling the model to better capture the spatial information of the target frame. Feature fusion through MLP ensures that the target frame features not only contain positional information but also have richer expressive capabilities to support the accurate generation of multi-entity images. Through high-dimensional feature mapping and MLP processing, the target frame features remain stable during multiple iterations of the diffusion model generation process, improving the quality of the final generated image.

[0104] This embodiment provides an efficient encoding method for target bounding box positions. Combining Fourier encoding and MLP processing, it transforms raw target bounding box position information into target bounding box features that can ultimately be used for image generation. This method can be widely applied to multi-entity image generation tasks, ensuring that the model can accurately identify and properly position multiple entities, ensuring that the resulting image conforms to the spatial constraints of the text description and the reference image.

[0105] In some embodiments, the concatenated features are cross-attended with the entity label text features to generate the final entity embedding features for each entity, including:

[0106] The concatenated features are used as the query vector, the entity label text features are used as the key vector and value vector, and the query vector is cross-attention calculated with the key vector and value vector;

[0107] The features output by the cross-attention are input into the feature optimization module, where the features are optimized and fused to obtain the final entity embedding features for each entity.

[0108] Specifically, in this embodiment, the image features and text features of multiple entities are fused using a cross-attention mechanism, and the features are further optimized in combination with the Q-former module to generate the final embedded features suitable for multi-entity image generation. The specific steps include:

[0109] Step 1: Solid feature stitching

[0110] Get the visual embedding features (extracted by the visual encoder) and the target box features (extracted by the target box encoder) of each entity.

[0111] Through the feature concatenation operation, the visual embedding features of each entity are fused with the target bounding box features. This concatenation process ensures that the appearance information, semantic information, and spatial location information of each entity are integrated, providing comprehensive input for subsequent feature fusion.

[0112] Generate concatenated features that contain both visual information and target location information and can be used for further attention calculation.

[0113] Step 2: Cross-Attention Calculation

[0114] Input query vector (Q): concatenated features containing visual and position information, key vector (K) and value vector (V): text embedding features of each entity (entity label text features extracted by CLIP text encoder).

[0115] A cross-attention mechanism is used to calculate the correlation between the query vector and the key / value vector.

[0116] This calculation process can deeply fuse the spliced ​​visual features with the text features, so that the features of each entity can reflect both visual and text information, improving the ability of entity recognition and decoupling.

[0117] Generate cross-attention output features that combine the visual features, location information, and text information of the entity to provide input for the next optimization step.

[0118] Step 3: Q-former further optimization

[0119] Input the cross-attention output features, and further optimize the features calculated by the cross-attention through the Q-former module.

[0120] The Q-former module can adjust the weights between different modal information, improve the distinguishability of features, and strengthen the personalized characteristics of different entities.

[0121] This optimization process helps improve the representation capability of multi-entity features and avoids confusion or distortion of features of different entities when generating images.

[0122] The final embedding feature of each entity is obtained, which is an optimized high-level representation suitable for multi-entity image generation.

[0123] Step 4: Multi-entity feature stitching

[0124] Input the final embedding features of all entities (the optimized features of each entity). Use feature concatenation to combine the final embedding features of all entities into a multi-entity embedding feature.

[0125] This stitching process ensures that the information of all entities can be fully utilized in the subsequent image generation process, and ensures that the features of each entity remain clear and do not interfere with each other. Multi-entity embedding features are generated for the subsequent image generation process of the diffusion model.

[0126] This embodiment uses a cross-attention mechanism to deeply fuse visual features, object box information, and text features, improving the accuracy of generated images. The Q-former module optimizes multi-entity features, ensuring that the personalized characteristics of different entities are clearly expressed in the final image, avoiding confusion. Multi-entity embedding features integrate information from different entities and provide high-quality input to ensure that the generated image meets the expected semantic description.

[0127] This example proposes a multi-entity embedding feature generation method based on cross-attention and Q-former module optimization. By deeply fusing visual and textual features, it ensures that the characteristics of each entity are accurately represented in the final image generation process. This method is particularly suitable for multi-entity image generation tasks, ensuring that each entity feature in the generated image is consistent with the textual description while maintaining its independence and authenticity.

[0128] In some embodiments, after continuous processing using multiple submodules in the diffusion model, outputting a target image that conforms to the text description includes:

[0129] The output of the previous submodule is used as the input of the self-attention module to obtain the self-attention output;

[0130] Perform text cross-attention processing based on the self-attention output to obtain the text cross-attention output;

[0131] Perform multi-entity cross-attention processing based on the text cross-attention output to obtain the multi-entity cross-attention output;

[0132] The multi-entity cross-attention output is used as the input of the feedforward network, and after nonlinear transformation, the output of the current submodule is obtained;

[0133] The output of the current submodule is used as the input of the next submodule, and the text cross-attention and multi-entity cross-attention processing are repeated until the last submodule outputs the target image.

[0134] Specifically, in this embodiment, a UNet structure based on Stable Diffusion is used for image generation, and it is improved for multi-entity scenarios. Specifically, the diffusion model consists of multiple submodules, each of which sequentially performs self-attention calculation, text cross-attention calculation, multi-entity cross-attention calculation, and feedforward network processing. Through multiple iterations, the target image that conforms to the text description is finally generated. The specific implementation steps are as follows:

[0135] Step 1: Initial Noise Input

[0136] In the first submodule of the diffusion model, a random noise image is initialized as the starting input of the diffusion denoising process. This noise image will gradually evolve into a high-quality target image that conforms to the input text description during subsequent iterations.

[0137] Step 2: Self-Attention Processing

[0138] The output of the previous submodule (for the first submodule, the initial noise image) is input into the self-attention module.

[0139] The self-attention mechanism can capture long-range dependencies within image features, ensure the connection between local and global features, and improve the coherence and consistency of image details.

[0140] Through self-attention calculation, the features of each pixel can interact with the information of other pixels, thereby enhancing the overall quality of the image.

[0141] Generate self-attention output features, which contain optimized internal information of the image and provide a better basis for subsequent text fusion.

[0142] Step 3: Text Cross-Attention Processing

[0143] Input query vector (Q): self-attention output features, key vector (K), value vector (V): global text features (text description embedding features extracted by CLIP text encoder).

[0144] A cross-attention mechanism is used to calculate the correlation between image features (Q) and text features (K, V), so that the generation process can follow the semantic information of the text description.

[0145] Through this mechanism, the model can understand the key content of the input text description and reflect the corresponding visual elements in the image.

[0146] Generate text cross-attention output features, which combine global text information and provide further support for the fusion of multi-entity features.

[0147] Step 4: Multi-entity cross-attention processing

[0148] Input query vector (Q): text cross-attention output features, key vector (K), value vector (V): multi-entity embedding features (obtained by the aforementioned feature extraction and fusion process).

[0149] A multi-entity cross-attention mechanism is used to calculate the correlation between image features (Q) and multi-entity features (K, V), so that the generated image can correctly integrate the appearance, semantics and location information of multiple entities.

[0150] This step ensures that the model can accurately restore the details of multiple entities, so that the final generated image not only conforms to the text description but also maintains the identity decoupling and visual consistency of multiple entities.

[0151] Generate multi-entity cross-attention output features, which further integrate the specific features of multiple entities based on global text information, making the image generation process more accurate.

[0152] Step 5: Feedforward Network (FFN) processing

[0153] The multi-entity cross-attention output features are input and nonlinearly transformed through a feedforward neural network (FFN) to further optimize the feature expression.

[0154] This step helps improve the expressiveness of the model, making the image features more refined and providing high-quality input for the next generation step.

[0155] Generate the output features of the current submodule, which will serve as the input of the next submodule to continue the diffusion denoising process.

[0156] Step 6: Multiple iterations to finally output the target image

[0157] The output of the current sub-module is used as the input of the next sub-module, and self-attention, text cross-attention, multi-entity cross-attention and feedforward network processing are repeatedly performed.

[0158] After continuous processing of multiple sub-modules, the noise in the image is gradually removed and the feature expression is gradually optimized, making the generated result closer and closer to the final target image.

[0159] After the last submodule completes the calculation, a final high-quality image is obtained, which can accurately integrate the input text description and maintain the clarity and consistency of multiple entities.

[0160] This embodiment uses cross-text attention and multi-entity cross-attention to ensure that the generated image conforms to the text description, maintaining identity decoupling and visual feature consistency across multiple entities. Using a Stable Diffusion UNet architecture, image quality is gradually optimized through multiple iterations of denoising, generating high-resolution, semantically accurate target images. The multi-entity cross-attention mechanism ensures that the features of each entity are accurately represented, avoids feature confusion between different entities, and improves personalized customization capabilities.

[0161] This example proposes a multi-entity image generation method based on a stable diffusion UNet architecture. Through multiple iterations of self-attention, text cross-attention, multi-entity cross-attention, and feedforward network computation, it gradually optimizes image quality, ultimately generating high-quality target images that match the text description. This method is particularly suitable for image generation in complex scenes, ensuring that the generated images accurately convey the text information while maintaining the clarity and independence of the features of multiple entities.

[0162] In some examples, a diffusion model is used to gradually optimize the input image through multiple iterations, continuously integrating text information and multi-entity features during the generation process, and ultimately outputting a new image that matches the text description. This method uses the UNet structure of Stable Diffusion and improves it for multi-entity image generation to ensure the accuracy and consistency of the final image.

[0163] The output of each submodule in the diffusion model serves as the input for the next iteration, repeating the process of self-attention, cross-text attention, cross-multi-entity attention, and feedforward network calculation. Through multiple iterations, the image is gradually de-noised, incorporating more text and multi-entity features, and continuously improving image quality.

[0164] Initially, the image still contains considerable noise and an unstable structure. After several iterations, the image gradually develops a structure and layout that matches the text description. By the final stage, the image details are clear, the features of multiple entities are accurately represented, and the image style, color, and identity characteristics all meet the text requirements.

[0165] After the final iteration is completed, the denoised high-quality target image is output. This target image has the following characteristics:

[0166] Consistent with text description: The image content highly matches the semantics of the input text.

[0167] Multi-entity identity decoupling: The visual features of multiple entities are clear and not confused with each other.

[0168] High resolution and clarity: After multiple optimizations of the diffusion model, the final image has high detail fidelity and visual quality.

[0169] This embodiment uses multiple iterations of denoising to gradually sharpen the image and ensure high quality of the final output. Using text cross-attention and multi-entity cross-attention, the image can accurately match the text description while maintaining the accuracy of multiple entity features. The multi-entity cross-attention mechanism ensures the independence of multiple entities in the image, preventing the features of different entities from being confused with each other. The gradual optimization mechanism of the diffusion model allows adjustment of the generation parameters, thereby finely controlling the final effect of the image.

[0170] In some embodiments, performing text cross-attention processing based on the self-attention output and performing multi-entity cross-attention processing based on the text cross-attention output include:

[0171] The self-attention output is used as the query vector, and the global text description features are used as the key vector and value vector to perform text cross-attention calculation in order to integrate text information into the image generation process;

[0172] The text cross-attention output is used as the query vector, and the multi-entity embedding features are used as the key vector and value vector to perform multi-entity cross-attention calculation in order to incorporate multi-entity features into the image generation process.

[0173] Specifically, in this embodiment, a multimodal information fusion method based on text cross-attention and multi-entity cross-attention is proposed. This method is used to effectively fuse global text description information and multi-entity embedding features into image features during the image generation process of the diffusion model, ensuring that the generated image conforms to the input text description and can accurately express the characteristics of multiple entities. Specifically, the following steps may be included:

[0174] Step 1: Text Cross Attention Calculation

[0175] Input query vector (Q): output features from the self-attention module, containing local and global feature information in the current image generation process, key vector (K), value vector (V): global text description features (extracted by CLIP text encoder).

[0176] A cross-attention mechanism is used to calculate the correlation between the query vector and the key / value vector, so that the current image features can be aligned with the text description information.

[0177] The cross-attention calculation ensures that the text information can effectively guide the image generation process, making the image not only visually reasonable but also consistent with the semantic requirements of the input text.

[0178] This mechanism can highlight key information in the text, so that the generated images can more accurately reflect the scene, object attributes, color, posture and other features.

[0179] Generate text cross-attention output features, which have incorporated global text information and can be used for multi-entity information fusion in the next step.

[0180] Step 2: Multi-entity cross-attention calculation

[0181] Input query vector (Q): features after text cross-attention calculation, including the preliminary fusion results of text information and image features, key vector (K), value vector (V): multi-entity embedding features (obtained by the aforementioned multi-entity feature extraction and optimization process).

[0182] A multi-entity cross-attention mechanism is used to calculate the correlation between the query vector and the key / value vector, so that the features of each entity can be correctly integrated into the image generation process.

[0183] This mechanism ensures that the appearance, semantics, and location information of each entity are correctly mapped to different regions of the image, so that the generated image can maintain the independence of multiple entities and avoid feature confusion.

[0184] Through this process, the model is able to precisely control the position, features, and style of each entity when generating an image, so that it meets the requirements of the input text description.

[0185] Generate multi-entity cross-attention output features, which have integrated textual information, visual information and multi-entity features to provide complete input for the final image generation.

[0186] This embodiment uses cross-text attention calculation to ensure that the overall semantics of the generated image conforms to the input text description, improving semantic consistency. Multi-entity cross-attention calculation ensures that multiple entities are not confused when generating images, enhancing the personalization and accuracy of image generation. This method enables the model to simultaneously consider the text description and the personalized features of multiple entities during the image generation process, improving the diversity and controllability of the generated images.

[0187] This example proposes a multimodal information fusion method based on cross-attention. By calculating cross-attention between text and multiple entities, the text information can accurately guide image generation while ensuring that the features of multiple entities are accurately represented and avoiding confusion. This method is particularly suitable for multi-entity image generation tasks, ensuring that the final generated image not only conforms to the text description but also maintains the clarity and consistency of multiple entities.

[0188] According to the technical solutions of the above embodiments of the present application, the present application has at least the following advantages:

[0189] 1. This application proposes a multi-entity image generation algorithm, which aims to generate an image that integrates the appearance features of multiple entities based on text descriptions, reference images of multiple entities and their corresponding entity labels and target box positions.

[0190] 2. This application proposes an image feature extraction module for extracting image embedding features for each entity. It uses the Vision Transformer (VIT) image encoder and the CLIP image encoder to extract the appearance visual features and semantic visual features of the entity reference image, respectively, capturing the image's visual details and semantic information. These features are then fused through cross-attention and self-attention mechanisms to form a more comprehensive entity feature representation.

[0191] 3. This application proposes an entity feature fusion process for extracting comprehensive embedding features of multiple entities. The visual embedding features of each entity are spliced ​​with the target box features to integrate appearance, semantics, and position information. Through the cross-attention mechanism, the spliced ​​features are fused with the entity label text embedding features to better combine image features with text features. The Q-former module is further used to optimize and fuse features to obtain the final embedding features of each entity. Finally, the final embedding features of each entity are spliced ​​to obtain multi-entity embedding features, providing a comprehensive feature representation for subsequent image generation.

[0192] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0193] Figure 3 This is a schematic diagram of the structure of the multi-entity image generation device provided in the embodiment of the present application. Figure 3 As shown, the multi-entity image generating device includes:

[0194] An acquisition module 301 is configured to acquire a text description of a plurality of entities, reference images of the plurality of entities, entity labels, and a target box position indicating a position of each entity in the reference image;

[0195] An encoding module 302 is configured to encode the text description to obtain global text description features, and to encode the entity label to obtain entity label text features;

[0196] Extraction module 303, configured to extract visual features from the reference image of each entity to obtain visual embedding features of each entity, and encode the target frame position of each entity to obtain target frame features of each entity;

[0197] A concatenation module 304 is configured to concatenate the visual embedding features with the target box features, perform cross-attention calculation on the concatenated features and the entity label text features, generate a final entity embedding feature for each entity, and concatenate the final entity embedding features of multiple entities to obtain a multi-entity embedding feature.

[0198] The output module 305 is used to input the global text description features and the multi-entity embedding features into the diffusion model, and output a target image that meets the text description after continuous processing using multiple submodules in the diffusion model.

[0199] In some embodiments, Figure 3 The encoding module 302 inputs the text description into the text encoder, extracts the embedded features of the global text description, obtains the global text description features, inputs the entity label into the text encoder, and extracts the entity label text features used to identify the entity.

[0200] In some embodiments, Figure 3 The extraction module 303 uses the first image encoder to extract the appearance visual features of the reference image; uses the second image encoder to extract the semantic visual features of the reference image; uses cross attention to fuse the appearance visual features and the semantic visual features; performs self-attention processing on the fused features and inputs them into the feedforward network to obtain the visual embedding features of the entity.

[0201] In some embodiments, Figure 3 The extraction module 303 inputs the target frame position of each entity into the position encoder for position encoding to obtain the encoding features of the target frame position, inputs the encoding features of the target frame position into the multi-layer perceptron, and uses the multi-layer perceptron to process to obtain the target frame features.

[0202] In some embodiments, Figure 3 The splicing module 304 uses the spliced ​​features as the query vector, the entity label text features as the key vector and the value vector, and performs cross-attention calculation on the query vector, the key vector and the value vector; the features output by the cross-attention are input into the feature optimization module, and the features are optimized and fused in the feature optimization module to obtain the final entity embedding features of each entity.

[0203] In some embodiments, Figure 3The output module 305 takes the output of the previous submodule as the input of the self-attention module to obtain the self-attention output; performs text cross-attention processing based on the self-attention output to obtain the text cross-attention output; performs multi-entity cross-attention processing based on the text cross-attention output to obtain the multi-entity cross-attention output; uses the multi-entity cross-attention output as the input of the feedforward network, and obtains the output of the current submodule after nonlinear transformation; uses the output of the current submodule as the input of the next submodule, and repeats the text cross-attention and multi-entity cross-attention processing until the last submodule outputs the target image.

[0204] In some embodiments, Figure 3 The output module 305 uses the self-attention output as the query vector and the global text description feature as the key vector and the value vector to perform text cross-attention calculation so as to integrate the text information into the image generation process; and uses the text cross-attention output as the query vector and the multi-entity embedding feature as the key vector and the value vector to perform multi-entity cross-attention calculation so as to integrate the multi-entity features into the image generation process.

[0205] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0206] Figure 4 Schematic diagram of the structure of the electronic device 4 provided in the embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps of the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0207] For example, computer program 403 may be divided into one or more modules / units, which are stored in memory 402 and executed by processor 401 to implement the present application. One or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of computer program 403 in electronic device 4.

[0208] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include but is not limited to a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0209] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0210] Memory 402 can be an internal storage unit of electronic device 4, such as a hard drive or memory of electronic device 4. Memory 402 can also be an external storage device of electronic device 4, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on electronic device 4. Furthermore, memory 402 can include both an internal storage unit of electronic device 4 and an external storage device. Memory 402 is used to store computer programs and other programs and data required by the electronic device. Memory 402 can also be used to temporarily store data that has been output or is about to be output.

[0211] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0212] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0213] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0214] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which may be electrical, mechanical or other forms.

[0215] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0217] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0218] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for generating a multi-entity image, characterized in that: include: Obtaining a text description including descriptions of multiple entities, reference images of the multiple entities, entity labels, and target box positions indicating positions of the respective entities in the reference images; Encoding the text description to obtain a global text description feature, and encoding the entity label to obtain an entity label text feature; Extract visual features from the reference image of each entity to obtain the visual embedding features of each entity, and encode the target frame position of each entity to obtain the target frame features of each entity; The visual embedding feature is concatenated with the target frame feature, and the concatenated feature is subjected to cross-attention calculation with the entity label text feature to generate a final entity embedding feature for each entity. The final entity embedding features of multiple entities are concatenated to obtain a multi-entity embedding feature. The global text description feature and the multi-entity embedding feature are input into a diffusion model, and after continuous processing using multiple submodules in the diffusion model, a target image that conforms to the text description is output.

2. The method according to claim 1, characterized in that The encoding of the text description to obtain a global text description feature, and the encoding of the entity tag to obtain an entity tag text feature, includes: The text description is input into a text encoder, and the embedding features of the global text description are extracted to obtain the global text description features. The entity label is input into a text encoder, and the entity label text features for identifying the entity are extracted.

3. The method according to claim 1, characterized in that The visual feature extraction of the reference image of each entity to obtain the visual embedding feature of each entity includes: extracting appearance visual features of the reference image using a first image encoder; extracting semantic visual features of the reference image using a second image encoder; Using cross attention to fuse the appearance visual features with the semantic visual features; The fused features are self-attention processed and input into the feedforward network to obtain the visual embedding features of the entity.

4. The method according to claim 1, wherein The target frame position of each entity is encoded to obtain the target frame features of each entity, including: The target frame position of each entity is input into a position encoder for position encoding to obtain the encoding features of the target frame position, and the encoding features of the target frame position are input into a multi-layer perceptron, which is used to process the target frame features to obtain the target frame features.

5. The method according to claim 1, characterized in that The cross-attention calculation is performed on the concatenated features and the entity label text features to generate the final entity embedding features of each entity, including: Using the concatenated features as a query vector, using the entity tag text features as a key vector and a value vector, and performing a cross-attention calculation on the query vector, the key vector, and the value vector; The features output by the cross attention are input into the feature optimization module, where the features are optimized and fused to obtain the final entity embedding features for each entity.

6. The method according to claim 1, characterized in that After continuously processing using multiple submodules in the diffusion model, outputting a target image that conforms to the text description includes: The output of the previous submodule is used as the input of the self-attention module to obtain the self-attention output; Performing text cross-attention processing according to the self-attention output to obtain a text cross-attention output; Performing multi-entity cross-attention processing according to the text cross-attention output to obtain a multi-entity cross-attention output; The multi-entity cross attention output is used as the input of the feedforward network, and the output of the current submodule is obtained through nonlinear transformation; The output of the current submodule is used as the input of the next submodule, and the text cross-attention and multi-entity cross-attention processing are repeatedly performed until the last submodule outputs the target image.

7. The method according to claim 6, characterized in that Performing text cross-attention processing according to the self-attention output, and performing multi-entity cross-attention processing according to the text cross-attention output, including: Using the self-attention output as a query vector and the global text description features as key vectors and value vectors, a text cross-attention calculation is performed to incorporate text information into the image generation process; The text cross-attention output is used as a query vector, and the multi-entity embedding features are used as key vectors and value vectors to perform multi-entity cross-attention calculation so as to integrate multi-entity features into the image generation process.

8. A multi-entity image generation device, characterized in that: include: An acquisition module is configured to acquire a text description including descriptions of multiple entities, reference images of the multiple entities, entity labels, and a target box position indicating a position of each entity in the reference image; An encoding module, configured to encode the text description to obtain a global text description feature, and to encode the entity label to obtain an entity label text feature; The extraction module is used to extract visual features from the reference image of each entity to obtain the visual embedding features of each entity, and encode the target frame position of each entity to obtain the target frame features of each entity; A splicing module is used to splice the visual embedding feature with the target box feature, and perform cross-attention calculation on the spliced ​​feature and the entity label text feature to generate a final entity embedding feature for each entity, and splice the final entity embedding features of multiple entities to obtain a multi-entity embedding feature; The output module is used to input the global text description feature and the multi-entity embedding feature into a diffusion model, and output a target image that conforms to the text description after continuous processing using multiple submodules in the diffusion model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images