Image generation method and device, electronic equipment and medium

Abstract

The invention relates to the technical field of image generation, and provides an image generation method and device, electronic equipment and a medium. The method can perform fusion processing on an image feature vector of a prompt style image and a time feature vector of a preset time step to obtain a time style fusion feature vector of the prompt style image; determining a style fusion feature vector of the prompt style image according to the image feature vector of the prompt style image; performing fusion processing on the image feature vectors of the to-be-identified image and the prompt style image to obtain an initial style feature vector of the to-be-identified image; determining a target feature vector of the to-be-recognized image according to the image feature vector of the to-be-recognized image; and denoising the noise map based on the feature vectors through a diffusion model to obtain a target image. According to the method, the efficiency and the effect of image style conversion are improved by optimizing feature fusion and denoising processing.

Description

Technical Field

[0001] The present disclosure relates to the field of image generation technology, and in particular to an image generation method, device, electronic device and medium. Background Art

[0002] With the rapid development of Internet technology, diffusion models have made significant progress in the field of image generation, reigniting interest in image style transfer technology. To address this problem, researchers proposed an end-to-end trained style transfer model. This model explicitly distinguishes content and style features through an independent feature injection mechanism, thereby achieving more efficient style transfer. This unified framework not only supports image-driven style transfer, but also extends to text-driven and text editing-driven stylized synthesis, demonstrating its diverse application potential. However, existing methods, such as DDIM-based inversion or adjustment of pre-trained IP-Adapter feature injection layers, and inversion of content and style images, can effectively extract features and inject them into specific layers, but these methods often lead to increased inference time and may lose important information during feature extraction.

[0003] Therefore, to address these deficiencies, an image generation method is provided. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide an image generation method, device, electronic device and computer-readable storage medium to solve the technical problems that the image generation method in the prior art has a long reasoning time and may also lose important information during the feature extraction process.

[0005] According to a first aspect of an embodiment of the present disclosure, an image generation method is provided, including: obtaining an image to be recognized, a prompt text, and a prompt style image; fusing an image feature vector of the prompt style image and a time feature vector of a preset time step to obtain a time style fusion feature vector of the prompt style image; determining a style fusion feature vector of the prompt style image according to the image feature vector of the prompt style image; fusing the image feature vectors of the image to be recognized and the prompt style image to obtain an initial style feature vector of the image to be recognized; determining a target feature vector of the image to be recognized according to the image feature vector of the image to be recognized; denoising a noise map based on a diffusion model based on the semantic feature vector of the prompt text, the time style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be recognized, and the target feature vector of the image to be recognized to obtain a target image.

[0006] According to a second aspect of an embodiment of the present disclosure, an image generation device is provided, including: an acquisition module, used to acquire an image to be recognized, a prompt text, and a prompt style image; a first fusion module, used to fuse the image feature vector of the prompt style image and the time feature vector of a preset time step to obtain the time style fusion feature vector of the prompt style image; a first determination module, used to determine the style fusion feature vector of the prompt style image according to the image feature vector of the prompt style image; a second fusion module, used to fuse the image feature vectors of the image to be recognized and the prompt style image to obtain the initial style feature vector of the image to be recognized; a second determination module, used to determine the target feature vector of the image to be recognized according to the image feature vector of the image to be recognized; a generation module, used to denoise the noise map based on the semantic feature vector of the prompt text, the time style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be recognized, and the target feature vector of the image to be recognized through a diffusion model to obtain a target image.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0009] Compared with the prior art, the disclosed embodiment has the following beneficial effects: the disclosed embodiment can effectively capture the style information in the time dimension by fusing the image feature vector of the prompt style image with the time feature vector of the preset time step, thereby improving the diversity and flexibility of style conversion. By independently determining the style fusion feature vector of the prompt style image, the extraction of style information is clearer, thereby improving the accuracy and consistency of style conversion. The image feature vectors of the image to be identified and the prompt style image are fused, and the initial style feature vector obtained can better reflect the style requirements of the image to be identified, thereby enhancing the quality of the generated result. The diffusion model is used to denoise the noise map, and combined with the semantic feature vector of the prompt text, the content information of the image can be effectively retained, and stylized synthesis can be achieved at the same time, ensuring that the generated target image meets the style requirements and has good visual effects. The method optimizes the calculation process and reduces the inference time during the feature injection and fusion process. Compared with the traditional method, it is more practical and suitable for real-time image style conversion scenarios. In addition, the method not only supports image-driven style conversion, but also can process text-driven and text editing-driven stylized synthesis, further expanding the scope of application and meeting the needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of an embodiment of the present invention can be applied;

[0012] Figure 2 is a flowchart of an image generation method provided by an embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram of an application framework of an image generation method provided by an embodiment of the present disclosure;

[0014] Figure 4 It is a schematic diagram of the framework of the first half of the submodules in the diffusion module provided by the embodiment of the present disclosure;

[0015] Figure 5 It is a schematic diagram of the framework of the latter half of the submodules in the diffusion module provided by the embodiment of the present disclosure;

[0016] Figure 6 is a block diagram of an image generating device provided by an embodiment of the present disclosure;

[0017] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0019] It should be noted that the user information (including but not limited to terminal device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0020] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of an embodiment of the present invention can be applied.

[0021] like Figure 1 As shown, the system architecture 100 may include one or more of a first terminal device 101, a second terminal device 102, and a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0022] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0023] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send data, etc. The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, portable computers, desktop computers, and the like.

[0024] The server 105 may be a server that provides various services. For example, the server 105 may obtain an image to be identified, a prompt text, and a prompt style image from the first terminal device 103 (or the second terminal device 102 or the third terminal device 103); fuse the image feature vector of the prompt style image and the time feature vector of the preset time step to obtain the time style fusion feature vector of the prompt style image; determine the style fusion feature vector of the prompt style image according to the image feature vector of the prompt style image; fuse the image feature vectors of the image to be identified and the prompt style image to obtain the initial style feature vector of the image to be identified; determine the target feature vector of the image to be identified according to the image feature vector of the image to be identified; denoise the noise image based on the semantic feature vector of the prompt text, the time style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified through the diffusion model to obtain the target image.

[0025] In some embodiments, the image generation method provided in the embodiments of the present invention is generally executed by the server 105, and accordingly, the image generation device is generally disposed in the server 105. In other embodiments, some terminal devices may have functions similar to those of the server to execute the method. Therefore, the image generation method provided in the embodiments of the present invention is not limited to being executed on the server side.

[0026] The image generation method and device according to the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0027] Figure 2 is a flowchart of an image generation method provided by an embodiment of the present disclosure. The method provided by an embodiment of the present disclosure can be executed by any electronic device with computer processing capabilities, for example, the electronic device can be Figure 1 The server is shown.

[0028] like Figure 2 As shown, the image generating method includes steps S210 to S260.

[0029] In step S210, an image to be recognized, a prompt text, and a prompt style image are obtained.

[0030] Step S220 , fusing the image feature vector of the hint style image and the time feature vector of the preset time step to obtain the time style fusion feature vector of the hint style image.

[0031] In step S230 , a style fusion feature vector of the hint style image is determined according to the image feature vector of the hint style image.

[0032] In step S240, the image feature vectors of the image to be identified and the prompt style image are fused to obtain an initial style feature vector of the image to be identified.

[0033] In step S250, a target feature vector of the image to be identified is determined according to the image feature vector of the image to be identified.

[0034] In step S260, the noise image is denoised by a diffusion model based on the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified to obtain a target image.

[0035] This method can effectively capture the style information in the time dimension by fusing the image feature vector of the prompt style image with the time feature vector of the preset time step, thereby improving the diversity and flexibility of style transfer. By independently determining the style fusion feature vector of the prompt style image, the extraction of style information is clearer, thereby improving the accuracy and consistency of style transfer. The image feature vectors of the image to be identified and the prompt style image are fused, and the initial style feature vector obtained can better reflect the style requirements of the image to be identified, thereby enhancing the quality of the generated results. The diffusion model is used to denoise the noise map, and combined with the semantic feature vector of the prompt text, the content information of the image can be effectively retained, and stylized synthesis can be achieved at the same time, ensuring that the generated target image meets the style requirements and has good visual effects. This method optimizes the calculation process and reduces the inference time during the feature injection and fusion process. Compared with traditional methods, it is more practical and suitable for real-time image style transfer scenarios. In addition, this method not only supports image-driven style transfer, but also can process text-driven and text editing-driven stylized synthesis, further expanding the scope of application and meeting the needs of different users.

[0036] In some embodiments of the present disclosure, the image to be identified is the original image that needs to be style-converted, and it can be any type of entity reference image. This image usually contains content information that is expected to be retained in the final generated image. For example, a landscape photo, a portrait, or a still life image. The content features of the image to be identified will serve as a basis to help the generated image remain consistent with the original image in morphology and structure. The prompt text is text information used to describe the morphology, posture, and other contents of the entity object in the final generated image. It provides semantic guidance for the generation model and helps define the features that the generated image should have. For example, the prompt text can be "a girl in a red dress dancing on the beach" or "a lonely tree in a sunny valley". These descriptions provide a specific semantic framework for the generation process, so that the model can understand and generate image content that meets expectations. The prompt style image is a style reference image used to replace the style in the image to be identified. It provides the required visual style features to help the generated image be distinguished from the image to be identified in style. For example, an impressionist painting, a modern art work, or a certain artistic style image (such as a watercolor painting, an oil painting, etc.). The features of the hint style image will be extracted and applied to the image to be identified, thereby achieving style transfer, so that the final generated image visually conforms to the selected style. Through the combination of these three elements, the image to be identified provides the content basis, the hint text provides semantic guidance, and the hint style image provides style reference, thereby achieving efficient and accurate image style transfer. This multimodal input method allows the generated image to retain the structure of the original image while also having a new artistic style, meeting the personalized needs of users.

[0037] refer to Figure 3, the prompt text is a text description provided by the user to guide the content and features of the generated image. These text descriptions can include information such as the shape, color, action, environment, etc. of the object. The prompt text is processed using a text encoder (such as natural language processing models such as BERT and GPT). The text encoder can convert natural language text into a high-dimensional feature vector to capture the semantic information in the text. After encoding, the feature vector (i.e., text features) of the prompt text will contain semantic features related to the text content. These feature vectors will be used to generate the model to ensure that the final image meets the user's description. The preset time step (usually expressed as time step t) is a key parameter in the generation process, referring to a specific stage or state of the model when generating an image. The preset time step is feature extracted by a multi-layer perceptron. MLP can extract features of the input data through multiple layers of nonlinear transformations. After MLP processing, the feature vector of the preset time step obtained will contain dynamic feature information related to the time step. These feature vectors help the model consider the impact of time changes on the image style during the generation process. The prompt style image is an image used to provide a style reference, usually an image with a specific artistic style (such as impressionist paintings, modern art works, etc.). The prompt style image is processed using an image encoder (such as a convolutional neural network (CNN). The image encoder can extract visual features of the image, including information such as color, texture, and shape. After encoding, the image feature vector of the prompt style image obtained will contain visual features related to the style. These features will be used to introduce the target style in the generation process so that the final generated image can reflect the artistic features of the prompt style image. By extracting features of the prompt text, preset time step, and prompt style image, information from different modalities can be effectively integrated. This integration not only enhances the ability of the generation model to understand user needs, but also improves the expressiveness of the generated image in terms of content and style, ensuring that the final result not only meets the user's description but also has the required artistic style.

[0038] In some embodiments of the present disclosure, the image feature vector of the prompt style image and the time feature vector of the preset time step are fused to obtain the time style fusion feature vector of the prompt style image, including: processing the image feature vector of the prompt style image by a first multi-layer perceptron to obtain an adapted style image feature vector; adding the time feature vector of the preset time step and the adapted style image feature vector to obtain the time style fusion feature vector of the prompt style image.

[0039] refer to Figure 3, the image feature vector of the hint style image is usually a high-dimensional feature vector extracted by an image encoder, which contains information such as color, texture and shape of the style image. The first multi-layer perceptron (MLP1): MLP1 is used to process the image feature vector of the hint style image so as to adjust it to the adapted style image feature vector. For example, MLP1 learns a deeper representation of the style features through multiple layers of nonlinear transformations. This process includes operations such as scaling, translating and nonlinear activation of the features, so that the feature vector is more suitable for subsequent fusion operations. The temporal feature vector of the preset time step is usually a dynamic feature related to the time step extracted by a multi-layer perceptron (MLP). This feature vector contains time-related change information and can reflect the dynamic features that the image should have at a specific time step. The adapted style image feature vector is added to the temporal feature vector of the preset time step. This process can be regarded as a weighted combination of two different features, aiming to effectively combine the style features with the temporal features. Through the addition operation, the temporal style fusion feature vector of the hint style image obtained will contain both the visual features of the style image and the dynamic features of the time step. This fused feature vector can provide rich information for subsequent image generation, ensuring that the generated image meets expectations in terms of style and dynamic performance. By fusing the image feature vector of the prompt style image and the time feature vector of the preset time step, an image with rich style and dynamic features can be generated. This process not only improves the quality and expressiveness of the generated image, but also enhances the adaptability of the model to user needs, ensuring that the generated results are more in line with the user's personalized expectations.

[0040] In some embodiments of the present disclosure, determining the style fusion feature vector of the prompt style image based on the image feature vector of the prompt style image includes: processing the image feature vector of the prompt style image through a first multi-layer perceptron to obtain an adapted style image feature vector; performing cross-attention processing on the adapted style image feature vector to obtain a style fusion feature vector of the prompt style image.

[0041] refer to Figure 3, the image feature vector of the prompt style image is processed by MLP1 to obtain the adapted style image feature vector. The adapted style image feature vector is processed by the learnable style cross-attention module 2 to obtain the style fusion feature vector of the prompt style image. For example, the image feature vector of the prompt style image: the feature vector is extracted by the image encoder and contains visual information such as color, texture, and shape of the image. MLP1 can be composed of multiple fully connected layers, and each layer usually contains an activation function (such as ReLU or Leaky ReLU) to introduce nonlinear characteristics. MLP1 transforms and adapts the input image feature vector. This process includes: adjusting the scale of the feature to make it more suitable for subsequent processing. Through weight learning, important features are highlighted and irrelevant features are suppressed. Introducing nonlinearity enables the model to capture more complex feature relationships. After being processed by MLP1, the adapted style image feature vector obtained will be more in line with the needs of subsequent fusion operations. The learnable style cross-attention module (Style Cross-Attention Module 2) is a network structure based on the attention mechanism, which aims to enhance the feature expression ability by focusing on the relationship between different features. This module can introduce information from other features (such as text features or time features) into the adapted style image feature vector, thereby achieving interaction and fusion between features. By calculating the similarity between features, the module can dynamically adjust the importance of features and highlight features related to the current task. Calculation process: Calculate the similarity between input features, usually using dot product or cosine similarity. Generate attention weights based on the similarity to indicate the importance of different features. Apply the attention weights to the input features to generate weighted feature representations. After being processed by the cross-attention module, the style fusion feature vector of the cued style image obtained will contain rich style information and context relevance, and can better reflect the target style. The image feature vector of the cued style image is adapted by MLP1 and further processed by the learnable style cross-attention module. The final generated style fusion feature vector can effectively integrate the information of different features. This process not only enhances the style expression of the image, but also improves the model's ability to understand complex relationships, providing strong feature support for subsequent image generation.

[0042] In some embodiments of the present disclosure, the image feature vectors of the image to be recognized and the prompt style image are fused to obtain the initial style feature vector of the image to be recognized, including: performing style semantic extraction on the image feature vector of the prompt style image to obtain the style semantic feature vector of the prompt style image; processing the image feature vector of the prompt style image by a first multi-layer perceptron to obtain an adapted style image feature vector; processing the style semantic feature vector of the prompt style image, the adapted style image feature vector, and the image to be recognized by a control network to obtain the initial style feature vector of the image to be recognized.

[0043] refer to Figure 3, the image feature vector of the prompt style image is subjected to style semantic extraction by BLIP-Qformer to obtain the style semantic feature vector of the prompt style image. The image feature vector of the prompt style image is processed by MLP1 to obtain the adapted style image feature vector. The adapted style image feature vector is processed by the learnable style cross-attention module 1 to obtain the cross-attention vector. Then the entity reference image, the style semantic feature vector of the prompt style image, and the cross-attention vector are input into the control network, which processes them to obtain the initial style feature vector of the image to be identified. For example, BLIP-Qformer is a model that combines visual and language features and can extract semantic information from images through multimodal learning. The image feature vector of the prompt style image is usually a high-dimensional feature representation obtained by an image encoder. BLIP-Qformer identifies semantic information related to style by analyzing image features. This process involves understanding the content of the image, such as object recognition, scene analysis, and induction of style features. After being processed by BLIP-Qformer, the style semantic feature vector of the prompt style image obtained will contain rich style semantic information, which can reflect the style features and context of the image. The image feature vector of the hint style image is processed by MLP1, that is, the input features are transformed through multiple fully connected layers to adapt to the subsequent processing requirements. The features are adjusted to make them more balanced in value and highlight the important features. Nonlinearity is introduced through the activation function to enhance the expressiveness of the model. The adapted style image feature vector is obtained and prepared for subsequent processing. The learnable style cross-attention module 1 is based on the attention mechanism and aims to enhance the feature expression by modeling the relationship between different features. The adapted style image feature vector is processed by this module. Processing process: Calculate the similarity between the input feature and other related features (such as style semantic features). Generate attention weights based on the similarity to indicate the importance of different features. Apply the attention weights to the adapted style image feature vector to generate a cross-attention vector. The obtained cross-attention vector will fuse the style image features and other modal information to enhance the contextual relevance of the features. Control the processing process of the network, and the input features are the entity reference image, the style semantic feature vector of the hint style image, and the cross-attention vector. The entity reference image is a reference image provided by the user, which contains the required style or content features. The style semantic feature vector of the hint style image is the style semantic feature extracted from BLIP-Qformer. The cross attention vector is the feature obtained from the learnable style cross attention module 1. The control network is a network structure used to integrate and adjust input features, aiming to generate feature representations that meet specific requirements. Specifically, the input entity reference image, style semantic feature vector and cross attention vector are processed comprehensively.Through the learning ability of the network, the features are adjusted to ensure that the generated feature vector can accurately reflect the style requirements of the image to be identified. After being processed by the control network, the initial style feature vector of the image to be identified is obtained, which will serve as the basis for the subsequent generation process.

[0044] The style semantic features are extracted through BLIP-Qformer, the style image features are adapted using MLP1, and the cross-attention vector is generated through the learnable style cross-attention module 1. Finally, these features are input into the control network for processing, and the initial style feature vector of the image to be identified is finally obtained. This series of steps ensures the effective fusion of different features, so that the generated feature vector can accurately reflect the user's style requirements and image content, providing strong support for subsequent image generation.

[0045] In some embodiments of the present disclosure, determining the target feature vector of the image to be identified based on the image feature vector of the image to be identified includes: processing the image feature vector of the image to be identified by a second multi-layer perceptron to obtain an adapted entity image feature vector; and performing cross-attention processing on the adapted entity image feature vector to obtain a target feature vector of the image to be identified.

[0046] refer to Figure 3, the entity reference image is encoded by an image encoder to obtain an image feature vector of the image to be identified. The image feature vector of the image to be identified is processed by a second multi-layer perceptron (MLP2) to obtain an adapted entity image feature vector. The adapted entity image feature vector is cross-attention processed by a learnable content cross module to obtain the target features of the image to be identified. For example, an image encoder is a neural network model that aims to convert an input image into a high-dimensional feature vector to capture the visual information of the image. The entity reference image is usually an image provided by a user, which contains the content features that the image to be identified should have. The image encoder processes the input image through structures such as convolutional layers and pooling layers to extract information such as color, texture, and shape of the image. After encoding, an image feature vector of the image to be identified is obtained, which is usually a high-dimensional vector that can effectively represent the main visual features of the image. The image feature vector of the image to be identified is processed by MLP2, for example, MLP2 is composed of multiple fully connected layers and contains activation functions (such as ReLU or Sigmoid) to introduce nonlinearity. Specifically, the input feature vector is transformed to better meet the needs of subsequent processing. Through the learned weights, important features are highlighted and redundant information is suppressed. After processing by MLP2, the adapted entity image feature vector is obtained, which will prepare for the subsequent cross-attention processing. The learnable content cross module is a network structure that uses the attention mechanism to enhance the feature expression ability by focusing on the relationship between different features. This module calculates the similarity between the adapted entity image features and other related features (such as style features or contextual information). Attention weights are generated based on the similarity to indicate the importance of different features. The attention weights are applied to the adapted entity image feature vector to generate a weighted feature representation. After processing by the content cross module, the target features of the image to be identified will integrate multiple information and can effectively reflect the content characteristics of the image to be identified.

[0047] The entity reference image is encoded by the image encoder to generate the image feature vector of the image to be recognized; then, the feature vector is adapted by the second multi-layer perceptron (MLP2) to obtain the adapted entity image feature vector; finally, the cross-attention processing is performed by the learnable content cross module to generate the target features of the image to be recognized. This series of steps ensures the effective extraction and fusion of features, providing strong support for the subsequent image generation or recognition process.

[0048] In some embodiments of the present disclosure, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified obtained by the above method are input into the diffusion model.

[0049] refer to Figure 3 The diffusion model may include at least two submodules, for example, a front-half module and a back-half submodule, wherein one or more submodules may be configured in the front-half module and the back-half submodule. Taking the configuration of one submodule as an example, the input of the front-half module is a noise map, a semantic feature vector of the prompt text, a temporal style fusion feature vector of the prompt style image, and a target feature vector of the image to be identified. The input of the back-half submodule is a first fusion feature vector, a semantic feature vector of the prompt text, a temporal style fusion feature vector of the prompt style image, a style fusion feature vector of the prompt style image, and an initial style feature vector of the image to be identified.

[0050] Based on the aforementioned embodiment, the noise map is denoised based on the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified through a diffusion model to obtain a target image, including: fusing the noise map, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, and the target feature vector of the image to be identified to obtain a first fused feature vector; fusing the first fused feature vector, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, and the initial style feature vector of the image to be identified to obtain a second fused feature vector; and generating a target image based on the second fused feature vector.

[0051] In some embodiments of the present disclosure, a noise map, a semantic feature vector of a prompt text, a temporal style fusion feature vector of a prompt style image, and a target feature vector of an image to be identified are fused to obtain a first fused feature vector, including: performing self-attention processing on the noise map to obtain a self-attention feature vector, performing text cross-attention processing on the self-attention feature vector and the semantic feature vector of the prompt text to obtain a text cross-attention vector, performing content cross-attention processing on the text cross-attention vector and the target feature vector of the image to be identified to obtain a content cross-attention vector, performing adaptive normalization processing on the temporal style fusion feature vector of the prompt style image and the content cross-attention vector to obtain a first normalized vector, performing residual processing on the normalized vector and the noise map to obtain a residual vector, processing the residual vector through a feedforward network to obtain a feedforward vector, performing adaptive normalization processing on the feedforward vector and the temporal style fusion feature vector of the prompt style image to obtain a second normalized vector, and performing residual processing on the second normalized vector and the residual vector to obtain a first fused feature vector.

[0052] refer to Figure 4 and Figure 5,The architecture in the first half of the sub-modules is the same as the second half of the sub-modules, which include self-attention layer, text cross-attention layer, style cross-attention layer, adaptive normalization layer (AdaLN), and feed-forward network.

[0053] Specifically, refer to Figure 4 The noise map is processed by self-attention layer to obtain a self-attention feature vector, the self-attention feature vector and the semantic feature vector of the prompt text are processed by text cross-attention layer to obtain a text cross-attention vector, the text cross-attention vector and the target feature vector of the image to be identified are processed by content cross-attention layer to obtain a content cross-attention vector, the temporal style fusion feature vector of the prompt style image and the content cross-attention vector are adaptively normalized by an adaptive normalization layer (AdaLN) to obtain a first normalized vector, the normalized vector and the noise map are residually processed to obtain a residual vector, the residual vector is processed by a feedforward network to obtain a feedforward vector, the feedforward vector and the temporal style fusion feature vector of the prompt style image are adaptively normalized by an adaptive normalization layer (AdaLN) to obtain a second normalized vector, the second normalized vector and the residual vector are residually processed to obtain a first fused feature vector, that is, the output of the first half of the sub-module is the first fused feature vector.

[0054] Based on the above embodiments, the noise map is used as the initial input, which is usually a randomly generated image. The self-attention layer contains query, key, and value. The noise map is linearly transformed to generate query (Query), key (Key), and value (Value) vectors. The dot product of the query and the key is calculated to obtain the attention weight, and the value vector is weighted and summed. A self-attention feature vector is obtained, which captures important information and long-distance dependencies in the noise map. The text cross-attention layer contains query, key, and value: the self-attention feature vector is used as the query, and the semantic feature vector of the prompt text is used as the key and value. The attention weight between the self-attention feature and the text feature is calculated to emphasize the information related to the text. The text cross-attention vector is obtained, which combines the self-attention feature with the text semantic feature. The style cross-attention layer contains query, key, and value: the text cross-attention vector is used as the query, and the target feature vector of the image to be identified is used as the key and value. The attention weight between the text feature and the target feature is calculated to extract information related to the target feature. The content cross-attention vector is obtained, which combines the text feature with the target image feature. Adaptive Normalization Layer (AdaLN): Normalizes the content cross attention vector. Adjusts the normalized features according to the temporal style fusion feature vector of the cued style image. Obtains the first normalized vector, which enhances the adaptability of the features. Adds the first normalized vector to the noise map to form a residual connection to preserve the original information. Obtains the residual vector, which combines the normalized features with the noise map information. Feedforward Network: Processes the residual vector through a series of fully connected layers, usually including an activation function (such as ReLU) to increase nonlinearity. Obtains the feedforward vector, which is transformed nonlinearly to further extract features. The second adaptive normalization process normalizes the feedforward vector. Adjusts according to the temporal style fusion feature vector of the cued style image. Obtains the second normalized vector, which enhances the adaptability of the feedforward vector. Finally, the residual process adds the second normalized vector to the residual vector to form the final feature fusion. Obtains the first fused feature vector, which is the final output of the first half of the submodules and contains the feature information after multiple layers of processing. Through the above steps, the first half of the sub-module effectively converts the noise map into a first fused feature vector with rich features, ready for subsequent processing and generation of the target image. Each level of processing emphasizes different features to ensure that the generated image can accurately reflect the prompt text and style requirements. In this example, if the number of sub-modules in the first half is greater than or equal to 2, the first sub-module input can be a noise map and other feature vectors, and starting from the second sub-module, the input is the output of the previous module and other feature vectors. Other feature vectors can be the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, and the target feature vector of the image to be identified.

[0055] In some embodiments of the present disclosure, a first fused feature vector, a semantic feature vector of a prompt text, a temporal style fusion feature vector of a prompt style image, a style fusion feature vector of a prompt style image, and an initial style feature vector of an image to be identified are fused to obtain a second fused feature vector, including: performing self-attention processing on the first fused feature vector to obtain a self-attention feature vector, performing text cross-attention processing on the self-attention feature vector and the semantic feature vector of the prompt text to obtain a text cross-attention vector, performing style cross-attention processing on the text cross-attention vector and the initial style feature vector of the image to be identified to obtain a style cross-attention vector, performing adaptive normalization processing on the temporal style fusion feature vector of the prompt style image and the style cross-attention vector to obtain a first normalized vector, performing residual processing on the normalized vector and the first fused feature vector to obtain a residual vector, processing the residual vector through a feedforward network to obtain a feedforward vector, performing adaptive normalization processing on the feedforward vector and the temporal style fusion feature vector of the prompt style image to obtain a second normalized vector, and performing residual processing on the second normalized vector, the residual vector, and the initial style feature vector of the image to be identified to obtain a second fused feature vector.

[0056] Specifically, refer to Figure 5 The self-attention layer is used to perform self-attention processing on the first fused feature vector to obtain a self-attention feature vector, the text cross-attention layer is used to perform text cross-attention processing on the self-attention feature vector and the semantic feature vector of the prompt text to obtain a text cross-attention vector, the style cross-attention layer is used to perform style cross-attention processing on the text cross-attention vector and the initial style feature vector of the image to be identified to obtain a style cross-attention vector, the adaptive normalization layer is used to perform adaptive normalization processing on the temporal style fusion feature vector and the style cross-attention vector of the prompt style image to obtain a first normalized vector, the normalized vector and the first fused feature vector are subjected to residual processing to obtain a residual vector, the residual vector is processed through a feedforward network to obtain a feedforward vector, the feedforward vector and the temporal style fusion feature vector of the prompt style image are subjected to adaptive normalization processing through an adaptive normalization layer to obtain a second normalized vector, and the second normalized vector, the residual vector and the initial style feature vector of the image to be identified are subjected to residual processing to obtain a second fused feature vector.

[0057] Based on the above embodiment, the first fused feature vector comes from the output of the first half of the sub-modules, and contains feature information processed by multiple layers. The self-attention layer contains query, key, and value: the first fused feature vector is linearly transformed to generate query (Query), key (Key), and value (Value) vectors. The dot product of the query and the key is calculated to obtain the attention weight, and the value vector is weighted and summed. A self-attention feature vector is obtained, which emphasizes the important information and long-distance dependencies in the first fused feature vector. The text cross-attention layer contains query, key, and value: the self-attention feature vector is used as the query, and the semantic feature vector of the prompt text is used as the key and value. The attention weight between the self-attention feature and the text feature is calculated to emphasize the information related to the text. The text cross-attention vector is obtained, which combines the self-attention feature with the text semantic feature. The style cross-attention layer contains query, key, and value: the text cross-attention vector is used as the query, and the initial style feature vector of the image to be identified is used as the key and value. The attention weight between the text feature and the initial style feature is calculated to extract information related to the style. The style cross-attention vector is obtained, which combines the text feature with the initial style feature. Adaptive Normalization Layer (AdaLN): Normalizes the style cross attention vector. Adjusts the normalized features according to the temporal style fusion feature vector of the cued style image. Obtains the first normalized vector, which enhances the adaptability of the style features. Adds the first normalized vector to the first fused feature vector to form a residual connection to preserve the original information. Obtains a residual vector, which combines the information of the normalized features with the first fused features. Feedforward Network: Processes the residual vector through a series of fully connected layers, usually including an activation function (such as ReLU) to increase nonlinearity. Obtains a feedforward vector, which is further transformed through nonlinear transformation to extract features. Input feedforward vector for the second adaptive normalization process: output from the feedforward network. Temporal style fusion feature vector of the cued style image: also used for adjustment. Normalize the feedforward vector. Adjust according to the temporal style fusion feature vector of the cued style image. Obtains the second normalized vector, which enhances the adaptability of the feedforward vector. The input to the final residual process is the second normalized vector: output from the second adaptive normalization process. Residual vector: from the previous residual processing. Initial style feature vector of the image to be identified: represents the initial style features of the target image to be generated. The second normalized vector, the residual vector and the initial style feature vector of the image to be identified are added to form the final feature fusion. The second fused feature vector is obtained, which is the final output of the second half submodule and contains feature information after multi-layer processing. Through the above steps, the second half submodule effectively converts the first fused feature vector into the second fused feature vector, ready for subsequent generation of the target image.Each level of processing emphasizes different features to ensure that the generated image can accurately reflect the prompt text and style requirements. In this example, if the number of sub-modules in the second half is greater than or equal to 2, the input of the first sub-module can be the output of the last sub-module in the first half of the sub-module and other feature vectors, and the input from the second sub-module is the output of the previous sub-module and other feature vectors. Other feature vectors can be the first fused feature vector, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, and the initial style feature vector of the image to be identified.

[0058] Figure 6 It is a block diagram of an image generating device provided by an embodiment of the present disclosure.

[0059] like Figure 6 As shown, the image generating device 600 includes an acquisition module 610 , a first fusion module 620 , a first determination module 630 , a second fusion module 640 , a second determination module 660 and a generation module 660 .

[0060] Specifically, the acquisition module 610 is used to acquire the image to be recognized, the prompt text, and the prompt style image.

[0061] The first fusion module 620 is used to fuse the image feature vector of the hint style image and the time feature vector of the preset time step to obtain the time style fusion feature vector of the hint style image.

[0062] The first determination module 630 is configured to determine a style fusion feature vector of the hint style image according to the image feature vector of the hint style image.

[0063] The second fusion module 640 is used to fuse the image feature vectors of the image to be identified and the prompt style image to obtain an initial style feature vector of the image to be identified.

[0064] The second determination module 650 is used to determine a target feature vector of the image to be identified according to the image feature vector of the image to be identified.

[0065] The generation module 660 is used to denoise the noise image through a diffusion model based on the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified to obtain a target image.

[0066] The image generation device 600 can effectively capture the style information in the time dimension by fusing the image feature vector of the prompt style image with the time feature vector of the preset time step, thereby improving the diversity and flexibility of style conversion. By independently determining the style fusion feature vector of the prompt style image, the extraction of style information is clearer, thereby improving the accuracy and consistency of style conversion. The image feature vectors of the image to be identified and the prompt style image are fused, and the initial style feature vector obtained can better reflect the style requirements of the image to be identified, thereby enhancing the quality of the generated result. The noise map is denoised by using a diffusion model, and combined with the semantic feature vector of the prompt text, the content information of the image can be effectively retained, and stylized synthesis can be achieved at the same time, ensuring that the generated target image meets the style requirements and has good visual effects. The method optimizes the calculation process and reduces the reasoning time during feature injection and fusion. Compared with traditional methods, it is more practical and suitable for real-time image style conversion scenarios. In addition, the method not only supports image-driven style conversion, but also can process text-driven and text editing-driven stylized synthesis, further expanding the scope of application and meeting the needs of different users.

[0067] In some embodiments of the present disclosure, the first fusion module 620 is configured to: process the image feature vector of the prompt style image through a first multi-layer perceptron to obtain an adapted style image feature vector; add the time feature vector of a preset time step and the adapted style image feature vector to obtain a time style fusion feature vector of the prompt style image.

[0068] In some embodiments of the present disclosure, the first determination module 630 is configured to: process the image feature vector of the prompt style image through a first multi-layer perceptron to obtain an adapted style image feature vector; perform cross-attention processing on the adapted style image feature vector to obtain a style fusion feature vector of the prompt style image.

[0069] In some embodiments of the present disclosure, the second fusion module 640 is configured to: perform style semantic extraction on the image feature vector of the prompt style image to obtain the style semantic feature vector of the prompt style image; process the image feature vector of the prompt style image through the first multi-layer perceptron to obtain the adapted style image feature vector; process the style semantic feature vector of the prompt style image, the adapted style image feature vector, and the image to be identified through the control network to obtain the initial style feature vector of the image to be identified.

[0070] In some embodiments of the present disclosure, the second determination module 650 is configured to: process the image feature vector of the image to be identified through a second multi-layer perceptron to obtain an adapted entity image feature vector; perform cross-attention processing on the adapted entity image feature vector to obtain a target feature vector of the image to be identified.

[0071] In some embodiments of the present disclosure, the generation module 660 is configured to: fuse the noise map, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, and the target feature vector of the image to be identified to obtain a first fused feature vector; fuse the first fused feature vector, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, and the initial style feature vector of the image to be identified to obtain a second fused feature vector; and generate a target image based on the second fused feature vector.

[0072] In some embodiments of the present disclosure, a noise map, a semantic feature vector of a prompt text, a temporal style fusion feature vector of a prompt style image, and a target feature vector of an image to be identified are fused to obtain a first fused feature vector, including: performing self-attention processing on the noise map to obtain a self-attention feature vector, performing text cross-attention processing on the self-attention feature vector and the semantic feature vector of the prompt text to obtain a text cross-attention vector, performing content cross-attention processing on the text cross-attention vector and the target feature vector of the image to be identified to obtain a content cross-attention vector, performing adaptive normalization processing on the temporal style fusion feature vector of the prompt style image and the content cross-attention vector to obtain a first normalized vector, performing residual processing on the normalized vector and the noise map to obtain a residual vector, processing the residual vector through a feedforward network to obtain a feedforward vector, performing adaptive normalization processing on the feedforward vector and the temporal style fusion feature vector of the prompt style image to obtain a second normalized vector, performing residual processing on the second normalized vector and the residual vector to obtain the first fused feature vector; performing residual processing on the first fused feature vector and the semantic feature vector of the prompt text The method comprises: performing self-attention processing on the first fused feature vector to obtain a self-attention feature vector, performing text cross-attention processing on the self-attention feature vector and the semantic feature vector of the prompt text to obtain a text cross-attention vector, performing style cross-attention processing on the text cross-attention vector and the initial style feature vector of the image to be recognized to obtain a style cross-attention vector, performing adaptive normalization processing on the temporal style fusion feature vector of the prompt style image and the style cross-attention vector to obtain a first normalized vector, performing residual processing on the normalized vector and the first fused feature vector to obtain a residual vector, processing the residual vector through a feedforward network to obtain a feedforward vector, performing adaptive normalization processing on the feedforward vector and the temporal style fusion feature vector of the prompt style image to obtain a second normalized vector, and performing residual processing on the second normalized vector, the residual vector and the initial style feature vector of the image to be recognized to obtain a second fused feature vector.

[0073] Figure 7 Schematic diagram of an electronic device 7 provided by an embodiment of the present disclosure. Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program 703, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 701 executes the computer program 703, the functions of the modules in the above-mentioned device embodiments are implemented.

[0074] The electronic device 7 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 7 may include, but is not limited to, a processor 701 and a memory 702. Those skilled in the art will appreciate that Figure 7 The electronic device 7 is merely an example and does not limit the electronic device 7 , and may include more or less components than those shown in the figure, or different components.

[0075] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0076] The memory 702 may be an internal storage unit of the electronic device 7, for example, a hard disk or memory of the electronic device 7. The memory 702 may also be an external storage device of the electronic device 7, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. The memory 702 may also include both an internal storage unit of the electronic device 7 and an external storage device. The memory 702 is used to store computer programs and other programs and data required by the electronic device.

[0077] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0079] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. An image generation method, characterized in that: include: Obtain the image to be recognized, the prompt text, and the prompt style image; Fusing the image feature vector of the hint style image and the time feature vector of a preset time step to obtain a time style fusion feature vector of the hint style image; Determining a style fusion feature vector of the hint style image according to the image feature vector of the hint style image; Performing a fusion process on the image feature vectors of the image to be identified and the prompt style image to obtain an initial style feature vector of the image to be identified; Determining a target feature vector of the image to be identified based on the image feature vector of the image to be identified; The noise image is denoised by a diffusion model based on the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified to obtain a target image.

2. The method according to claim 1, characterized in that The image feature vector of the hint style image and the time feature vector of the preset time step are fused to obtain the time style fusion feature vector of the hint style image, including: Processing the image feature vector of the prompt style image by a first multi-layer perceptron to obtain an adapted style image feature vector; The temporal feature vector of the preset time step and the adapted style image feature vector are added together to obtain the temporal style fusion feature vector of the prompt style image.

3. The method according to claim 1, characterized in that Determining, according to the image feature vector of the hint style image, the style fusion feature vector of the hint style image comprises: Processing the image feature vector of the prompt style image by a first multi-layer perceptron to obtain an adapted style image feature vector; A cross-attention process is performed on the adapted style image feature vector to obtain a style fusion feature vector of the prompt style image.

4. The method according to claim 1, characterized in that The image feature vectors of the image to be identified and the prompt style image are fused to obtain the initial style feature vector of the image to be identified, including: Performing style semantic extraction on the image feature vector of the prompt style image to obtain the style semantic feature vector of the prompt style image; Processing the image feature vector of the prompt style image by a first multi-layer perceptron to obtain an adapted style image feature vector; The style semantic feature vector of the prompt style image, the adapted style image feature vector, and the image to be identified are processed by a control network to obtain an initial style feature vector of the image to be identified.

5. The method according to claim 1, characterized in that Determining a target feature vector of the image to be identified according to the image feature vector of the image to be identified includes: Processing the image feature vector of the image to be identified by a second multi-layer perceptron to obtain an adapted entity image feature vector; The adapted entity image feature vector is subjected to cross attention processing to obtain a target feature vector of the image to be identified.

6. The method according to claim 1, characterized in that The noise image is denoised by using a diffusion model based on the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified, to obtain a target image, including: fusing the noise map, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, and the target feature vector of the image to be identified to obtain a first fusion feature vector; fusing the first fused feature vector, the semantic feature vector of the prompt text, the temporal style fused feature vector of the prompt style image, the style fused feature vector of the prompt style image, and the initial style feature vector of the image to be identified to obtain a second fused feature vector; The target image is generated based on the second fused feature vector.

7. The method according to claim 6, characterized in that The noise map, the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, and the target feature vector of the image to be identified are fused to obtain a first fused feature vector, which includes: Performing self-attention processing on the noise image to obtain a self-attention feature vector, performing text cross-attention processing on the self-attention feature vector and the semantic feature vector of the prompt text to obtain a text cross-attention vector, performing content cross-attention processing on the text cross-attention vector and the target feature vector of the image to be identified to obtain a content cross-attention vector, performing adaptive normalization processing on the temporal style fusion feature vector of the prompt style image and the content cross-attention vector to obtain a first normalized vector, performing residual processing on the normalized vector and the noise image to obtain a residual vector, processing the residual vector through a feedforward network to obtain a feedforward vector, performing adaptive normalization processing on the feedforward vector and the temporal style fusion feature vector of the prompt style image to obtain a second normalized vector, and performing residual processing on the second normalized vector and the residual vector to obtain the first fused feature vector; The first fused feature vector, the semantic feature vector of the prompt text, the temporal style fused feature vector of the prompt style image, the style fused feature vector of the prompt style image, and the initial style feature vector of the image to be identified are fused to obtain a second fused feature vector, including: Perform self-attention processing on the first fused feature vector to obtain a self-attention feature vector, perform text cross-attention processing on the self-attention feature vector and the semantic feature vector of the prompt text to obtain a text cross-attention vector, perform style cross-attention processing on the text cross-attention vector and the initial style feature vector of the image to be identified to obtain a style cross-attention vector, perform adaptive normalization processing on the temporal style fusion feature vector of the prompt style image and the style cross-attention vector to obtain a first normalized vector, perform residual processing on the normalized vector and the first fused feature vector to obtain a residual vector, process the residual vector through a feedforward network to obtain a feedforward vector, perform adaptive normalization processing on the feedforward vector and the temporal style fusion feature vector of the prompt style image to obtain a second normalized vector, perform residual processing on the second normalized vector, the residual vector and the initial style feature vector of the image to be identified to obtain the second fused feature vector.

8. An image generating device, characterized in that: include: An acquisition module, used to acquire an image to be recognized, a prompt text, and a prompt style image; A first fusion module is used to fuse the image feature vector of the hint style image and the time feature vector of a preset time step to obtain the time style fusion feature vector of the hint style image; A first determination module, configured to determine a style fusion feature vector of the hint style image according to the image feature vector of the hint style image; A second fusion module is used to fuse the image feature vectors of the image to be identified and the prompt style image to obtain an initial style feature vector of the image to be identified; A second determination module, used to determine a target feature vector of the image to be identified based on the image feature vector of the image to be identified; A generation module is used to denoise the noise image through a diffusion model based on the semantic feature vector of the prompt text, the temporal style fusion feature vector of the prompt style image, the style fusion feature vector of the prompt style image, the initial style feature vector of the image to be identified, and the target feature vector of the image to be identified to obtain a target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images