Image generation method and apparatus, and computer device and computer-readable storage medium

Abstract

An image generation method and apparatus, and a computer device and a computer-readable storage medium. The method comprises: acquiring a part image corresponding to each part of a first target object, and acquiring part identification text corresponding to each part image (101); acquiring a reference image including a reference object (102); on the basis of each piece of part identification text, performing part analysis processing on the reference image, so as to obtain feature information of the distribution, in the reference image, of a part indicated by each piece of part identification text, wherein the feature information of the distribution includes the correlation between pixels in the reference image and the part identification text (103); on the basis of an image feature of the part image corresponding to each part, and the feature information of the distribution, performing feature fusion processing, so as to obtain part feature information of each part (104); and on the basis of the part feature information of each part, performing image generation processing, so as to obtain a target image (105). Therefore, the consistency between each part of the first target object in the generated target image and the part in each part image is high, thereby improving an image generation effect.

Description

Image generation method, apparatus, computer device and computer-readable storage medium

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202410815084.8, filed on June 21, 2024, entitled "Image Generation Method, Apparatus, Computer Equipment and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of communication technology, specifically to an image generation method, apparatus, computer device, and computer-readable storage medium, wherein the storage medium is a computer-readable storage medium, and the product is a computer program product. Background Technology

[0004] In the field of Artificial Intelligence Generative Content (AICG), users can obtain the images they want by inputting different text prompts. However, it is impossible to generate different images for the same object by generating different objects that match the text prompts each time. For example, the same object may be in different scenes or the same object may be in different poses. Summary of the Invention

[0005] This disclosure provides an image generation method, apparatus, computer device, and computer-readable storage medium, which can improve the consistency between the parts of the first target object in the generated target image and the parts in the images of each part, thereby improving the image generation effect.

[0006] This disclosure provides an image generation method, including:

[0007] Obtain the part images corresponding to each part of the first target object, and obtain the part identification text corresponding to each part image;

[0008] Obtain the reference image containing the reference object;

[0009] The reference image is analyzed based on the identification text of each part to obtain the distribution feature information of the part indicated by each identification text in the reference image. The distribution feature information includes the correlation between the pixels in the reference image and the identification text of the part.

[0010] Based on the image features of the corresponding body parts and the distribution feature information, feature fusion processing is performed to obtain the body part feature information of each body part;

[0011] Image generation processing is performed based on the feature information of each part to obtain the target image.

[0012] Accordingly, this disclosure also provides an image generation apparatus, comprising:

[0013] The first acquisition unit is configured to acquire the part images corresponding to each part of the first target object, and acquire the part identification text corresponding to each part image;

[0014] The second acquisition unit is configured to acquire a reference image containing the reference object;

[0015] The part analysis unit is configured to perform part analysis processing on the reference image according to each of the part identification texts, and obtain the distribution feature information of the part indicated by each part identification text in the reference image, wherein the distribution feature information includes the correlation between pixels in the reference image and part identification texts;

[0016] The feature fusion unit is configured to perform feature fusion processing based on the image features of the corresponding part images and the distribution feature information to obtain the part feature information of each part;

[0017] The image generation unit is configured to perform image generation processing based on the feature information of each part to obtain the target image.

[0018] Accordingly, this disclosure also provides a computer device including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any of the image generation methods provided in this disclosure.

[0019] Accordingly, embodiments of this disclosure also provide a computer-readable storage medium for storing a computer program, which is loaded by a processor to execute any of the image generation methods provided in embodiments of this disclosure.

[0020] This embodiment of the disclosure obtains part images corresponding to each part of a first target object, and obtains part identification text corresponding to each part image; obtains a reference image containing a reference object; performs part analysis processing on the reference image according to each part identification text to obtain the distribution feature information of the part indicated by each part identification text in the reference image, the distribution feature information including the correlation between pixels in the reference image and part identification text; performs feature fusion processing based on the image features and distribution feature information of the part images corresponding to each part to obtain part feature information of each part; and performs image generation processing based on the part feature information of each part to obtain a target image.

[0021] As can be seen from the above, in this embodiment of the present disclosure, each part of the first target object is processed separately, which can improve the feature extraction capability of each part. Moreover, in the part feature information after the image feature information and distribution feature information are fused, the part that is more related to the part has stronger feature performance. In the image generation process, more attention can be paid to the part related to the part, so that each part of the first target object in the generated target image has high consistency with the parts in each part image, thereby improving the image generation effect. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 is a flowchart of an image generation method provided in one of the embodiments of this disclosure;

[0024] Figure 2 is a schematic flowchart of obtaining a part image provided in one of the embodiments of this disclosure;

[0025] Figure 3 is a schematic flowchart of obtaining distribution feature information provided in one of the embodiments of this disclosure;

[0026] Figure 4 is a schematic diagram of the structure of an image generation model provided in one of the embodiments of this disclosure;

[0027] Figure 5 is a schematic diagram of the image generation process provided in one of the embodiments of this disclosure;

[0028] Figure 6 is a schematic diagram of an image generation apparatus provided in one of the embodiments of this disclosure;

[0029] Figure 7 is a schematic diagram of the structure of a computer device provided in one of the embodiments of this disclosure. Detailed Implementation

[0030] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0031] This disclosure provides an image generation method, apparatus, computer device, and computer-readable storage medium. The image generation apparatus can be integrated into a computer device, which may be a server or a terminal, etc.

[0032] The terminal may include mobile phones, wearable smart devices, tablets, laptops, personal computers (PCs), and in-vehicle computers, etc.

[0033] The server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0034] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0035] This embodiment will be described from the perspective of an image generating device, which can be integrated into a computer device, such as a server or a terminal.

[0036] This disclosure provides an image generation method, as shown in Figure 1. The specific process of this image generation method is as follows:

[0037] 101. Obtain the part images corresponding to each part of the first target object, and obtain the part identification text corresponding to each part image.

[0038] The first target object can include real entities, such as real people or real objects, or virtual objects, such as game characters, cartoon characters, or IP characters.

[0039] The image content of each part image includes the body parts of the first target object. Different part images correspond to different body parts. For example, the first target object can be divided into three basic parts: head, upper body, and lower body, with each part corresponding to one part image. The first object can also be further subdivided into more specific parts, such as arms, legs, torso, and head, with each part corresponding to one part image. The specific settings can be determined based on the actual application scenario and the shape characteristics of the first target object. For example, if the first target object is a game character with very strong arms and a complex arm design, then the first target object can be divided into arms, head, and the remaining parts.

[0040] The part identification text can describe the part of the first object. For example, the part identification text can be "head", "upper body", "lower body", "arm", etc. The part identification text can serve as prompt text to identify the corresponding part from the image.

[0041] The images for each body part can be user-uploaded images. For example, a user can upload an image containing a head, an image containing an upper body, and an image containing a lower body; or an image containing a head, an image containing an arm, an image containing a leg, and an image containing a torso, etc. The user-uploaded images of different body parts can originate from the same or different objects, and the images can also be user-created images. Based on the images of body parts from different objects, the first target object formed by them combines the characteristics of multiple objects, such as a combination of a human head and a fish body, or a dog's head and a human body, etc.

[0042] Optionally, an object image containing at least a portion of the first target object can also be obtained, and the portion images can be extracted from the object image. That is, in one embodiment, the step "obtaining portion images corresponding to each portion of the first target object" includes:

[0043] Obtain an object image that contains at least a portion of the first target object;

[0044] Based on the image content that matches the text markings of each part in the object image, the position indication information of each part of the first target object in the object image is obtained;

[0045] Extract the corresponding part images from the object image based on the location indication information of each part.

[0046] For example, the first target object can be a complete individual, and the object image contains the complete first target object. If the first target object is composed of different parts of multiple objects, then there can be multiple object images. Each object image includes a group of objects that synthesize the first target object (hereinafter referred to as the composed objects for distinction). Based on the image content that matches the part identification text in the object image, the position indication information of each part of the first target object in the object image can be obtained. This can be done by processing each object image. For example, the part identification text corresponding to each object image can be preset. For each object image, the position indication information of the corresponding part can be obtained based on the part identification text.

[0047] You can refer to step 103 to obtain the distribution feature information of the parts indicated by the part identification text in the object image. The distribution feature information includes the correlation between each pixel in the object image and the part identification text, that is, the probability that each pixel is the part indicated by the part identification text. Therefore, the distribution feature information can be used as the location indication information of the parts in the object image.

[0048] The part image can be extracted from the object image based on the pixels whose correlation with the part identification text in the object image indicated by the distribution feature information is greater than a preset correlation threshold.

[0049] Optionally, the distribution feature information can be mapped to 0 or 1 according to a preset correlation threshold to obtain information indicating whether the content presented by the pixel is a part. Here, 0 can represent not a part, 1 can represent a part, or other values. That is, in one embodiment, the step "obtaining the position indication information of each part of the first target object in the object image based on the image content matching the part identifier text of each part in the object image" can include:

[0050] For each part of the identification text, the object image is analyzed based on the part identification text to obtain the distribution feature information of the object image with respect to each part of the identification text;

[0051] Based on a preset correlation threshold, the distribution feature information of the object image with respect to the identification text of each part is mapped to obtain the position indication information of each part of the first target object.

[0052] For example, a preset correlation threshold can be used as a standard to determine whether the image content presented by a pixel is the part indicated by the part identification text. If it is greater than or equal to the preset correlation threshold, the image content presented by the pixel is the part indicated by the part identification text; if it is less than the preset correlation threshold, the image content presented by the pixel is not the part indicated by the part identification text.

[0053] The color values ​​of pixels with a correlation greater than or equal to a preset correlation threshold can be mapped to the color value corresponding to black, and the color values ​​of pixels with a correlation less than the preset correlation threshold can be mapped to the color value corresponding to white, thereby obtaining the position indication information of each part of the first target object.

[0054] Compared to segmenting an object image into multiple part images, based on distribution feature information, each part of the first target object can be accurately extracted from the object image. The resulting part images contain more precise content, reduce content unrelated to the part in the image, and have less interference in the image feature extraction process, thus extracting more accurate image feature information.

[0055] For example, taking the first target object's various parts, including the head, upper body, and lower body, and the corresponding part identification text as "head," "upper body," and "lower body," as shown in Figure 2, the object image and the corresponding part identification text are input into the DMM model. The DMM model inputs the heatmap corresponding to each part as shown in Figure 2. In the heatmap, bright colors indicate high correlation, and dark colors indicate low correlation. The correlation represented by the heatmap is dynamic; the closer the color is to a bright color, the higher the correlation, and the closer the color is to a dark color, the lower the correlation. The location indication information obtained by mapping based on the preset correlation threshold is shown in Figure 2, where the white area is the image area where the part is located. The part image extracted from the object image based on the location indication information is shown in Figure 2.

[0056] Optionally, the content extracted from each object image can be adjusted by adjusting a preset correlation threshold. For example, a lower correlation threshold can be set for more important parts, so that in addition to extracting the key information of the parts, the edge information of the parts can also be extracted, making the generated parts more detailed. For parts of moderate importance, a higher correlation threshold can be set, which can improve the efficiency of extracting part images and the efficiency of generating target images. That is, in one embodiment, the step "based on the preset correlation threshold, matching the correlation distribution information of each part of the object image to obtain the position indication information of each part of the first target object" can include:

[0057] Determine the weights corresponding to each part of the first target object;

[0058] Based on the weights corresponding to each part, the preset correlation thresholds are adjusted to obtain the adjusted correlation thresholds for each part.

[0059] Based on the adjusted correlation threshold, the object image is mapped to various distribution feature information to obtain the position indication information of each part of the first target object.

[0060] The weights assigned to each part indicate its importance. For example, if the generated target image mainly displays the clothing of the first target object, the upper and lower body can be given greater weight than the head. If the generated target image mainly displays the crown of the first target object, the head can be given greater weight than the upper and lower body.

[0061] The larger the weight, the smaller the corresponding adjusted correlation threshold; the smaller the weight, the larger the corresponding adjusted correlation threshold. For example, the weight can be a threshold between [0-1]. The preset correlation threshold is adjusted based on the weight. The difference between the weight and 1 can be multiplied by the preset correlation threshold to obtain the adjusted correlation threshold. The distribution feature information is then mapped based on the adjusted correlation threshold to obtain the position indication information of each part of the first target object.

[0062] Optionally, the object image can be segmented based on the shape and structure of the first target object, dividing the object image into part images that match the identifier text of each part. The shape and structure can indicate the distribution of each part of the first target object, as well as the connection relationships between the parts.

[0063] For example, the position of each part of the object can be determined based on the shape and structure of the first target object, and the object image can be segmented based on the position of each part of the first target object to obtain part images. Each part image contains one part of the first target object.

[0064] In one embodiment, the shape structure can also indicate the proportions of each part of the first target object. Based on the proportions of each part, the object image can be divided into multiple images. Assuming that the ratio of the human head, upper body, and lower body is 1:2:3, the object image can be segmented according to this ratio to obtain part images corresponding to each part, with the height ratio of each part image being 1:2:3.

[0065] Optionally, keypoint recognition can be performed on the object image to identify key points connecting two or more body parts. The object image can then be segmented based on the positions of these key points to obtain corresponding part images for each body part. For example, to divide the human body into the head, upper body, and lower body, human keypoint recognition can be performed on the object image to determine the positions of key points connecting the head and upper body, as well as key points connecting the upper and lower body. The object image can then be segmented based on these key points to obtain corresponding part images for each body part.

[0066] 102. Obtain a reference image containing the reference object.

[0067] The reference object can be an object different from the first target object. The composition of the reference image can serve as a guide for the target image to be generated. The position of the reference object in the reference image serves as a reference for the position of the first target object in the target image to be generated.

[0068] The reference object and the first target object can be objects of the same type, such as both being people or both being cats. The reference object can also be an object of a different type from the first target object, such as the reference object being a person and the first target object being a cat, or the reference object being an elephant and the first target object being a lion.

[0069] If the reference object and the first target object are different types of organisms, then an organism with the characteristics of both organisms can be generated. For example, if the reference object is a human and the first target object is a puppy, then an anthropomorphic dog can be generated.

[0070] 103. Perform part analysis processing on the reference image according to the part identification text of each part to obtain the distribution feature information of the part indicated by each part identification text in the reference image. The distribution feature information includes the correlation between the pixels in the reference image and the part identification text.

[0071] The correlation between pixels and location label text indicates the degree of relevance between the image content displayed by the pixel and the location indicated by the location label text. For example, correlation can be represented by color, and distribution feature information can be a heatmap or grayscale image. Correlation can also be represented numerically, such as using a 0-1 range to represent correlation, where 0 represents no correlation and 1 represents complete correlation. Values ​​between 0 and 1 indicate a correlation between no correlation and complete correlation; the closer to 0, the less correlated; the closer to 1, the more correlated. Other numerical ranges can also be set.

[0072] Among them, part analysis processing can determine the location of the part indicated by the part identification text in the reference image. Part analysis processing can include object detection, text-image attribution analysis, etc.

[0073] For example, the correlation between pixels and location-identifying text can be calculated, and distribution feature information can be generated based on the pixel's position in the reference image and its correlation with the location-identifying text. In one embodiment, the correlation between pixels and location-identifying text can be captured using a neural network model. For instance, a DAAM (Diffusion Attentive Attribution Maps) model can be used. The DAAM model includes a diffusion model and a cross-attention mechanism. By using the diffusion model and cross-attention mechanism in the DAAM model, a heatmap can be generated. The heatmap can indicate the image content in the reference image related to the location-identifying text at the pixel level.

[0074] Taking the first target object's various parts, including the head, upper body, and lower body, and the corresponding part label text as "head," "upper body," and "lower body," as an example, as shown in Figure 3, the reference image and the corresponding part label text are input into the DMM model. The DMM model inputs the heat map corresponding to each part as shown in Figure 3. In the heat map, bright colors indicate high correlation, and dark colors indicate low correlation. The correlation represented by the heat map is dynamic; the closer the color is to a bright color, the higher the correlation, and the closer the color is to a dark color, the lower the correlation.

[0075] For target detection, the parts to be detected in the reference image can be determined based on the part identification text. For each part identification text, a detection box corresponding to the part is detected in the reference image. The image content selected by the detection box is related to the part identification text, while the image content outside the detection box is unrelated to the reference image. The image content at the center of the detection box is more important to the part indicated by the part identification text. The position of the detection box and the positional relationship between the pixels and the detection box can be used to generate distribution feature information. The distribution feature information can be a distribution image of the same size as the reference image. In the distribution image, the pixels corresponding to the image area outside the detection box in the reference image can be set to white or other colors that can indicate that they are unrelated to the part identification text. The color of the part corresponding to the detection box in the distribution image gradually changes from the center outward to indicate that the correlation between the pixel and the part identification text becomes weaker from the center outward. Optionally, other colors can also be used to indicate the correlation.

[0076] The distribution feature information contains the correlation between each pixel in the reference image and the part identification text. By fusing the distribution feature information and the image feature information of the part image, pixel-level precise control of the image feature information can be achieved.

[0077] Optionally, the target image region where the reference object is located in the reference image can be determined. Outside the target image region, the correlation between pixels and part identification text is 0. For pixels within the target image region, the correlation between each pixel in the target image region and the part identification text is calculated. Based on this, distribution feature information can be obtained. In one embodiment, the step "perform part analysis processing on the reference image according to each part identification text to obtain the distribution feature information corresponding to each identification text" may include:

[0078] The target image region containing the reference object is obtained by performing target detection processing on the reference image.

[0079] Based on the correlation between each pixel in the target image region and the identification text of each part, the distribution feature information of the part indicated by the identification text in the reference image is obtained.

[0080] For example, target detection processing can be performed on the reference image to determine the image region where the reference object is located, i.e., the target image region. For the image part outside the target image region in the reference image, the correlation between each pixel and the part identification text is 0. For the target image region in the reference image, the correlation between each pixel in the target image region and the part identification text can be calculated. Combining the correlation between each pixel in the target image region and the part identification text, as well as the correlation between the image part outside the target image region in the reference image, distribution feature information can be obtained.

[0081] 104. Based on the image feature information and distribution feature information of the corresponding part images, feature fusion processing is performed to obtain the part feature information of each part.

[0082] For example, for a target region (one of the various regions), its corresponding distribution feature information indicates the position of the target region of the reference object in the reference image, and also indicates the position of the target region of the first target object in the target image to be generated. The image feature information of the region image includes the features of the target region. By performing feature fusion processing on the image feature information and distribution feature information of the target region, we can obtain region feature information that contains both the position of the target region of the first target object in the target image to be generated and the features of the target region.

[0083] By performing corresponding processing on each part, the part feature information corresponding to each part of the first target object can be obtained.

[0084] In one embodiment, a cross-attention mechanism can be used to perform feature fusion processing on the image feature information and distribution feature information of the part images corresponding to each part. The cross-attention mechanism can establish the connection between the distribution feature information and the part images. The distribution feature information indicates the target region of the part of the first target object in the target image to be generated, while the part images provide information for generating the corresponding part in the target region. The cross-attention mechanism can fuse the image feature information of the part images into the target region indicated by the distribution feature information to obtain part feature information. The obtained part feature information includes both the part features of the target shape part and the features indicating the position of the target shape part in the target image to be generated.

[0085] The distribution feature information contains the correlation between each pixel in the reference image and the part identification text. By fusing the distribution feature information and the image feature information of the part image, precise control of the pixel-level attention mechanism can be achieved.

[0086] In this embodiment of the disclosure, by dividing the first target object into various parts and processing them separately, more accurate feature information describing the features of each part can be obtained, and the first target object has a high degree of restoration in the generated target image.

[0087] 105. Based on the feature information of each part, perform image generation processing to obtain the target image.

[0088] Since the feature information of each part contains the features of the part and its position information in the target image to be generated, a target image of each part of the first target object at a specified position can be generated based on the feature information of each part.

[0089] Optionally, feature fusion processing can be performed on the feature information of each part to obtain object feature information containing the feature information of each part, and the target image can be generated based on the object feature information. For example, the target image can be generated based on the object feature information through a diffusion model.

[0090] Alternatively, a reference image can be used as the original image, and a target image with a scene similar to the reference image can be generated through an image generation model. In one embodiment, the step "image generation processing based on the feature information of each part to obtain the target image" may include:

[0091] The feature information of each part is fused to obtain the object feature information of the first target object;

[0092] The image generation model performs diffusion processing based on the reference image and object feature information to obtain a target image containing the first target object. The scene of the target image is similar to the scene of the reference image.

[0093] The image generation model can include models that can generate corresponding images based on object feature information, such as diffusion models.

[0094] For example, feature fusion processing can be to splice together the feature information of each part to obtain object feature information containing the feature information of each part; or to add the feature information of each part together, that is, to add the elements with the same position in the feature information of each part.

[0095] The reference image can be used as the original image, and the object feature information can be used as noise. The noise is then diffused through an image generation model to generate a target image containing the first target object. Since the object feature information contains the features of the first target object, adding noise to the reference image based on the object feature information allows the first target object to be generated based on the scene in the reference image. Therefore, the scene in the generated target image is similar to that in the reference image.

[0096] There can be one or more first target objects. If there are multiple first target objects, the feature information of each part of each first target object can be fused to obtain object feature information containing the object features of multiple first target objects. Correspondingly, there can be a corresponding number of reference objects in the reference image, or there can be multiple reference images. In multiple reference images, the reference objects are located in different positions in the image, and one reference image is selected as the original image for image generation. Based on the object feature information, a target image containing multiple objects can be generated.

[0097] If there is only one first target object, a second target object can be generated based on the generated target image containing the first target object. That is, in one embodiment, the image generation method provided by this disclosure may further include:

[0098] The image region in the target image to be generated as the second target object is marked to obtain the marked target image;

[0099] The second target object is used as the first target object. Then, the process returns to obtain the part images corresponding to each part of the first target object and the part identification text corresponding to each part image until the part feature information of each part of the second target object is obtained.

[0100] Based on the feature information of each part of the labeled target image and the second target object, the second target object is generated at the labeled image region of the labeled target image.

[0101] For example, a mask can be used to mark the image region in the target image where the second target object is to be generated. Specifically, the target image can be masked according to the region specified by the user. The user uploads a mask image specifying the image region in the target image where the second target object is to be generated. The mask image consists of 0 and 1, where "1" indicates the position of the second target object to be generated in the image. According to the position of 1 in the mask image, the corresponding position in the target image is marked to obtain the marked target image.

[0102] Optionally, the image region in the target image where the second target object is to be generated can be marked using a reference image corresponding to the second target object. The position of the reference object in the reference image corresponding to the second target object can be used as a reference for the position where the second target object is to be generated in the target image. Specifically, refer to step 103 to obtain the distribution feature information of each part of the second target object in the target image. The target image can be marked based on the feature parts in the distribution feature information that meet the correlation conditions (e.g., greater than a preset threshold). For example, the distribution feature information can be a distribution feature map. The image parts in the distribution feature map that meet the correlation conditions can be determined, and the corresponding image regions in the target image can be marked to indicate the image region in the target image where the second target object is to be generated.

[0103] By marking the image region of the second target object to be generated on the target image containing the first target object, the second target object can be generated at the specified location, resulting in an image containing multiple target objects, thus realizing multi-IP generation.

[0104] After marking the image region of the target image to be generated as the second target object, the second target object can be used as the first target object, and the process can return to steps 101-104 to obtain the feature information of each part of the second target object. In step 102, the reference image can be the second reference image corresponding to the second target object. The second reference image is different from the first reference image. The position of the reference object in the second reference image can be used as a reference for the position of the second target object to be generated in the target image.

[0105] By obtaining the feature information of each part of the second target object, an image containing the first and second target objects can be generated based on the target image.

[0106] Understandably, a third target object can be generated on the image containing the first and second target objects. The third target object can be used as the second target object, and the same steps as generating the second target object on the target image can be performed. A third target object can be generated in the image, and a fourth target object, a fifth target object, and so on can also be generated.

[0107] In one embodiment, the model structure of the image generation model can be as shown in Figure 4. The image generation model includes a pre-trained diffusion model, a text extraction network, and a multi-part feature extraction network.

[0108] The text extraction network can include a text encoder and a cross-attention network. The input to the text extraction network can include a text prompt input by the user. The text prompt input by the user can be a scene cue text that guides the image generation model to generate a scene that matches the scene cue text, or it can be text that is adaptively generated by the diffusion model.

[0109] The part feature extraction network includes an image encoder, a linear layer, a normalization layer, and a cross-attention network. The image encoder encodes the part image and distribution feature information. The linear layer performs a linear transformation on the feature information output by the image encoder. After normalization by the normalization layer, the image feature information of the part image is obtained. The image feature information of the part image and the distribution feature information are input into the cross-attention mechanism. The cross-attention network repairs the target region in the distribution feature information based on the image feature information of the part image to obtain the part feature information. For each part, part feature information can be obtained.

[0110] The feature extraction network can be an IP adapter, which enables the diffusion model to generate images using image cues.

[0111] By fusing the feature information of each part through a fusion network, object feature information representing the features of each part of the first target object can be obtained.

[0112] The image generation model includes a decoupled cross-attention module for image processing and a decoupled cross-attention module for text processing. By using the decoupled cross-attention module, text and images can be processed separately, which allows the image generation model to better understand and utilize the features of the image.

[0113] The features of the image and the text are combined and then input into the denoising network of the diffusion model to gradually generate the target image.

[0114] The image encoder can be a pre-trained CLIP (Contrastive Lanquaqe-Image Pre-training) image encoder. The CLIP model is trained on a large number of image and text pairs through contrastive learning. It can understand the image content to extract the features of the object image and generate text descriptions related to the image, enabling the diffusion model to generate images using image cues.

[0115] If there are other control conditions, features can be extracted through the ControlNet module and merged with the part feature information and text feature information in Figure 4 and input into the denoising network to guide the diffusion model to generate the target image. The structure of ControlNet can be the same as the encoder structure of the diffusion model. Through training, different conditions can be controlled for the image generation model.

[0116] The control conditions are used to guide the image generation model to generate specified image content. Therefore, control conditions can be used to control the content of the target image to be generated, such as controlling the pose of the first target object, the scene of the target image, and its style. Control conditions can include text, images, and other forms. Control conditions can include object pose images (i.e., pose control images). Object pose images can be used to guide the image generation model to generate a first target object that makes the pose indicated by the object pose image. The object pose image can be a preset pose image, or it can be obtained by pose recognition based on an image provided by the user, or it can be obtained based on pose instruction text provided by the user. For example, the pose instruction text could be "A man wearing a light green robe and a straw hat, standing with a sword in front of a snow-capped mountain," or "A man standing with a sword," or "An elderly Japanese woman in a kimono, leaning on a cane, looking up at a tree full of blooming cherry blossoms," etc.

[0117] As can be seen from the above, the embodiments of this disclosure obtain part images corresponding to each part of the first target object, and obtain part identification text corresponding to each part image; obtain a reference image containing the reference object; perform part analysis processing on the reference image according to each part identification text to obtain the distribution feature information of the part indicated by each part identification text in the reference image, the distribution feature information including the correlation between pixels in the reference image and part identification text; perform feature fusion processing based on the image features and distribution feature information of the part images corresponding to each part to obtain part feature information of each part; and perform image generation processing based on the part feature information of each part to obtain the target image.

[0118] In this embodiment, each part of the first target object is processed separately, which can improve the feature extraction capability of each part. In the part feature information after the image feature information and distribution feature information are fused, the part that is more related to the part has stronger feature performance. In the image generation process, more attention can be paid to the part related to the part, so that each part of the first target object in the generated target image has high consistency with the parts in each part image, thereby improving the image generation effect.

[0119] Based on the above embodiments, the image generation method provided in this disclosure will be further described below with the first target object as the person. The flow of the image generation method in the embodiments of this disclosure can be shown in Figure 5.

[0120] 1. Obtain the part images of each part of the first target object.

[0121] For example, object description text can be obtained, and an object image containing the first target object can be generated based on the object description text using a text-based graph model.

[0122] Specifically, this could involve obtaining the text input by the user based on the characteristics of the object to be generated, i.e., the object description text; using the object description text as a text prompt; outputting a text-generated graph model; and using the text-generated graph model to generate an object image based on the object description text, with the object image containing the first target object that matches the object description text.

[0123] The object image may include a front image, a side image, and a back image of the first target object, so that one can be selected for generating the model image according to one's own intention.

[0124] If a user wants to generate images of multiple people, they can enter a text prompt for each target object in this step to obtain an image corresponding to each person.

[0125] Contrastive Language-Image Pre-Training (Clip) models or DALL models (DALL-E3 or DALL-E2, etc.) can be used.

[0126] After obtaining the first object image, as shown in Figure 2, the object image and the corresponding part identification text can be input into the DMM model. The DMM model inputs the heat map (i.e. distribution feature map) corresponding to each part as shown in Figure 2. In the heat map, bright colors indicate high correlation and dark colors indicate low correlation. The correlation represented by the heat map is dynamic. The closer the color is to the bright color, the higher the correlation. The closer the color is to the dark color, the lower the correlation.

[0127] The distribution feature map can be one form of distribution feature information in the above embodiments.

[0128] Then, the heatmap is mapped based on a preset correlation threshold to obtain a location indication map as shown in Figure 2, where the white area represents the image region where the location is located. The location indication map can be one form of location indication information from the above embodiments.

[0129] Further, based on location indication information, the part image is extracted from the object image. The extracted part image can be shown in Figure 2.

[0130] 2. Obtain the distribution feature map of each part in the reference image.

[0131] For example, a heatmap can be generated using the DAAM (Diffusion Attentive Attribution Maps) model, which includes a diffusion model and a cross-attention mechanism. The heatmap can indicate the image content in the reference image related to the location identification text at the pixel level.

[0132] Taking the first target object's various parts, including the head, upper body, and lower body, and the corresponding part label text as "head," "upper body," and "lower body," as an example, as shown in Figure 3, the reference image and the corresponding part label text are input into the DMM model. The DMM model inputs the heat map corresponding to each part as shown in Figure 3. In the heat map, bright colors indicate high correlation, and dark colors indicate low correlation. The correlation represented by the heat map is dynamic; the closer the color is to a bright color, the higher the correlation, and the closer the color is to a dark color, the lower the correlation.

[0133] III. Image Generation.

[0134] For example, a cross-attention mechanism can be used to perform feature fusion processing on the image feature information and distribution feature information of the corresponding part images. The cross-attention mechanism can establish the connection between the distribution feature information and the part images. The distribution feature information indicates the target region of the first target object in the target image to be generated, while the part images provide information for generating the corresponding part in the target region. The cross-attention mechanism can fuse the image feature information of the part images into the target region indicated by the distribution feature information to obtain part feature information. The obtained part feature information contains both the part features of the target shape and the features indicating the position of the target shape in the target image to be generated.

[0135] For example, feature fusion processing can be to splice together the feature information of each part to obtain object feature information containing the feature information of each part; or to add the feature information of each part together, that is, to add the elements with the same position in the feature information of each part.

[0136] The reference image can be used as the original image, and the object feature information can be used as noise. The noise is then diffused through an image generation model to generate a target image containing the first target object. Since the object feature information contains the features of the first target object, adding noise to the reference image based on the object feature information allows the first target object to be generated based on the scene in the reference image. Therefore, the scene in the generated target image is similar to that in the reference image.

[0137] The object feature information includes the features of each part of the first target object and its position to be generated. Therefore, the first target object in the target image generated based on the object feature information is consistent with the first target object in the target image, and the position of the first target object in the target image matches the position of the reference object in the target image. The addition of the reference image makes the scene in the generated target image similar to the reference image.

[0138] As can be seen from the above, in this embodiment of the present disclosure, each part of the first target object is processed separately, which can improve the feature extraction capability of each part. In the distribution feature map generated based on the correlation between each pixel in the reference image and the part identification text, the pixels with stronger correlation are displayed more prominently. In the part feature information after the image feature information and distribution feature information are fused, the part that is more related to the part has stronger feature performance. In the image generation process, more attention can be paid to the part related to the part, so that each part of the first target object in the generated target image has high consistency with the parts in each part image, thereby improving the image generation effect.

[0139] To facilitate better implementation of the image generation method provided in this disclosure, an image generation apparatus is also provided in one embodiment. The meanings of the terms used are the same as in the image generation method described above, and specific implementation details can be found in the description of the method embodiments.

[0140] The image generation device can be integrated into a computer device, as shown in Figure 6. The image generation device may include: a first acquisition unit 301, a second acquisition unit 302, a part analysis unit 303, a feature fusion unit 304, and an image generation unit 305, as detailed below:

[0141] (1) The first acquisition unit 301 is configured to acquire the part images corresponding to each part of the first target object, and acquire the part identification text corresponding to each part image.

[0142] In one embodiment, the first acquisition unit 301 includes:

[0143] The image acquisition subunit is configured to acquire an object image containing at least a portion of the aforementioned first target object;

[0144] The matching subunit is configured to perform image content matching with the identification text of each part in the object image to obtain the position indication information of each part of the first target object in the object image.

[0145] The extraction subunit is configured to extract the corresponding part image from the object image based on the position indication information of each part.

[0146] In one embodiment, the matching subunit includes:

[0147] The analysis module is configured to perform part analysis processing on the object image based on the part identification text for each part, and obtain the distribution feature information of the object image with respect to each part identification text.

[0148] The mapping module is configured to perform mapping processing on the distribution feature information of the object image for each part of the identification text based on a preset correlation threshold, so as to obtain the position indication information of each part of the first target object.

[0149] In one embodiment, the mapping module includes:

[0150] The determination submodule is configured to determine the weights corresponding to each part of the first target object mentioned above.

[0151] The adjustment submodule is configured to adjust the preset correlation thresholds according to the weights corresponding to each of the above parts, so as to obtain the adjusted correlation thresholds for each part.

[0152] The processing submodule is configured to perform mapping processing on the object image for each distribution feature information based on the above-mentioned adjusted correlation threshold, so as to obtain the position indication information of each part of the first target object.

[0153] (2) The second acquisition unit 302 is configured to acquire a reference image containing a reference object.

[0154] (3) The part analysis unit 303 is configured to perform part analysis processing on the reference image according to each of the above-mentioned part identification texts, and obtain the distribution feature information of the part indicated by each part identification text in the reference image, wherein the distribution feature information includes the correlation between the pixels in the reference image and the part identification text.

[0155] In one embodiment, the above-mentioned part analysis unit 303 includes:

[0156] The target detection subunit is configured to perform target detection processing on the reference image to obtain the target image region where the reference object is located in the reference image.

[0157] The computational subunit is configured to perform the task of obtaining the distribution feature information of the parts indicated by the part identification text in the reference image based on the correlation between each pixel in the target image region and the part identification text.

[0158] (4) Feature fusion unit 304 is configured to perform feature fusion processing based on the image features of the corresponding part images and the above distribution feature information to obtain the part feature information of each part.

[0159] (5) The image generation unit 305 is configured to perform image generation processing based on the feature information of each part to obtain the target image.

[0160] In one embodiment, the image generation unit 305 includes:

[0161] The part feature fusion subunit is configured to perform feature fusion processing on the part feature information of each part to obtain the object feature information of the first target object mentioned above.

[0162] The diffusion subunit is configured to perform diffusion processing based on the reference image and the object feature information through an image generation model to obtain a target image containing the first target object, wherein the scene of the target image is similar to the scene of the reference image.

[0163] In one embodiment, the image generation apparatus may further include:

[0164] The marking unit is configured to mark the image region in the target image to be generated as the second target object, thereby obtaining the marked target image;

[0165] The update unit is configured to execute the above-mentioned first target object as the second target object, and then return to execute the above-mentioned acquisition of the part images corresponding to each part of the first target object, and acquisition of the part identification text corresponding to each part image, until the part feature information of each part of the second target object is obtained.

[0166] The generation unit is configured to generate the second target object at the marked image region of the marked target image based on the feature information of each part of the marked target image and the second target object.

[0167] As can be seen from the above, the image generation apparatus of this embodiment acquires part images corresponding to each part of the first target object and part identification text corresponding to each part image through the first acquisition unit 301; the second acquisition unit 302 acquires a reference image containing the reference object; the part analysis unit 303 performs part analysis processing on the reference image according to each part identification text to obtain the distribution feature information of the part indicated by each part identification text in the reference image, the distribution feature information including the correlation between pixels in the reference image and part identification text; the feature fusion unit 304 performs feature fusion processing based on the image features and distribution feature information of the part images corresponding to each part to obtain part feature information of each part; and the image generation unit 305 performs image generation processing based on the part feature information of each part to obtain the target image.

[0168] In this embodiment, each part of the first target object is processed separately, which can improve the feature extraction capability of each part. In the part feature information after the image feature information and distribution feature information are fused, the part that is more related to the part has stronger feature performance. In the image generation process, more attention can be paid to the part related to the part, so that each part of the first target object in the generated target image has high consistency with the parts in each part image, thereby improving the image generation effect.

[0169] Accordingly, this disclosure also provides a computer device, which can be a terminal. As shown in FIG7, FIG7 is a schematic diagram of the structure of the computer device provided in this disclosure embodiment. The computer device 500 includes a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, and a computer program stored on the memory 502 and executable on the processor. The processor 501 is electrically connected to the memory 502. Those skilled in the art will understand that the computer device structure shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0170] The processor 501 is the control center of the computer device 500. It connects various parts of the computer device 500 through various interfaces and lines. By running or loading software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, it performs various functions of the computer device 500 and processes data, thereby monitoring the computer device 500 as a whole.

[0171] In this embodiment of the disclosure, the processor 501 in the computer device 500 loads the instructions corresponding to the processes of one or more application programs into the memory 502 according to the following steps, and the processor 501 runs the application programs stored in the memory 502 to realize various functions:

[0172] Obtain the part images corresponding to each part of the first target object, and obtain the part identification text corresponding to each part image;

[0173] Obtain the reference image containing the reference object;

[0174] Based on the identification text of each part, the reference image is analyzed and processed to obtain the distribution feature information of the part indicated by each identification text in the reference image. The distribution feature information includes the correlation between the pixels in the reference image and the identification text of the part.

[0175] Feature fusion processing is performed based on the image features and distribution features of the corresponding body parts to obtain the body part feature information;

[0176] Image generation processing is performed based on the feature information of each part to obtain the target image.

[0177] In one embodiment, the above-mentioned acquisition of the part images corresponding to each part of the first target object can be achieved by the following steps: acquiring an object image containing at least part of the first target object; obtaining position indication information of each part of the first target object in the object image based on the image content that matches the part identification text in the object image; and extracting the part images corresponding to each part from the object image based on the position indication information of each part.

[0178] In one embodiment, the above-mentioned method of obtaining the position indication information of each part of the first target object in the object image based on the image content that matches the part identification text of each part can be achieved through the following steps: for each part identification text, perform part analysis processing on the object image according to the part identification text to obtain the distribution feature information of the object image with respect to each part identification text; based on a preset correlation threshold, perform mapping processing on the distribution feature information of the object image with respect to each part identification text to obtain the position indication information of each part of the first target object.

[0179] In one embodiment, the above-mentioned matching processing of the correlation distribution information of each part of the object image based on a preset correlation threshold to obtain the position indication information of each part of the first target object can be achieved through the following steps: determining the weight corresponding to each part of the first target object; adjusting the preset correlation threshold according to the weight corresponding to each part to obtain the adjusted correlation threshold corresponding to each part; and mapping the object image to each distribution feature information based on the adjusted correlation threshold to obtain the position indication information of each part of the first target object.

[0180] In one embodiment, the above-mentioned image generation processing based on the feature information of each part to obtain the target image can be achieved through the following steps: performing feature fusion processing on the feature information of each part to obtain the object feature information of the first target object; and performing diffusion processing based on the reference image and the object feature information through an image generation model to obtain a target image containing the first target object, wherein the scene of the target image is similar to the scene of the reference image.

[0181] In one embodiment, the above-mentioned part analysis processing of the reference image based on the part identification text to obtain the distribution feature information corresponding to each part identification text can be achieved through the following steps: performing target detection processing on the reference image to obtain the target image region where the reference object is located in the reference image; and obtaining the distribution feature information of the part indicated by each part identification text in the reference image based on the correlation between each pixel in the target image region and each part identification text.

[0182] In one embodiment, the following steps may also be performed: marking the image region in the target image to be used to generate the second target object, thereby obtaining a marked target image; using the second target object as the first target object, and then returning to obtain the part images corresponding to each part of the first target object, and obtaining the part identification text corresponding to each part image, until the part feature information of each part of the second target object is obtained; based on the part feature information of the marked target image and the second target object, generating the second target object at the marked image region in the marked target image.

[0183] As can be seen from the above, in this embodiment of the present disclosure, each part of the first target object is processed separately, which can improve the feature extraction capability of each part. Moreover, in the part feature information after the image feature information and distribution feature information are fused, the part that is more related to the part has stronger feature performance. In the image generation process, more attention can be paid to the part related to the part, so that each part of the first target object in the generated target image has high consistency with the parts in each part image, thereby improving the image generation effect.

[0184] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0185] Optionally, as shown in Figure 7, the computer device 500 further includes a touch screen display 503, a radio frequency circuit 504, an audio circuit 505, an input unit 506, and a power supply 507. The processor 501 is electrically connected to the touch screen display 503, the radio frequency circuit 504, the audio circuit 505, the input unit 506, and the power supply 507. Those skilled in the art will understand that the computer device structure shown in Figure 7 does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0186] The touch display screen 503 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 503 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the computer device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 501. It can also receive and execute commands from the processor 501. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 501 to determine the type of touch event. Subsequently, the processor 501 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 503 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 503 can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 503 can also be used as part of the input unit 506 to achieve input functions.

[0187] The radio frequency circuit 504 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other computer devices, and to transmit and receive signals with network devices or other computer devices.

[0188] Audio circuitry 505 can be used to provide an audio interface between a user and a computer device via a speaker and a microphone. Audio circuitry 505 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 505, converted back into audio data, and output to processor 501 for processing. The audio data is then transmitted via radio frequency circuitry 504 to, for example, another computer device, or output to memory 502 for further processing. Audio circuitry 505 may also include an earphone jack to facilitate communication between peripheral headphones and the computer device.

[0189] The input unit 506 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0190] Power supply 507 is used to supply power to various components of computer device 500. Optionally, power supply 507 can be logically connected to processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 507 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0191] Although not shown in Figure 7, the computer device 500 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0192] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0193] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0194] Therefore, embodiments of this disclosure provide a computer-readable storage medium storing a plurality of computer programs that can be loaded by a processor to execute steps in any of the image generation methods provided in embodiments of this disclosure. For example, the computer program can execute the following steps:

[0195] Obtain the part images corresponding to each part of the first target object, and obtain the part identification text corresponding to each part image;

[0196] Obtain the reference image containing the reference object;

[0197] Based on the identification text of each part, the reference image is analyzed and processed to obtain the distribution feature information of the part indicated by each identification text in the reference image. The distribution feature information includes the correlation between the pixels in the reference image and the identification text of the part.

[0198] Feature fusion processing is performed based on the image features and distribution features of the corresponding body parts to obtain the body part feature information;

[0199] Image generation processing is performed based on the feature information of each part to obtain the target image.

[0200] In one embodiment, the above-mentioned acquisition of the part images corresponding to each part of the first target object can be achieved by the following steps: acquiring an object image containing at least part of the first target object; obtaining position indication information of each part of the first target object in the object image based on the image content that matches the part identification text in the object image; and extracting the part images corresponding to each part from the object image based on the position indication information of each part.

[0201] In one embodiment, the above-mentioned method of obtaining the position indication information of each part of the first target object in the object image based on the image content that matches the part identification text of each part can be achieved through the following steps: for each part identification text, perform part analysis processing on the object image according to the part identification text to obtain the distribution feature information of the object image with respect to each part identification text; based on a preset correlation threshold, perform mapping processing on the distribution feature information of the object image with respect to each part identification text to obtain the position indication information of each part of the first target object.

[0202] In one embodiment, the above-mentioned matching processing of the correlation distribution information of each part of the object image based on a preset correlation threshold to obtain the position indication information of each part of the first target object can be achieved through the following steps: determining the weight corresponding to each part of the first target object; adjusting the preset correlation threshold according to the weight corresponding to each part to obtain the adjusted correlation threshold corresponding to each part; and mapping the object image to each distribution feature information based on the adjusted correlation threshold to obtain the position indication information of each part of the first target object.

[0203] In one embodiment, the above-mentioned image generation processing based on the feature information of each part to obtain the target image can be achieved through the following steps: performing feature fusion processing on the feature information of each part to obtain the object feature information of the first target object; and performing diffusion processing based on the reference image and the object feature information through an image generation model to obtain a target image containing the first target object, wherein the scene of the target image is similar to the scene of the reference image.

[0204] In one embodiment, the above-mentioned part analysis processing of the reference image based on the part identification text to obtain the distribution feature information corresponding to each part identification text can be achieved through the following steps: performing target detection processing on the reference image to obtain the target image region where the reference object is located in the reference image; and obtaining the distribution feature information of the part indicated by each part identification text in the reference image based on the correlation between each pixel in the target image region and each part identification text.

[0205] In one embodiment, the following steps may also be performed: marking the image region in the target image to be used to generate the second target object, thereby obtaining a marked target image; using the second target object as the first target object, and then returning to obtain the part images corresponding to each part of the first target object, and obtaining the part identification text corresponding to each part image, until the part feature information of each part of the second target object is obtained; based on the part feature information of the marked target image and the second target object, generating the second target object at the marked image region in the marked target image.

[0206] As can be seen from the above, in this embodiment of the present disclosure, each part of the first target object is processed separately, which can improve the feature extraction capability of each part. Moreover, in the part feature information after the image feature information and distribution feature information are fused, the part that is more related to the part has stronger feature performance. In the image generation process, more attention can be paid to the part related to the part, so that each part of the first target object in the generated target image has high consistency with the parts in each part image, thereby improving the image generation effect.

[0207] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0208] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0209] The foregoing has provided a detailed description of an image generation method, apparatus, computer device, and computer storage medium provided by the embodiments of this disclosure. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this disclosure. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. An image generation method, comprising: Obtain the part images corresponding to each part of the first target object, and obtain the part identification text corresponding to each part image; Obtain the reference image containing the reference object; The reference image is analyzed based on the identification text of each part to obtain the distribution feature information of the part indicated by each identification text in the reference image. The distribution feature information includes the correlation between the pixels in the reference image and the identification text of the part. Based on the image features of the corresponding body parts and the distribution feature information, feature fusion processing is performed to obtain the body part feature information of each body part; Image generation processing is performed based on the feature information of each part to obtain the target image.

2. The method according to claim 1, wherein, The step of obtaining the part images corresponding to each part of the first target object includes: Obtain an object image that contains at least a portion of the first target object; Based on the image content that matches the text markings of each part in the object image, the position indication information of each part of the first target object in the object image is obtained; Based on the location indication information of each part, extract the corresponding part image from the object image.

3. The method according to claim 2, wherein, The step of obtaining the position indication information of each part of the first target object in the object image based on the image content that matches the part identification text of each part in the object image includes: For each part identification text, the object image is subjected to part analysis processing based on the part identification text to obtain the distribution feature information of the object image with respect to each part identification text; Based on a preset correlation threshold, the distribution feature information of the object image for the identification text of each part is mapped to obtain the position indication information of each part of the first target object.

4. The method according to claim 3, wherein, The step of matching the correlation distribution information of each part of the object image based on a preset correlation threshold to obtain the position indication information of each part of the first target object includes: Determine the weights corresponding to each part of the first target object; Based on the weights corresponding to each part, the preset correlation thresholds are adjusted to obtain the adjusted correlation thresholds for each part. Based on the adjusted correlation threshold, the object image is mapped to various distribution feature information to obtain the position indication information of each part of the first target object.

5. The method according to claim 1, wherein, The image generation process based on the feature information of each part to obtain the target image includes: The feature information of each part is fused to obtain the object feature information of the first target object; An image generation model performs diffusion processing based on the reference image and the object feature information to obtain a target image containing the first target object, wherein the scene of the target image is similar to the scene of the reference image.

6. The method according to claim 1, wherein, The step of performing part analysis processing on the reference image based on the part identification text of each part to obtain the distribution feature information corresponding to each part identification text includes: The reference image is subjected to target detection processing to obtain the target image region where the reference object is located in the reference image; Based on the correlation between each pixel in the target image region and the identification text of each part, the distribution feature information of the part indicated by the identification text in the reference image is obtained.

7. The method according to any one of claims 1-6, wherein, The method further includes: The image region in the target image to be used to generate the second target object is marked to obtain the marked target image; The second target object is used as the first target object, and then the process of obtaining the part images corresponding to each part of the first target object and obtaining the part identification text corresponding to each part image is returned until the part feature information of each part of the second target object is obtained. Based on the feature information of each part of the marked target image and the second target object, the second target object is generated at the marked image area of ​​the marked target image.

8. An image generation apparatus, comprising: The first acquisition unit is configured to acquire the part images corresponding to each part of the first target object, and acquire the part identification text corresponding to each part image; The second acquisition unit is configured to acquire a reference image containing the reference object; The part analysis unit is configured to perform part analysis processing on the reference image according to each of the part identification texts, and obtain the distribution feature information of the part indicated by each part identification text in the reference image, wherein the distribution feature information includes the correlation between pixels in the reference image and part identification texts; The feature fusion unit is configured to perform feature fusion processing based on the image features of the corresponding part images and the distribution feature information to obtain the part feature information of each part; The image generation unit is configured to perform image generation processing based on the feature information of each part to obtain the target image.

9. A computer device comprising a memory and a processor; the memory storing a computer program, and the processor executing the computer program in the memory to perform the image generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium for storing a computer program, the computer program being loaded by a processor to perform the image generation method according to any one of claims 1 to 7.

Images