High-quality human body image generation method and device and computer equipment

Abstract

The invention relates to the technical field of image generation, and discloses a high-quality human body image generation method and device, and computer equipment. The method comprises the following steps: constructing a training data set, wherein the training data set comprises a sample image of a human body, a text description of a corresponding human body attribute, a human body analysis graph and a plurality of attribute tags; an image generation network is constructed, the image generation network comprises an encoder, a UNet and a decoder which are connected in sequence, the image generation network further comprises a time feature aggregation module and an attribute perception reward module, and the attribute perception reward module is used for predicting a target reward score and a prediction reward score; training an image generation network by minimizing the difference between the target reward score and the predicted reward score; and inputting the text description of the human body image to be processed and the human body analysis graph into the trained image generation network to obtain a target image. By adopting the method, the high-quality human body image with space alignment and consistent attributes can be generated.

Description

Technical Field

[0001] This application relates to the field of image generation technology, and more specifically, to a method, apparatus, and computer device for generating high-quality human body images. Background Technology

[0002] Controllable generation of high-quality human images is a fundamental yet highly challenging task. However, current generative models fall short in achieving accurate spatial alignment and attribute consistency in the generated images. These limitations lead to mismatches between the generated human images and the expected structural and semantic conditions, especially when dealing with complex attribute text descriptions. Furthermore, the lack of fine-grained attribute annotations and multimodal data in current human image datasets restricts progress in spatial alignment and attribute consistency.

[0003] Latent Diffusion Models (LDMs) have become a powerful generative framework. Through improvements in training methods and sampling strategies, LDMs have achieved remarkable results in image generation. LDMs further enhance generation efficiency by shifting the denoising process to the latent space. To achieve conditional control in text-to-image LDMs, several studies have proposed more precise spatial control methods. ControlNet and T2IAdapter introduce additional trainable modules for guiding image generation. Despite these advances, ensuring that the generated image remains aligned with the input conditions remains challenging.

[0004] To improve the alignment between generated images and control inputs, ControlNet++ proposes a reward-based loss function that explicitly optimizes pixel-level loop consistency during training.

[0005] However, these methods still have the following drawbacks: they mainly focus on coarse-grained spatial control and have not yet achieved fine-grained semantic or attribute alignment; it is difficult to achieve explicit attribute consistency (especially at the level of visual detail). Summary of the Invention

[0006] To address the shortcomings of current human image generation methods in terms of spatial alignment and attribute consistency, this invention provides a high-quality human image generation method, apparatus, and computer equipment that ensures the generated human image matches the attributes of the text description.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a method for generating high-quality human images is provided, the method comprising: Construct a training dataset, which includes sample images of the human body and corresponding annotation information. The annotation information includes text descriptions of human body attributes, human body analysis diagrams, and multiple attribute labels. An image generation network is constructed, comprising an encoder, a UNet, and a decoder connected sequentially, and a temporal feature aggregation module connected to the UNet. The encoder receives sample images; the UNet receives text descriptions and human anatomy diagrams corresponding to the sample images, and extracts input features through its downsampling and bottleneck layers using hook functions; the temporal feature aggregation module processes the input features with temporal step embedding as a condition to obtain output features, and returns the output features to the upsampling layer of the UNet; the decoder obtains the predicted image based on the output of the upsampling layer. The image generation network also includes an attribute-aware reward module, which includes an attribute encoder, an image feature extraction module, and a reward score calculation module. The attribute encoder is used to obtain attribute embeddings based on the attribute labels corresponding to the sample images. The image feature extraction module is used to output image features based on the sample images and output predicted image features based on the predicted images. The reward score calculation module is used to predict the target reward score based on the attribute embeddings and image features, and predict the predicted reward score based on the attribute embeddings and predicted image features. The image generation network is trained by minimizing the difference between the target reward score and the predicted reward score. The text description and human body analysis image of the human body to be processed are input into the trained image generation network to obtain the target image.

[0008] Furthermore, the above-mentioned high-quality human image generation method also includes obtaining time step embedding based on a given time step and an encoding function; the encoding function includes sinusoidal embedding and multilayer perceptron.

[0009] Furthermore, the temporal feature aggregation module is used to process the input features with time step embedding as a condition to obtain output features. This includes projecting the input features; modulating the projected features and the time step embedding through multiplication; and concatenating the modulated features and the input features through residuals to obtain the output features.

[0010] Furthermore, the attribute encoder is also used to convert each attribute label corresponding to the sample image into a dense vector; obtain the attribute tensor based on the dense vector; and obtain the attribute embedding based on the attribute tensor.

[0011] Furthermore, the reward score calculation module is also used to project image features into the same dimensional space as the attribute embedding to obtain fused features; project both the fused features and the attribute embedding into the same shared embedding space to obtain semantic alignment representation; project the predicted image features into the same dimensional space as the attribute embedding to obtain predicted fused features; project both the predicted fused features and the attribute embedding into the same shared embedding space to obtain predicted semantic alignment representation; obtain the target reward score based on the semantic alignment representation; and obtain the predicted reward score based on the predicted semantic alignment representation.

[0012] Furthermore, the attribute encoder and reward score calculation module both employ multilayer perceptrons, while the image feature extraction module uses the ResNet-50 model.

[0013] Furthermore, the image generation network also includes a noise-adding module and a noise-reducing module. The decoder is also used to obtain a noisy prediction image based on the output of the upsampling layer. The noise-adding module is used to add Gaussian noise to the image output by the encoder to obtain a noisy image, and send the noisy image to the downsampling layer of UNet. The noise-reducing module is used to obtain prediction noise based on the noisy image, a given time step, and text description and human body parsing image. The prediction noise is removed from the noisy prediction image, and the prediction image is output.

[0014] According to a second aspect of the present invention, a high-quality human body image generation apparatus is also provided, comprising: The dataset construction module is used to construct the training dataset, which includes sample images of the human body and corresponding annotation information. The annotation information includes text descriptions of human body attributes, human body analysis diagrams, and multiple attribute labels. The image generation network construction module includes an encoder, a UNet, and a decoder connected in sequence, and a temporal feature aggregation module connected to the UNet. The encoder receives sample images; the UNet receives text descriptions and human anatomy diagrams corresponding to the sample images, and extracts input features through its downsampling and bottleneck layers using hook functions; the temporal feature aggregation module processes the input features with temporal step embedding as a condition to obtain output features, and returns the output features to the upsampling layer of the UNet; the decoder obtains the predicted image based on the output of the upsampling layer. The image generation network also includes an attribute-aware reward module, which includes an attribute encoder, an image feature extraction module, and a reward score calculation module. The attribute encoder is used to obtain attribute embeddings based on the attribute labels corresponding to the sample images. The image feature extraction module is used to output image features based on the sample images and output predicted image features based on the predicted images. The reward score calculation module is used to predict the target reward score based on the attribute embeddings and image features, and predict the predicted reward score based on the attribute embeddings and predicted image features. The image generation network training module is used to train the image generation network by minimizing the difference between the target reward score and the predicted reward score; The target image generation module is used to input the text description and human body analysis map of the human body image to be processed into the trained image generation network to obtain the target image.

[0015] According to a third aspect of the invention, a computer device is also provided, comprising a memory, a processor, and a computer program stored in the memory, the processor executing the computer program to implement the steps of any of the methods described above.

[0016] According to a fourth aspect of the invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the above methods.

[0017] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: This invention provides a high-quality human image generation method. First, a multimodal dataset is constructed. Then, within the framework of existing image generation networks, spatial control and attribute-level supervision are unified to generate high-quality human images that are both spatially aligned and attribute-consistent. Specifically, by combining temporal step embedding with UNet's downsampling features, progressive spatial control from coarse to fine is achieved, ensuring the spatial alignment of the generated human images. An attribute-aware reward module serves as fine-grained attribute-level supervision for the entire image generation network, ensuring that the attributes in the generated human images are consistent with those in the text description and correspond to the human anatomy map. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a high-quality human image generation method provided in this application embodiment; Figure 2 This is an overall architecture diagram of a high-quality human image generation method provided in an embodiment of this application; Figure 3 An architecture diagram of the time feature aggregation module provided in the embodiments of this application; Figure 4 An architecture diagram of the attribute-aware reward module provided in an embodiment of this application; Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0022] like Figure 1 As shown, a method for generating high-quality human images is provided. This method can be executed by a terminal or by a server communicating with the terminal via a network. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The server can be a standalone server or a server cluster consisting of multiple servers. Taking the application of this method to a terminal as an example, the steps include: Step 101: Construct a training dataset. The training dataset includes sample images of the human body and corresponding annotation information. The annotation information includes text descriptions of human body attributes, human body analysis diagrams, and multiple attribute labels.

[0023] Step 102: Construct an image generation network, which includes an encoder, a UNet, and a decoder connected in sequence, and also includes a temporal feature aggregation module connected to the UNet. The encoder is used to receive sample images. The UNet is used to receive the text description and human body analysis image corresponding to the sample images, and uses hook functions to extract input features through its downsampling layer and bottleneck layer. The temporal feature aggregation module is used to process the input features with time step embedding as a condition to obtain output features, and returns the output features to the upsampling layer of the UNet. The decoder is used to obtain the predicted image based on the output of the upsampling layer.

[0024] For example, a time-step embedding is obtained based on a given time step and an encoding function; the encoding function includes sinusoidal embedding and multilayer perceptron. A time feature aggregation module is used to project the input features; the projected features are modulated with the time-step embedding through multiplication; the modulated features and the input features are concatenated through residuals to obtain the output features.

[0025] Step 103: The image generation network further includes an attribute-aware reward module, which includes an attribute encoder, an image feature extraction module, and a reward score calculation module. The attribute encoder is used to obtain attribute embeddings based on the attribute labels corresponding to the sample images. The image feature extraction module is used to output image features based on the sample images and output predicted image features based on the predicted images. The reward score calculation module is used to predict the target reward score based on the attribute embeddings and image features, and predict the predicted reward score based on the attribute embeddings and predicted image features.

[0026] The attribute encoder and reward score calculation module both use multilayer perceptrons, while the image feature extraction module uses the ResNet-50 model.

[0027] For example, the attribute encoder is also used to convert each attribute label corresponding to the sample image into a dense vector; obtain an attribute tensor based on the dense vector; and obtain the attribute embedding based on the attribute tensor.

[0028] The reward score calculation module is also used to project image features into the same dimensional space as the attribute embedding to obtain fused features; project both the fused features and the attribute embedding into the same shared embedding space to obtain semantic alignment representation; project the predicted image features into the same dimensional space as the attribute embedding to obtain predicted fused features; project both the predicted fused features and the attribute embedding into the same shared embedding space to obtain predicted semantic alignment representation; obtain the target reward score based on the semantic alignment representation; and obtain the predicted reward score based on the predicted semantic alignment representation.

[0029] Step 104: Train the image generation network by minimizing the difference between the target reward score and the predicted reward score.

[0030] Step 105: Input the text description and human body analysis image of the human body image to be processed into the trained image generation network to obtain the target image.

[0031] The target image is the final generated human body image, which is spatially aligned and has consistent attributes, i.e., a high-quality human body image.

[0032] For example, a test dataset is constructed; the test dataset includes test images of the human body and corresponding annotation information; the text description and human body analysis map of a test image in the test dataset are input into the trained image generation network to obtain the target image corresponding to the test image.

[0033] The aforementioned high-quality human image generation method first constructs a multimodal dataset, and then unifies spatial control and attribute-level supervision within the framework of existing image generation networks to generate spatially aligned and attribute-consistent high-quality human images. Specifically, by constructing a temporal feature aggregation module, temporal step embedding is combined with UNet's downsampling features to achieve progressive spatial control from coarse to fine, ensuring the spatial consistency of the generated target images; by constructing an attribute-aware reward module as fine-grained attribute-level supervision of the model, the attributes in the generated target images are ensured to be consistent with the attributes in the text description.

[0034] In one embodiment, the image generation network further includes a noise-adding module and a noise-reducing module. The decoder is further configured to obtain a noisy prediction image based on the output of the upsampling layer. The noise-adding module is configured to add Gaussian noise to the image output by the encoder to obtain a noisy image and send the noisy image to the downsampling layer of the UNet. The noise-reducing module is configured to obtain prediction noise based on the noisy image, a given time step, and text description and human body parsing image. The prediction noise is removed from the noisy prediction image, and the prediction image is output.

[0035] In one embodiment, such as Figure 2 As shown, the architecture of a high-quality human image generation method includes the T2Human-MultiModal dataset, a Time-Feature Aggregation Module (TFAM), an Attribute-Aware Reward Module (AARM), and a denoising module.

[0036] Generating the T2Human - MultiModal dataset involves the following steps: Step 1: Select high-quality full-body human images from the public dataset CosmicMan-HQ 1.0 that meet the preset conditions. The preset conditions can be a clean background and complete human structure, so as to obtain a large-scale, high-quality multimodal T2Human - MultiModal dataset containing the whole human body.

[0037] Ultimately, 25,152 images were selected as the base material for the dataset.

[0038] Step 2: Preprocess the full-body human images that meet the prediction conditions, remove low-quality, half-body or incomplete human body samples, and ensure that all images in the dataset are full-body human images that meet the requirements, with a uniform resolution of 1024×512.

[0039] Step 3: Perform multi-dimensional annotation on each preprocessed image to generate annotation information; the annotation information includes text descriptions of human attributes (gender, hairstyle, clothing style) (also known as human attribute text), human body analysis diagrams, dense human posture information, human joint position information, human attribute labels, etc.

[0040] Among them, (1) text description: generate detailed text descriptions covering details such as human gender, hairstyle, and clothing style through the DeepSeek-r1-32B model; (2) human body parsing map: generate human semantic segmentation map for spatial control; (3) Densepose annotation: generate dense human pose information using the Detectron2 tool; (4) key point annotation: detect and filter human key points through the YOLO model to obtain human joint position information; (5) fine-grained human body attribute labels: label attribute labels in the form of key-value pairs such as gender, skin color, clothing material, and shoe type, such as {"gender": "female","ethnicity": "Caucasian", "age": "adult"}{"hair color": "brown", "hair style": "wavy", "hair length": "above nchest"},{"type": "blouse", "pattern": "solid color", "material": "silk", "sleeve length": "long sleeve", "toplength": "normal", "collar shape": "v neckline", "color": "white"}{"type": "skirt", "pattern": "solid color", "material": "silk", "length": "high-low", "bottom shape": "flare", "color": "green"}{"type": "high heels", "pattern": "solid color", "material": "leather", "boots length": "ankle boots", "color":"black"}{"pattern": "solid color", "material": "cotton"}{"pattern": "solidcolor", "material": "cotton"}.

[0041] Step 4: Divide all labeled images proportionally into a training dataset (23,152 samples) and a test dataset (2,000 samples).

[0042] Building a time feature aggregation module involves the following steps: Step 5, as follows Figure 3 As shown, based on a given time step and encoding function The time-step embedding is obtained; the encoding function consists of sinusoidal embedding and multilayer sensing mechanism.

[0043] For example, the time step embedding is calculated according to the following formula:

[0044]

[0045] in, Describes the multilayer perceptron function. Indicates time step embedding, Represents the time step vector. For time steps, The input dimension is the sinusoidal embedding. Step 6: Extract input features from the downsampling layers (UNet has three downsampling layers) and bottleneck layers of U-Net using hook functions. (i.e., the first feature, the second feature, the third feature, and the fourth feature); for input features Embedded by time step The conditions are processed to obtain the output features. .

[0046] Specifically, such as Figure 3 As shown, for input features Perform projection; embed the projected features with the time steps. Modulation is performed using multiplication; the modulated features are then optimized; finally, the optimized features and the input features are concatenated using residuals to obtain the output features. .

[0047] For example, the final output is calculated according to the following formula. :

[0048] in, By one It consists of a convolution and a ReLU activation function. By one Convolution, one It consists of convolution and two ReLU activation functions.

[0049] Building an attribute-aware reward module involves the following steps: Step 7: Convert attribute tags into dense vectors Each data sample contains a list of attribute labels. Each of them It is a key-value pair.

[0050] Each Encoded as a fixed-dimensional vector ,in For vector dimensions; compute each key Sum The hash values ​​are then assigned to vectors using the following formula. In the alternating positions:

[0051]

[0052] All vectors are stacked to form a complete attribute tensor:

[0053] Input A into a shared multilayer perceptron to obtain attribute embeddings. .

[0054] Step 8, as follows Figure 4 As shown, the image Input pre-trained ResNet-50 model output image features Simultaneously, attribute label A is input into the attribute encoder to obtain the attribute embedding. .

[0055] Through the feature projection module Image features Projection to and attribute embedding In the same dimensional space, the formula is as follows:

[0056] The features obtained afterwards Let Q be the query term, and embed the attribute. As a key ( ) and value ( ), and input it into the cross-attention module to obtain fused features. The formula is as follows:

[0057]

[0058] in, for The dimension of a vector.

[0059] Finally, the fused features and attribute embeddings are projected into the same shared embedding space to obtain a semantically aligned representation for reward prediction: .

[0060] The attribute reward score is calculated by the Reward Score Calculation (RSC) module, which is implemented using Multilayer Perceptrons (MLPs). The final reward calculation formula is as follows:

[0061] Building a noise reduction module involves the following steps: Step 9, a clear image output to the encoder. Gaussian noise is gradually added to model the forward diffusion process, as shown in the following formula:

[0062] in, and , This is a noisy image.

[0063] The goal of the diffusion model is to: based on noisy images Given the time step t and input conditions, the noise is predicted. The input criteria include a textual description of human body attributes. Human body analysis diagram .

[0064] The objective function for training the diffusion model is shown below:

[0065] in, It is a noise prediction model defined by parameter θ, and is the core training object of the diffusion model. It is a random noise vector that follows a standard normal distribution. This represents the square of the L2 norm, which is used to calculate the true noise. With model prediction noise The mean square error (MSE) between them. This represents the loss value. The smaller the loss value, the closer the noise predicted by the model is to the actual added noise. When denoising based on this predicted noise, the original data can be recovered more accurately. This indicates taking the expected value of the combination of random variables within the parentheses. Essentially, it aims to ensure that the model can learn stably under all possible input conditions, avoiding overfitting to a certain type of sample or a certain step.

[0066] Training an image generation network involves the following steps: Define the loss function for the attribute-aware reward module: , in, This represents the prediction reward score calculated by the attribute-aware reward module based on the attribute labels of the generated prediction image and the input sample image. It is used to measure the degree of matching between the generated prediction image and the human attributes of the input sample image. The target reward score is calculated by the attribute-aware reward module based on the input sample image and its attribute labels.

[0067] The image generation network is trained by minimizing the difference between the target reward score and the predicted reward score.

[0068] Using an image generation network involves the following steps: The text descriptions and human anatomy diagrams of the images in the test dataset are input into the trained image generation network to obtain the target images corresponding to the test images.

[0069] This embodiment provides an image generation method that utilizes temporal feature aggregation and attribute-aware reward. The temporal feature aggregation module combines temporal step embedding with UNet's downsampling features to achieve progressive spatial control from coarse to fine, ensuring the spatial alignment of the generated image. The attribute-aware reward module serves as fine-grained attribute-level supervision of the entire image generation network, ensuring that the attributes in the generated image are consistent with the attributes in the text description and correspond to the human body analysis image.

[0070] In one embodiment, a high-quality human image generation device is also provided, including a dataset construction module, an image generation network construction module, an image generation network training module, and a target image generation module.

[0071] The dataset construction module is used to construct a training dataset, which includes sample images of the human body and corresponding annotation information. The annotation information includes text descriptions of human body attributes, human body analysis diagrams, and multiple attribute labels.

[0072] The image generation network construction module includes an encoder, a UNet, and a decoder connected in sequence, and a temporal feature aggregation module connected to the UNet. The encoder is used to receive sample images; the UNet is used to receive the text description and human body analysis image corresponding to the sample images, and uses hook functions to extract input features through its downsampling layer and bottleneck layer; the temporal feature aggregation module is used to process the input features with time step embedding as a condition to obtain output features, and returns the output features to the upsampling layer of the UNet; the decoder is used to obtain the predicted image based on the output of the upsampling layer.

[0073] The image generation network also includes an attribute-aware reward module, which includes an attribute encoder, an image feature extraction module, and a reward score calculation module. The attribute encoder is used to obtain attribute embeddings based on the attribute labels corresponding to the sample images. The image feature extraction module is used to output image features based on the sample images and output predicted image features based on the predicted images. The reward score calculation module is used to predict the target reward score based on the attribute embeddings and image features, and predict the predicted reward score based on the attribute embeddings and predicted image features.

[0074] The image generation network training module is used to train the image generation network by minimizing the difference between the target reward score and the predicted reward score.

[0075] The target image generation module is used to input the text description and human body analysis map of the human body image to be processed into the trained image generation network to obtain the target image.

[0076] This application also provides a computer device, the internal structure diagram of which can be shown as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for generating high-quality human images.

[0077] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0078] like Figure 5 As shown, this application also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the above-described method embodiments.

[0079] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0080] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0081] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0082] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0084] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0085] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0086] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0087] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0088] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating high-quality human body images, characterized in that, include: Construct a training dataset, which includes sample images of the human body and corresponding annotation information, including text descriptions of human body attributes, human body analysis diagrams, and multiple attribute labels; An image generation network is constructed, which includes an encoder, a UNet and a decoder connected in sequence, and a temporal feature aggregation module connected to the UNet; The encoder is used to receive the sample image; the UNet is used to receive the text description and human body analysis image corresponding to the sample image, and uses a hook function to extract input features through its downsampling layer and bottleneck layer; The temporal feature aggregation module processes the input features with temporal step embedding as a condition to obtain output features, and returns the output features to the upsampling layer of the UNet; the decoder obtains the predicted image based on the output of the upsampling layer. The image generation network further includes an attribute-aware reward module, which comprises an attribute encoder, an image feature extraction module, and a reward score calculation module. The attribute encoder is used to obtain attribute embeddings based on the attribute labels corresponding to the sample image. The image feature extraction module is used to output image features based on the sample image and output predicted image features based on the predicted image. The reward score calculation module is used to predict the target reward score based on the attribute embeddings and the image features, and predict the predicted reward score based on the attribute embeddings and the predicted image features. The image generation network is trained by minimizing the difference between the target reward score and the predicted reward score; The text description and human body analysis image of the human body to be processed are input into the trained image generation network to obtain the target image.

2. The high-quality human image generation method as described in claim 1, characterized in that, The method further includes: Based on a given time step and an encoding function, the time step embedding is obtained; the encoding function includes sinusoidal embedding and multilayer perceptron.

3. The high-quality human image generation method as described in claim 1, characterized in that, The time feature aggregation module is also used to project the input features; The projected features and time step embeddings are modulated by multiplication. The modulated features and the input features are concatenated via residuals to obtain the output features.

4. The method for generating high-quality human images as described in claim 1, characterized in that, The attribute encoder is further configured to convert each attribute label corresponding to the sample image into a dense vector; obtain an attribute tensor based on the dense vector; and obtain an attribute embedding based on the attribute tensor.

5. The high-quality human image generation method as described in claim 4, characterized in that, The reward score calculation module is also used for: The image features are projected into the same dimensional space as the attribute embedding to obtain fused features; the fused features and the attribute embedding are both projected into the same shared embedding space to obtain semantically aligned representations. The predicted image features are projected into the same dimensional space as the attribute embedding to obtain the predicted fusion features; the predicted fusion features and the attribute embedding are both projected into the same shared embedding space to obtain the predicted semantic alignment representation. The target reward score is obtained based on the semantic alignment representation. The predicted reward score is obtained based on the predicted semantic alignment representation.

6. The method for generating high-quality human images as described in claim 1, characterized in that, Both the attribute encoder and the reward score calculation module employ a multilayer perceptron, while the image feature extraction module uses a ResNet-50 model.

7. The method for generating high-quality human images as described in claim 2, characterized in that, The image generation network also includes a noise-adding module and a noise-reducing module, and the decoder is further used to obtain a noisy predicted image based on the output of the upsampling layer; The noise-adding module is used to add Gaussian noise to the image output by the encoder to obtain a noisy image, and send the noisy image to the downsampling layer of the UNet; The denoising module is used to obtain predicted noise based on the noisy image, the given time step, the text description, and the human body analysis image; remove the predicted noise from the noisy predicted image, and output the predicted image.

8. A high-quality human body image generation device, characterized in that, include: The dataset construction module is used to construct a training dataset, which includes sample images of the human body and corresponding annotation information. The annotation information includes text descriptions of human body attributes, human body analysis diagrams, and multiple attribute labels. An image generation network construction module, wherein the image generation network includes an encoder, a UNet and a decoder connected in sequence, and also includes a temporal feature aggregation module connected to the UNet; The encoder is used to receive the sample image; the UNet is used to receive the text description and human body analysis image corresponding to the sample image, and uses a hook function to extract input features through its downsampling layer and bottleneck layer; The temporal feature aggregation module processes the input features with temporal step embedding as a condition to obtain output features, and returns the output features to the upsampling layer of the UNet; the decoder obtains the predicted image based on the output of the upsampling layer. The image generation network further includes an attribute-aware reward module, which comprises an attribute encoder, an image feature extraction module, and a reward score calculation module. The attribute encoder is used to obtain attribute embeddings based on the attribute labels corresponding to the sample image. The image feature extraction module is used to output image features based on the sample image and output predicted image features based on the predicted image. The reward score calculation module is used to predict the target reward score based on the attribute embeddings and the image features, and predict the predicted reward score based on the attribute embeddings and the predicted image features. An image generation network training module is used to train the image generation network by minimizing the difference between the target reward score and the predicted reward score; The target image generation module is used to input the text description and human body analysis map of the human body image to be processed into the trained image generation network to obtain the target image.

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the high-quality human image generation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the high-quality human image generation method according to any one of claims 1-7.

Images