Human body posture migration method based on image redrawing

Abstract

The invention discloses a human body posture migration method based on image redrawing, and belongs to the field of artificial intelligence image generation. Separating the background and the human body appearance in the source image through a foreground and background separation technology; a posture encoder is used for obtaining target human body posture features to guide a source human body to conduct posture migration; rich human body detail features are obtained through a human body appearance extraction network; a reference attention mechanism is used to guide an image redrawing network based on a diffusion model, and human body appearance and background features are effectively fused; and outputting a final generated image. According to the method, the naturalness and detail expression of human body posture migration are optimized by independently processing the human body and background parts of the image and utilizing the strong image generation capability of the diffusion model. According to the method, the consistency of the background can be kept in the human body posture migration, meanwhile, a high-quality image with real details is generated, and the authenticity and continuity of the visual effect are improved; in addition, the background area in the image can be replaced while the human body posture migration is realized.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence image generation, and in particular relates to a human body posture migration method based on image redrawing. Background Art

[0002] Human pose transfer is a challenging task in the field of computer vision. This task mainly refers to transferring the human body in the source human image from the source pose to any target pose. The related technology has great application value in artistic creation and generating training data for downstream tasks, so it has become one of the most popular research directions in recent years.

[0003] In the process of human pose transfer, the source human image needs to undergo a variety of changes, including complex spatial changes and invisible area generation. For the complex spatial changes in the process of human pose transfer, early methods tried to simulate such spatial changes through geometric changes and flow changes to achieve human pose transfer, but these methods often performed poorly when faced with the transfer between complex poses. Later, it was proposed to simulate such changes through attention mechanisms. Thanks to the global vision of the attention mechanism, these methods can well achieve the transfer between complex poses. For the generation of invisible parts in the process of human pose transfer, previous methods mainly rely on GAN. Recently, diffusion models have shown significant image generation capabilities in the fields of text-to-image, image-to-image conversion, and image editing. Therefore, diffusion models have begun to be used to solve the problem of invisible area generation in human pose transfer. For example, DreamPose and MagicDance, two human pose transfer methods, have shown excellent performance in human pose transfer tasks with the help of the powerful generation capabilities of diffusion models. However, these methods do not take into account the obvious changes in the human and background parts of the source human image during the process of human pose transfer. Specifically, the background of the source human image often only involves the generation of invisible areas during the entire human posture transfer process. However, the human body part of the source human image involves complex spatial changes in addition to the generation of invisible areas, that is, the originally visible human body parts need to undergo complex geometric changes in geometric position and shape. In order to improve the human appearance and posture effects of the generated image, these methods usually use neural networks to perform complex spatial transformations on the entire image. Although these transformations help the posture transformation of human body parts, they may also have unexpected effects on the background area, thereby introducing unrealistic details in the background of the generated image. A few methods, such as Disco, separate the human body and background in the source human image and use different mechanisms to process them to avoid the influence of complex geometric transformations of human body parts on the background area. However, due to the design of their models, they still have shortcomings in the consistent generation of human appearance and background. Summary of the invention

[0004] In view of the fact that the existing human posture transfer methods do not fully consider the obvious changes in the appearance of the human body and the background in the source image during the transfer process, the present invention proposes a human posture transfer method based on image redrawing, which regards human posture transfer as an image redrawing task and defines this process as redrawing a human body image with specified posture and appearance on a background image.

[0005] Technical solution: In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] A human posture transfer method based on image redrawing independently processes the human body and background parts of the image, and uses the image generation ability of the diffusion model to redraw the human body image with specified posture and appearance on the background image to generate a human posture transfer image with high consistency, thereby optimizing the naturalness and detail expression of human posture transfer. Specifically, it includes the following steps:

[0007] S1: Separate the background and human appearance in the source image through foreground and background separation technology;

[0008] S2: Use the posture encoder to extract the target human posture features to accurately guide the source human posture transfer;

[0009] S3: Obtain rich human detail features through human appearance extraction network;

[0010] S4: Use reference attention mechanism to guide diffusion model-based image repainting network to effectively integrate human appearance and background features;

[0011] S5: Output the final generated image.

[0012] Preferably, in step S1, the specific content of separating the background and the human body appearance in the source image by the foreground and background separation technology is:

[0013] First, the DINO model and the SAM model are used to obtain the mask M of the corresponding human body region from the source human body image. Then, the mask M of the human body region is used to separate the source human body appearance and the background to obtain the source human body appearance image I. App and the corresponding background image I Back .

[0014] Preferably, in step S2, the specific content of using the posture encoder to extract the posture features of the target human body to accurately guide the source human body to perform posture migration is:

[0015] First, the target human posture image I pose Input into a lightweight posture encoder composed of 5 layers of convolutional layers and corresponding activation functions SiLU to obtain the corresponding target human posture feature map F pose , and finally the target human posture feature map Fpose The data are respectively input into the human appearance extraction network and the diffusion model-based image redrawing network to accurately guide the source human posture migration. The accurate posture feature extraction in step S2 lays a solid foundation for the subsequent human appearance detail restoration and posture migration.

[0016] Preferably, in step S3, the specific content of obtaining rich human detail features through the human appearance extraction network is:

[0017] S3.1: A human appearance extraction network based on ResnetBlock and TransformerBlock is introduced. The unet specifically includes an input layer composed of convolutional layers and a feature extraction module composed of several ResnetBlocks and TransformerBlocks.

[0018] S3.2: Through the VAE encoder part of the stable diffusion, the human appearance image I App Mapping from pixel space to low-dimensional latent space as the latent space variable ε(I App ).

[0019] S3.3: The random noise Z t and the latent space variable ε(I App ) and the scaled human body area mask M are used as the input of the human appearance extraction network after channel splicing, as follows:

[0020]

[0021] Among them, γ represents the input of the human appearance extraction network, Z t represents random noise, M represents the mask of the human body area, and I App represents the source human appearance image, ε(I App ) indicates I App The latent space variables of human appearance after mapping to the latent space, It means that the final input dimension is 9×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0022] S3.4: The human body posture feature map F obtained in step S2 pose The matrix elements are added to the output of the input layer of the human appearance extraction network and used as the input of the feature extraction module to achieve preliminary guidance of the human posture and small-scale transformation of the human posture. The specific method is as follows:

[0023]

[0024] Among them, μ represents the input of the feature extraction module, F pose represents the human body posture feature map obtained in S2, F App represents the output of the input layer of the human appearance extraction network, It means that the final input dimension is 320×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0025] S3.5: Set the key K of the self-attention layer in the TransformerBlock of the feature extraction module App Sum value V App As the extracted fine-grained human appearance features, they are input into the image redrawing network based on the diffusion model.

[0026] Preferably, in S4, the reference attention mechanism is used to guide the image redrawing network based on the diffusion model, and the specific contents of effectively fusing the human appearance and background features are as follows:

[0027] S4.1: A unet based on ResnetBlock and TransformerBlock is introduced as an image redrawing network based on the diffusion model. The unet specifically includes an input layer composed of convolutional layers and a feature fusion module composed of several ResnetBlocks and TransformerBlocks, in which the self-attention layer in TransformerBlock is replaced by a reference attention layer.

[0028] S4.2: Through the VAE encoder part of the stable diffusion, the background image I Back Mapping from pixel space to low-dimensional latent space as background latent space variable ε(I Back ).

[0029] S4.3: The random noise Z t and the latent space variable ε(I Back ) and the scaled human body area mask M are used as the input of the image redrawing network after channel splicing, as follows:

[0030]

[0031] Among them, τ represents the input of the image redrawing network, Z t represents random noise, M represents the mask of the human body area, and I Back represents the source human appearance image, ε(I Back ) indicates I Back The background latent space variables after mapping to the latent space, It means that the final input dimension is 9×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0032] S4.4: The human body posture feature map F obtained in step S2 pose It is added to the output of the input layer of the image redrawing network by adding matrix elements and used as the input of the subsequent feature fusion module. It provides precise guidance for the subsequent fusion of human appearance features and background features, ensuring that the human appearance is introduced into the specified area of ​​the background image, and realizing the precise migration of human posture. The specific calculation method is as follows:

[0033]

[0034] in, represents the input of the feature extraction module, F pose represents the human body posture feature map obtained in S2, F Back represents the output of the input layer of the image redrawing network, It means that the final input dimension is 320×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0035] S4.5: With the help of the reference attention mechanism, the fine-grained human appearance features extracted by the human appearance extraction network are guided to the specified area of ​​the background image to achieve the fusion of human appearance features and background features, so as to achieve the redrawing of the background image and then generate the final human posture transfer image. The calculation method of the reference attention is as follows:

[0036]

[0037] Where Referrnce_Attn represents the calculation result of reference attention, Softmax() represents the sofmax function, Q Back , K Back Value V Back represent the query, key and value of the background feature in the image redrawing network, respectively, K App and V App represents the key and value of the human appearance features in the human appearance extraction network, [;] represents the feature concatenation operation, · represents vector multiplication, and d represents Q Back The number of channel dimensions.

[0038] S4.6: After completing the feature fusion of human appearance and background with the help of the reference attention mechanism, the image redrawing network based on the diffusion model completes the redrawing of the human image through continuous iterative denoising to generate the final virtual try-on image.

[0039] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0040] (1) The present invention proposes a human posture transfer method based on image redrawing, which can generate a human posture transfer image with high consistency by redrawing a human body image with specified posture and appearance on a background image.

[0041] (2) The present invention utilizes a human appearance extraction network to extract multi-scale human appearance features containing rich details from a source human appearance image, which prompts an image redrawing network to generate a target human appearance that is more consistent with the source human appearance, making the generated image more realistic and natural.

[0042] (3) The present invention utilizes a carefully designed reference attention mechanism to fully integrate the human appearance features extracted from the appearance extraction network and the background features in the image redrawing network. The reference attention mechanism can accurately introduce the human appearance features into specific areas of the background image under the guidance of the target posture, which enables the model to generate posture transfer images that are more in line with the target posture requirements.

[0043] (4) Benefiting from the decoupling of human body appearance and background, the present invention can replace the background area in the image while achieving human body posture migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is the overall flow chart of the present invention;

[0045] Figure 2 is a detailed network architecture diagram of the present invention;

[0046] Figure 3 This is a human body posture migration effect diagram of the present invention under the TikTok data set;

[0047] Figure 4 This is an effect diagram of the present invention realizing both human posture migration and background replacement under the TikTok dataset. DETAILED DESCRIPTION

[0048] The present invention is further illustrated below in conjunction with specific examples. The examples are implemented based on the technical solutions of the present invention. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0049] like Figure 1-2 As shown, the human body posture migration method based on image redrawing provided in this embodiment includes the following steps:

[0050] S1: Separate the background and human appearance in the source image through foreground and background separation technology;

[0051] First, the DINO (DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection) model and the SAM (Segment Anything Model) model are used to obtain the mask M of the corresponding human region from the source human image. Then, the mask M of the human region is used to separate the source human appearance and the background to obtain the source human appearance image I. App and the corresponding background image I Back .

[0052] S2: Use the posture encoder to extract the target human posture features to accurately guide the source human posture transfer;

[0053] First, the target human posture image I pose Input into a lightweight posture encoder composed of 5 layers of convolutional layers and corresponding activation functions SiLU to obtain the corresponding target human posture feature map F pose , and finally the target human posture feature map F pose They are respectively input into the human appearance extraction network and the diffusion model-based image redrawing network to accurately guide the migration of the source human posture.

[0054] S3: Obtain rich human detail features through human appearance extraction network;

[0055] S3.1: A unet based on ResnetBlock and TransformerBlock is introduced as a human appearance extraction network. The unet specifically includes an input layer composed of a convolutional layer and a feature extraction module composed of several ResnetBlocks and TransformerBlocks.

[0056] S3.2: First, the human appearance image I is transformed into App Mapping from pixel space to low-dimensional latent space as the latent space variable ε(I App ).

[0057] S3.3: The random noise Z t and the latent space variable ε(I App ) and the scaled human body area mask M are used as the input of the human appearance extraction network after channel splicing, as follows:

[0058]

[0059] Among them, γ represents the input of the human appearance extraction network, Z trepresents random noise, M represents the mask of the human body area, and I App represents the source human appearance image, ε(I app ) indicates I App The latent space variables of human appearance after mapping to the latent space, It means that the final input dimension is 9×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0060] S3.4: The human body posture feature map F obtained in step S2 pose The matrix elements are added to the output of the input layer of the human appearance extraction network and used as the input of the feature extraction module to achieve preliminary guidance of the human posture and small-scale transformation of the human posture. The specific method is as follows:

[0061]

[0062] Among them, μ represents the input of the feature extraction module, F pose represents the human body posture feature map of the human target obtained in step S2, F App represents the output of the input layer of the human appearance extraction network, It means that the final input dimension is 320×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0063] S3.5: Set the key K of the self-attention layer in the TransformerBlock of the feature extraction module App Sum value V App As the extracted fine-grained human appearance features, they are input into the image redrawing network based on the diffusion model.

[0064] S4: Use reference attention mechanism to guide diffusion model-based image repainting network to effectively integrate human appearance and background features;

[0065] S4.1: A unet based on ResnetBlock and TransformerBlock is introduced as an image redrawing network based on the diffusion model. The unet specifically includes an input layer composed of convolutional layers and a feature fusion module composed of several ResnetBlocks and TransformerBlocks, in which the self-attention layer in TransformerBlock is replaced by a reference attention layer.

[0066] S4.2: First, the background image I is transformed into Back Mapping from pixel space to low-dimensional latent space as background latent space variable ε(IBack ).

[0067] S4.3: The random noise Z t and the latent space variable ε(I Back ) and the scaled human body area mask M are used as the input of the image redrawing network after channel splicing, as follows:

[0068]

[0069] Among them, τ represents the input of the image redrawing network, Z t represents random noise, M represents the mask of the human body area, and I Back represents the source human appearance image, ε(I Back ) indicates I Back The background latent space variables after mapping to the latent space, It means that the final input dimension is 9×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0070] S4.4: The human body posture feature map F obtained in S2 pose It is added to the output of the input layer of the image redrawing network by adding matrix elements and used as the input of the subsequent feature fusion module. It provides precise guidance for the subsequent fusion of human appearance features and background features, ensuring that the human appearance is introduced into the specified area of ​​the background image, and realizing the precise migration of human posture. The specific calculation method is as follows:

[0071]

[0072] in, represents the input of the feature extraction module, F pose represents the human body posture feature map of the human target obtained in step S2, F Back represents the output of the input layer of the image redrawing network, It means that the final input dimension is 320×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

[0073] S4.5: With the help of the reference attention mechanism, the fine-grained human appearance features extracted by the human appearance extraction network are guided to the specified area of ​​the background image to achieve the fusion of human appearance features and background features, so as to achieve the redrawing of the background image and then generate the final human posture transfer image. The calculation method of the reference attention mechanism is as follows:

[0074]

[0075] Among them, Reference_Attn represents the calculation result of reference attention, Softmax() represents the sofmax function, Q Back , K Back Value V Back represent the query, key and value of the background feature in the image redrawing network, respectively, K App and V App Represents the key and value of the human appearance features in the human appearance extraction network. [;] represents the feature concatenation operation, · represents vector multiplication, and d represents Q Ba ck The number of channel dimensions.

[0076] S4.6: After completing the feature fusion of human appearance and background with the help of the reference attention mechanism, the image redrawing network based on the diffusion model completes the redrawing of the human image through continuous iterative denoising to generate the final virtual try-on image.

[0077] S5: Output the final generated image.

[0078] The effectiveness and efficiency of the method of the present invention are verified by the following experiments:

[0079] The present invention uses SSIM, LPIPS, FID and PSNR as evaluation criteria:

[0080] SSIM: An indicator used to evaluate image quality, which measures the similarity between two images by comparing the brightness, contrast, and structural similarity of the images.

[0081] LPIPS: It is used to measure the perceptual differences between images. It calculates the similarity between two images by learning the feature representation of the neural network, and more accurately captures the differences in human visual perception.

[0082] FID: It is used to evaluate the similarity between the generated image and the real image, and measures the quality of the generated image by calculating the difference in feature distribution of the intermediate layer of the neural network.

[0083] PSNR: Peak Signal-to-Noise Ratio is a measure of image quality. It is calculated by comparing the peak value of the denoised image with the root mean square of the noise. The higher the PSNR value, the better the quality of the denoised image.

[0084] The dataset used in the present invention is the TikTok dataset, which contains 350 single-person dance videos collected from the TikTok platform, most of which contain faces and the corresponding upper half of faces. By extracting frames from the video, 94,908 pairs of human images can be obtained, each of which consists of two human images with the same background and human appearance but different postures. The training set includes 90,785 human image pairs, and the test set includes 4,123 human image pairs, with a resolution of 512×512 pixels for each image.

[0085] The present invention uses the TikTok dataset, and compares the results of the method of the present invention with FOMM, TPS, Disco, MagicAnimate and other methods under the same resolution condition (512×512 pixels). Among these methods, FOMM and TPS are human posture transfer methods based on generative adversarial networks (GAN). Disco and MagicAnimate are the latest human posture transfer methods based on diffusion models. Among them, Disco also replaces the background of the image while migrating the human posture by decoupling the human appearance and the background. With the help of the image decoupling operation, the Disco method can also avoid the mutual interference of different transformations of the background and human appearance areas to a certain extent. MagicAnimate uses a carefully designed content control mechanism to ensure the consistency of image content before and after posture transfer.

[0086] The quantitative experimental results of the model comparison under the TikTok dataset are as follows:

[0087] Table 1 Quantitative experimental results of the present invention on the TikTok dataset

[0088] method FID↓ SSIM↑ PSNR↑ LPIPS↓ FOMM 85.03 0.648 29.01 0.335 TPS 53.78 0.673 29.18 0.299 Disco 30.75 0.668 29.03 0.292 MagicAnimate 32.09 0.714 29.16 0.239 Ours (This invention) 30.56 0.734 29.38 0.298

[0089] According to the results in Table 1, the method of the present invention significantly outperforms other comparison methods in multiple evaluation indicators such as FID, SSIM and PSNR, and also shows competitive performance in the LPIPS indicator. Compared with FOMM and TPS based on adversarial generative networks, Disco, MagicAnimate and the method of the present invention, which use diffusion models as the underlying architecture, show better results.

[0090] Although both MagicAnimate and the present invention use the diffusion model as the core architecture and introduce a carefully designed appearance control mechanism, their performance on multiple key indicators is not as good as the present invention because they ignore the different transformations experienced by the background and human appearance areas when processing human posture migration. In particular, the method of the present invention shows its obvious advantages in the FID and SSIM indicators that reflect the realism of the image and the image similarity. It is worth mentioning that although the Disco method also adopts a decoupling strategy to reduce the mutual interference between the transformation of the background and the human appearance area, the present invention has achieved significantly leading results in indicators such as FID, SSIM and PSNR through a more powerful human appearance extraction network and a carefully designed reference attention mechanism. This achievement not only proves the effectiveness of the method of the present invention, but also demonstrates its superior ability in handling complex posture migration tasks.

[0091] Figure 3 The visualization results of the method of the present invention and other comparative methods on the TikTok dataset are shown. In the TikTok dataset, the results of the present invention are compared with the TPS, Disco and MagicAnimate methods. Obviously, compared with other methods, the method of the present invention performs better in image consistency and precise control of posture. When performing complex posture migration, TPS, Disco and MagicAnimate have difficulty in generating reasonable facial and hand postures, and it is difficult to retain the appearance details of the source character, and often produce unrealistic human posture migration images.

[0092] Figure 4 The effect diagram of Disco and the method of the present invention realizing the simultaneous human posture migration and background replacement under the TikTok dataset is shown. From the figure, it can be seen that although Disco and the method of the present invention also adopt the method of separating the image background and the human appearance to avoid the mutual interference between the different transformations of the background and the human appearance areas, thanks to the image redrawing technology and the carefully designed reference attention mechanism, the method of the present invention can complete the accurate migration of the human posture while completing the effective replacement of the background image, thereby generating a higher quality human image.

[0093] The present invention optimizes the naturalness and detail performance of human posture migration by independently processing the human body and background parts of the image and utilizing the powerful image generation capability of the diffusion model. The method can maintain the consistency of the background in human posture migration, while generating high-quality images with real details, improving the authenticity and coherence of the visual effect. In addition, the method can also replace the background area in the image while achieving human posture migration.

[0094] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A human body posture transfer method based on image redrawing, characterized in that: The human body and background parts of the image are processed independently, and the image generation capability of the diffusion model is used to redraw the human body image with specified posture and appearance on the background image to generate a human body posture migration image with high consistency, thereby optimizing the naturalness and detail expression of human body posture migration.

2. The human body posture transfer method based on image redrawing according to claim 1, characterized in that: The specific steps include: S1: Separate the background and human appearance in the source image through foreground and background separation technology; S2: Use the posture encoder to extract the target human posture features to guide the source human posture transfer; S3: Obtaining detailed features of the human body through the human appearance extraction network; S4: Using reference attention mechanism to guide diffusion model-based image repainting network to integrate human appearance and background features; S5: Output the final generated image.

3. The human body posture transfer method based on image redrawing according to claim 2, characterized in that: In S1, the background and human appearance in the source image are separated using the foreground and background separation technology. The specific method is as follows: First, the DINO model and the SAM model are used to obtain the mask M of the corresponding human body area from the source human body image. Then, the mask M of the human body area is used to separate the source human body appearance and the background to obtain the source human body appearance image I. App and the corresponding background image I Back .

4. The method for human body posture transfer based on image redrawing according to claim 2, characterized in that: In step S2, the posture encoder is used to extract the posture features of the target human body to guide the source human body to perform posture migration. The specific process is: First, the target human posture image I pose Input into the lightweight posture encoder to obtain the corresponding target human posture feature map F pose , and then the target human posture feature map F pose They are respectively input into the human appearance extraction network and the diffusion model-based image redrawing network to guide the migration of the source human posture.

5. The method for human body posture transfer based on image redrawing according to claim 2, characterized in that: In step S3, the specific method of obtaining rich human detail features through the human appearance extraction network is: S3.1: Use unet based on ResnetBlock and TransformerBlock as the human appearance extraction network; S3.2: Through the VAE encoder part of Stablediffusion, the human appearance image I App Mapping from pixel space to low-dimensional latent space as the latent space variable ε(I App ); S3.3: Concatenate the random noise, the human appearance latent space variables and the scaled human region mask channel and input them into the human appearance extraction network; S3.4: The human body posture feature map F obtained in step S2 pose By adding matrix elements, it is added to the output of the input layer of the human appearance extraction network and used as the input of the feature extraction module to achieve preliminary guidance of human posture and small-scale transformation of human posture; S3.5: Set the key K of the self-attention layer in the TransformerBlock of the feature extraction module ApP Sum value V APP As the extracted fine-grained human appearance features, they are input into the image redrawing network based on the diffusion model.

6. The method for human body posture transfer based on image redrawing according to claim 5, characterized in that: The specific implementation of step S3.3 is as follows: Among them, γ represents the input of the human appearance extraction network, Z t represents random noise, M represents the mask of the human body area, and I App represents the source human appearance image, ε(I app ) indicates I App The human appearance latent space variables after being mapped to the human appearance latent space, It means the final input dimension is 9×h×w.

7. The method for human body posture transfer based on image redrawing according to claim 5, characterized in that: The specific implementation of step S3.4 is as follows: Among them, μ represents the input of the feature extraction module, F pose represents the human body posture feature map F obtained in step S2 pose , F App represents the output of the input layer of the human appearance extraction network, It means that the final input dimension is 320×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

8. The method for human body posture transfer based on image redrawing according to claim 2, characterized in that: In step S4, the reference attention mechanism is used to guide the diffusion model-based image redrawing network to effectively fuse the human appearance and background features. The specific content is: S4.1: Use unet based on ResnetBlock and TransformerBlock as the image redrawing network based on the diffusion model; S4.2: Through the VAE encoder part of Stablediffusion, the background image I Back Mapping from pixel space to low-dimensional latent space as background latent space variable ε(I Back ); S4.3: Concatenate the random noise, the human appearance latent space variables and the scaled human region mask channel as the input of the image redrawing network; S4.4: The human body posture feature map F obtained in step S2 pose It is added to the output of the image redrawing network input layer by adding matrix elements and used as the input of the subsequent feature fusion module; S4.5: Using the reference attention mechanism, the fine-grained human appearance features extracted by the human appearance extraction network are guided into the specified area of ​​the background image to achieve the fusion of human appearance features and background features, so as to achieve the redrawing of the background image and generate the final human posture transfer image; S4.6: After completing the feature fusion of human appearance and background with the help of the reference attention mechanism, the image redrawing network based on the diffusion model completes the redrawing of the human image through continuous iterative denoising to generate the final virtual try-on image.

9. The method for human body posture transfer based on image redrawing according to claim 8, characterized in that: The specific implementation of step S4.3 is as follows: Among them, τ represents the input of the image redrawing network, Z t represents random noise, M represents the mask of the human body area, and I Back represents the corresponding background image, ε(I Back ) indicates I Back The background latent space after mapping to the latent space, variable It means that the final input dimension is 9×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

10. The method for human body posture transfer based on image redrawing according to claim 8, characterized in that: The specific implementation of step S4.4 is as follows: in, represents the input of the feature extraction module, F pose represents the human body posture feature map of the human target obtained in step S2, F Back represents the output of the input layer of the image redrawing network, It means that the final input dimension is 320×h×w, where h=H / 8, w=W / 8, and H and W are the height and width of the original input image respectively.

11. The method for human body posture transfer based on image redrawing according to claim 2, characterized in that: In step S4.5, the reference attention mechanism is calculated as follows: Among them, Reference_Attn represents the calculation result of reference attention, Softmax() represents the sofmax function, Q Back , K Back Value V Back represent the query, key and value of the background feature in the image redrawing network, respectively, K App and V App represents the key and value of the human appearance features in the human appearance extraction network, [;] represents the feature concatenation operation, · represents vector multiplication, and d represents Q Back The number of channel dimensions.

Images