Dual GAN Pose Transfer for Spatial Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human pose transfer technologies using generative adversarial networks (GANs) face issues with spatial misalignment and lack of high-frequency details, especially when the pose change is significant and the background is complex.
Innovation Solution
A method involving two image generation models to align poses and improve image details, where a first model generates a target parsed image by transforming the pose, and a second model adjusts the combined images to produce a target image with spatial alignment and clear details.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single GAN network is used for pose transfer, then the process is simple, but spatial misalignment occurs when pose changes greatly
Solution Approach 1:
The patent divides the pose transfer process into two separate GAN networks: a first GAN network for generating a target parsed image with transformed pose, and a second GAN network for generating the final target image. This segmentation allows each network to specialize in specific tasks, with the first network handling pose transformation and the second network handling final image synthesis, thereby improving spatial alignment precision while managing complexity through modular design.
2Device complexity
If a single GAN network is used for pose transfer, then the model is simple, but image details lack high frequency information
Solution Approach 1:
The patent segments the image generation process into two stages with two separate GAN networks. The first network focuses on pose transformation and generates a target parsed image, while the second network focuses on generating the final high-quality image with preserved details. This division allows the second network to specialize in preserving high-frequency details while the first network handles pose transformation, resolving the contradiction between model simplicity and detail precision.
3Adaptability or versatility
If pose transfer is performed with large pose changes, then pose transformation capability is improved, but spatial misalignment appears
Solution Approach 1:
The patent introduces a target parsed image as an intermediary between the original image and the final target image. The first GAN network transforms the pose and generates this intermediate representation, which then serves as input for the second GAN network. This intermediary structure enables large pose transformations while maintaining spatial alignment, as the intermediate parsed image captures the transformed pose geometry that the second network can then accurately render.
Data Source
AI summary
The present disclosure provides a target image generation method. The method includes obtaining a first parsed image and a first pose image based on an original image, the first parsed image being an image labeled with parts of an object in the original image, the first pose image representing a pose of the object in the original image; inputting the first parsed image, the first pose image, and a second pose image representing a target pose into a first image generation model, and determining, a first transformation parameter and adjusting the first parsed image based on the first transformation parameter to obtain a target parsed image, a pose of the object in the target parsed image being the target pose; and inputting a first combined image and a second combined image into a second image generation model, and adjusting the first combined image to obtain a target image.