Digital Image Reposing Using Multi-View Selection Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional human reposing techniques using a single source image often generate output images with visual artifacts and occlusions when the target pose significantly differs from the source pose, as they infer invisible portions based on unavailable information.
Innovation Solution
A system utilizing multiple input views and machine learning models, including convolutional neural networks, transformers, and feature pyramid networks, to generate selection masks and composite flows, effectively combining visible and invisible portions from multiple images to create a realistic output image in the target pose.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional single-source image reposing techniques are used, then the process is simple and fast, but visual artifacts and occlusions occur when target pose significantly differs from source pose
Solution Approach 1:
The patent segments the reposing process into multiple independent modules: visibility module (generating visibility masks), selection module (generating selection masks), and generator module (synthesizing output image). Each module processes specific information from multiple input images independently, then combines results to produce the final output. This segmentation allows the system to handle complex pose transformations without generating artifacts, as each module can optimize for its specific function.
Solution Approach 2:
The patent merges information from multiple input images (showing person in different poses) by combining visibility masks, selection masks, and image data through the generator module. This merging process allows the system to reconstruct the target pose accurately by integrating information from multiple views, eliminating the artifacts that occur in single-image methods while maintaining computational efficiency.
2Measurement precision
If multiple input views are used to improve accuracy, then realism and accuracy of output image improve, but computational resources and processing time increase
Solution Approach 1:
The patent performs preliminary actions by pre-processing multiple input images to generate visibility masks and selection masks before the final synthesis step. The visibility masks identify which regions are visible in each input image, and selection masks determine which regions should be prioritized for the target pose. These pre-computed masks enable the generator module to efficiently synthesize the output image without re-processing all input data, significantly reducing computational resources while maintaining high pose accuracy.
3Productivity
If conventional methods infer invisible portions from single image, then processing is fast, but output contains occlusions and artifacts
Solution Approach 1:
The patent introduces intermediary elements (visibility masks and selection masks) that mediate between the multiple input images and the final output image. These masks act as intermediaries that preserve essential information about visible and invisible portions, allowing the generator module to accurately reconstruct the target pose without losing information. The visibility masks specifically serve as intermediaries that track which regions should be visible in the output, preventing occlusions and artifacts while maintaining processing efficiency.
Data Source
AI summary
In implementations of systems for human reposing based on multiple input views, a computing device implements a reposing system to receive input data describing: input digital images; pluralities of keypoints corresponding to the input digital images, the pluralities of keypoints representing poses of a person depicted in the input digital images; and a plurality of keypoints representing a target pose. The reposing system generates selection masks corresponding to the input digital images by processing the input data using a machine learning model. The selection masks represent likelihoods of spatial correspondence between pixels of an output digital image and portions of the input digital images. The reposing system generates the output digital image depicting the person in the target pose for display in a user interface based on the selection masks and the input data.