Automated Costume Augmentation via 3D Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for augmenting a human body with virtual costumes in augmented reality face challenges due to ambiguities in depth projection and variations in body shapes and poses, leading to insufficient 3D shape estimation and poor fitting of digital costumes.
Innovation Solution
An automated system using an artificial neural network (ANN) for 2D skeleton estimation, followed by 3D pose determination and shape estimation, parameterizes a virtual costume to fit realistically onto a posed figure, incorporating bone directions for improved fit and using masking and inpainting to enhance the image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D shape estimation is performed from RGB image using conventional methods, then the process is simple, but the estimation accuracy is poor due to depth projection ambiguities and body shape variations
Solution Approach 1:
The patent introduces an artificial neural network as an intermediary component between the RGB image input and 3D shape estimation output. The ANN processes the image data through multiple layers to extract meaningful features and generate accurate 3D shape estimates, resolving the depth projection ambiguities that conventional direct methods cannot handle.
Solution Approach 2:
The patent transforms the 2D image data into 3D shape information by using the ANN to infer depth and spatial relationships. This dimensional transformation allows the system to overcome the inherent limitations of 2D-to-3D reconstruction while maintaining automated processing.
2Manufacturing precision
If digital costume is overlaid onto human figure image based on 3D pose matching, then the process is simple and fast, but the fitting quality is poor with visible clothing or skin remaining
Solution Approach 1:
The patent performs preliminary 3D shape estimation and pose determination before applying the digital costume. By pre-processing the image to extract accurate 3D body shape information and bone directions, the system ensures that the costume can be fitted precisely to the body contours, eliminating visible gaps between clothing and skin.
Solution Approach 2:
The patent uses parameterized 3D models for both body shape and digital costumes. By adjusting the parameters of these models based on the estimated body shape and pose, the system dynamically adapts the costume to fit each individual's unique body geometry, achieving high fitting quality automatically.
Data Source
AI summary
An automated costume augmentation system includes a computing platform having a hardware processor and a system memory storing a software code. The hardware processor executes the software code to provide an image including a posed figure to an artificial neural network (ANN), receive from the ANN a 2D skeleton data including joint positions corresponding to the posed figure, and determine a 3D pose corresponding to the posed figure using an optimization algorithm applied to the skeleton data. The software code further identifies one or more proportion(s) of the posed figure based on the skeleton data, determines bone directions corresponding to the posed figure using another optimization algorithm applied to the 3D pose, parameterizes a costume for the posed figure based on the 3D pose, the proportion(s), and the bone directions, and outputs an enhanced image including the posed figure augmented with the fitted costume for rendering on a display.