3D Model Mesh Deformation for Virtual Try-On
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating 3D CG models for virtual try-on applications is costly and time-consuming, as it requires creating multiple models to account for various user states, such as direction, posture, and body type.
Innovation Solution
An image processing device acquires a 3D model of a subject, sets control points, extracts mesh data from clothing images, and modifies the mesh data based on control point movements to generate images of clothing in different states, allowing for virtual try-on without the need for extensive model creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple 3D CG models are prepared to account for various user states (direction, posture, body type), then the accuracy and adaptability of virtual try-on is improved, but the cost and time required for model generation increases
Solution Approach 1:
The patent segments the 3D model into a base model and deformable mesh data. Instead of creating multiple complete 3D models for different user states, the system extracts mesh data representing clothing deformation from 2D images and applies it to the base model. This segmentation allows one base model to serve multiple purposes by combining it with different mesh deformations, thereby reducing the time and cost of model generation while maintaining adaptability to various user states.
Solution Approach 2:
The patent uses 2D image data as a copy or representation of the 3D clothing deformation. Instead of directly creating multiple 3D models, the system captures 2D images of clothing on models with various postures and extracts mesh data from these 2D representations. This copying approach allows the system to obtain deformation information from multiple sources without creating corresponding 3D models for each, thus reducing generation time and cost while maintaining the ability to represent various user states.
2Adaptability or versatility
If multiple 3D CG models are prepared to account for various user states (direction, posture, body type), then the accuracy and adaptability of virtual try-on is improved, but the cost of generating 3D CG models increases
Solution Approach 1:
The patent segments the 3D model into a base model and deformable mesh data. Instead of creating multiple complete 3D models for different user states, the system extracts mesh data representing clothing deformation from 2D images and applies it to the base model. This segmentation allows one base model to serve multiple purposes by combining it with different mesh deformations, thereby reducing the time and cost of model generation while maintaining adaptability to various user states.
Solution Approach 2:
The patent uses 2D image data as a copy or representation of the 3D clothing deformation. Instead of directly creating multiple 3D models, the system captures 2D images of clothing on models with various postures and extracts mesh data from these 2D representations. This copying approach allows the system to obtain deformation information from multiple sources without creating corresponding 3D models for each, thus reducing generation time and cost while maintaining the ability to represent various user states.
3Productivity
If a single 3D CG model is used for virtual try-on, then the cost and time for model generation is reduced, but the ability to represent different user states (direction, posture, body type) deteriorates
Solution Approach 1:
The patent introduces dynamic mesh data that can be deformed and adjusted to represent different user states. Instead of using a static single 3D model, the system combines a base model with dynamic mesh deformations extracted from 2D images. This dynamic approach allows the same base model to adapt to various user states (different directions, postures, and body types) by applying appropriate mesh deformations, thereby maintaining productivity while improving adaptability.
Solution Approach 2:
The patent changes the parameters of the mesh data to represent different user states. By extracting mesh data from 2D images with varying parameters (posture, direction, body type) and applying these as deformations to the base model, the system enables a single model to represent multiple states. This parameter change approach allows efficient model generation while maintaining the ability to accurately represent different user states through modular deformation application.
Data Source
AI summary
According to one embodiment, an image processing device includes at least one processor. The at least one processor is configured to acquire a first three-dimensional model regarding a subject, set a plurality of first control points on the first three-dimensional model, acquire mesh data of a meshed image of a region of clothing extracted from a captured image, acquire a second three-dimensional model, modify the mesh data based on an amount of movement from each of the plurality of first control points, to each respective one of a plurality of second control points, and generate an image of the clothing using the captured image and the modified mesh data.