3D Mesh Generation from User Images Using Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for obtaining accurate virtual representations of products, especially wearable items, require bulky or expensive hardware like 3D scanners, making it difficult to provide seamless and instantaneous user experiences, particularly for personalized accessories where in-person measurements are cumbersome and impractical.
Innovation Solution
A system and method using advanced computer vision and machine learning techniques to generate accurate 3D mesh models and dimensions of products from images taken with ordinary electronic devices, such as smartphones, leveraging augmented reality SDKs to extract body part meshes and generate product meshes that fit anatomical features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D scanners or specialized sensors are used to obtain accurate body part measurements, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces mechanical 3D scanning systems with a computational approach using 2D images captured by standard camera sensors. Deep learning models process these 2D images to generate 3D body part meshes and measurements, substituting complex mechanical hardware with software-based solutions that run on mobile devices.
Solution Approach 2:
The system creates a virtual 3D copy of the body part from 2D image data. Instead of directly measuring the physical body part with specialized sensors, the deep learning model generates a digital mesh representation that replicates the geometric features, allowing measurements to be extracted from the virtual model.
2Manufacturing precision
If in-person measurements are conducted to ensure accurate product fit, then manufacturing precision is improved, but loss of time increases
Solution Approach 1:
The system performs measurements remotely before the manufacturing or customization process. Users capture images at their convenience, and the deep learning model immediately processes these images to generate accurate body part measurements, eliminating the need for scheduled in-person measurement sessions and enabling just-in-time product customization.
3Measurement precision
If specialized scanning hardware is required for product visualization, then measurement precision is improved, but ease of operation deteriorates
Solution Approach 1:
The system uses universal mobile device cameras that most users already possess, rather than requiring specialized scanning hardware. The deep learning model is trained to extract accurate measurements from standard 2D images, making the measurement capability accessible to anyone with a smartphone, thus achieving both precision and universality.
Data Source
AI summary
The present invention provides systems and methods for generating a 3D product mesh model and product dimensions from user images. The system is configured to receive one or more images of a user's body part, extract a body part mesh having a plurality of body part key points, generate a product mesh from an identified subset of the body part mesh, and generate one or more product dimensions in response to the selection of one or more key points from the product mesh. The system may output the product mesh, the product dimensions, or a manufacturing template of the product. In some embodiments, the system uses one or more machine learning modules to generate the body part mesh, identify the subset of the body part mesh, generate the product mesh, select the one or more key points, and/or generate the one or more product dimensions.