Wearable 3D Body Modeling Using Stereo Vision and Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for 3D modeling of a human body, especially when wearing a wearable device, face challenges in capturing detailed movements and shapes accurately, often requiring additional equipment and degrading user convenience due to the weight and size of imaging devices.
Innovation Solution
A wearable device equipped with multiple imaging devices, such as stereo vision cameras with fisheye lenses, generates feature maps to obtain joint keypoint information and initial shape coefficients, determining target 3D joint angles and shape coefficients using convolutional neural networks, and reconstructs a 3D mesh of the wearer without the need for external equipment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional 3D modeling methods are used, then modeling accuracy can be achieved, but additional imaging equipment is required which increases device complexity and user inconvenience
Solution Approach 1:
The patent merges the imaging function and 3D modeling function into a single wearable device. The wearable device includes an imaging device that captures images and a processor that performs 3D modeling based on these images, eliminating the need for separate external imaging equipment and achieving integrated functionality.
Solution Approach 2:
The wearable device is designed to perform multiple functions: it serves as both a wearable device (e.g., smartwatch, AR glasses) and a 3D modeling system. The imaging device and processor within the wearable device enable it to capture images and generate 3D models, making the device universal for both wearability and 3D modeling applications.
2Measurement precision
If imaging devices are added to capture detailed body movements, then 3D modeling accuracy improves, but the weight and size of the wearable device increase
Solution Approach 1:
The patent uses a single imaging device that captures multiple images from different angles rather than requiring multiple imaging devices. By processing these multiple images through the processor, the system achieves accurate 3D modeling without adding the weight and size of multiple separate imaging devices.
Solution Approach 2:
Instead of using multiple physical imaging devices to capture different perspectives, the system uses a single imaging device to capture multiple images and creates virtual copies of these images through image processing. The processor generates 3D models by analyzing these image copies, achieving the same result without the physical burden of multiple devices.
3Loss of information
If multiple imaging devices are used to capture body movements, then the number of observed joints increases, but the device becomes more complex and less convenient
Solution Approach 1:
The system dynamically processes multiple images captured by a single imaging device to track joint positions over time. The processor analyzes the sequence of images to determine the positions of multiple joints, achieving complete joint observation through dynamic image processing rather than static multiple device capture.
Solution Approach 2:
The processor acts as an intermediary that bridges the single imaging device and the 3D modeling output. It processes the images from the single imaging device to extract joint position information and generates accurate 3D models, effectively mediating between limited hardware and comprehensive modeling requirements.
Data Source
AI summary
A method with three-dimensional (3D) modeling of a wearer of a wearable device includes generating a feature map for each of a plurality of images of the wearer obtained from a plurality of imaging devices provided in the wearable device, obtaining joint keypoint information corresponding to joint positions of the wearer and initial shape coefficient information associated with a shape of the wearer based on the feature map for each of the images, determining a target 3D joint angle for 3D modeling of the wearer based on the joint keypoint information and the initial shape coefficient information, determining target shape coefficient information for 3D modeling of the wearer based on the joint keypoint information and the initial shape coefficient information, and obtaining a 3D mesh of the wearer based on the target 3D joint angle and the target shape coefficient information.