Integrating Depth and Visible Light TSDF for 3D Shape Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current three-dimensional shape reconstruction techniques using depth cameras struggle with materials that absorb infrared light, such as black hair, resulting in inaccurate depth measurements and incomplete three-dimensional shape estimation.
Innovation Solution
An image processing apparatus that integrates depth maps from depth cameras with visible light images from visible light cameras to generate an integrated TSDF space, enabling accurate three-dimensional shape analysis by combining depth information from both sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth camera is used for three-dimensional shape reconstruction, then depth information can be acquired, but accurate depth measurement becomes difficult for materials that absorb infrared light
Solution Approach 1:
The patent combines depth maps from depth cameras with visible light images from visible light cameras to create an integrated TSDF space. This merging of multiple data sources allows the system to compensate for depth measurement failures on infrared-absorbing materials by using visible light image information, thereby resolving the contradiction between depth measurement accuracy and depth acquisition reliability.
Solution Approach 2:
The patent introduces visible light images as an intermediary data source to bridge the gap where depth camera measurements fail. The visible light images serve as a mediator that provides complementary information for regions where infrared-based depth measurement is unreliable, enabling complete three-dimensional shape reconstruction.
2Device complexity
If only depth map is used for three-dimensional shape reconfiguration, then processing is simplified, but correct three-dimensional shape cannot be estimated for regions where depth cannot be accurately acquired
Solution Approach 1:
The patent merges depth map-based TSDF space with visible light image-based TSDF space into an integrated TSDF space. This combination maintains processing efficiency while significantly improving three-dimensional shape accuracy, particularly for regions where depth measurement fails, by incorporating visible light image information without substantially increasing processing complexity.
3Ease of operation
If depth camera uses infrared light for active sensing, then depth measurement can be performed, but highly accurate depth measurement becomes difficult for materials with weak reflection for infrared light
Solution Approach 1:
The patent changes the operational parameters by introducing visible light imaging as a complementary measurement modality. When infrared-based depth measurement parameters fail for certain materials, the system switches to or combines with visible light image parameters, thereby maintaining depth measurement capability while improving accuracy across diverse material types.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for highly accurate three-dimensional shape reconstruction even in regions where depth cameras struggle, such as black hair, by utilizing visible light images to complement depth map data, resulting in a more complete and accurate three-dimensional model.
Implementation Method 1
many depth cameras that are currently available at relatively low cost are cameras using a method called active sensing using infrared light (IR light)
Implementation Method 2
generates an integrated TSDF space by integration processing on the depth map based TSDF space and the visible light image based TSDF space
Data Source
AI summary
A three-dimensional shape of a subject is analyzed by inputting captured images of a depth camera and a visible light camera. There is provided an image processing unit configured to input captured images of the depth camera and the visible light camera, to analyze a three-dimensional shape of the subject. The image processing unit generates a depth map based TSDF space (TSDF Volume) by using a depth map acquired from a captured image of the depth camera, and generates a visible light image based TSDF space by using a captured image of the visible light camera. Moreover, an integrated TSDF space is generated by integration processing on the depth map based TSDF space and the visible light image based TSDF space, and three-dimensional shape analysis processing on the subject is executed using the integrated TSDF space.