3D Imaging of Transparent Objects Using Depth Map Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing three-dimensional imaging methods struggle with imaging transparent objects due to issues such as high surface texture requirements, poor resolution in close-range scenes, and inaccurate depth measurement on transparent surfaces.
Innovation Solution
A method involving image processing to segment and predict transparent objects, perform cutting processes on depth maps, and apply global optimization techniques to enhance depth accuracy, using a neural network model for segmentation and normal vector prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If structured light projection three-dimensional imaging is used, then the feature matching effect is improved, but the projected light is reflected on transparent objects making three-dimensional imaging impossible
Solution Approach 1:
The patent introduces an intermediary processing step between light projection and depth measurement. A neural network model processes the color image to predict transparent object regions, boundaries, and normal vectors, which then guide the depth completion process. This intermediary intelligence layer mediates between the projected light and the transparent objects, allowing the system to handle transparent objects that would otherwise reflect light and prevent imaging.
Solution Approach 2:
The patent replaces the traditional optical-mechanical depth measurement approach with a computational approach. Instead of relying solely on physical light projection and detection, the system uses neural network-based image processing and depth completion algorithms to infer depth information for transparent objects, substituting mechanical/optical measurement with computational intelligence.
2Measurement precision
If passive binocular three-dimensional imaging is used, then three-dimensional information is calculated through triangulation, but high surface texture features are required which cannot be met by transparent objects
Solution Approach 1:
The patent enables the system to serve itself by using the color image information already captured to generate depth information for transparent objects. The neural network model extracts features from the color image and uses them to predict transparent object characteristics, allowing the system to create its own depth data without requiring external texture features that transparent objects cannot provide.
Solution Approach 2:
The patent changes the parameters used for depth calculation by transitioning from texture-based feature matching to normal vector-based depth completion. Instead of relying on surface texture parameters, the system uses predicted normal vectors and boundary information to infer depth, fundamentally changing the parameter set required for three-dimensional imaging of transparent objects.
3Length of stationary object
If time-of-flight principle-based three-dimensional imaging is used, then long-range scenes are imaged, but resolution and accuracy are poor in close-range scenes
Solution Approach 1:
The patent applies local quality enhancement by using a neural network model to process and enhance depth information specifically in regions where transparent objects are detected. The system performs localized depth completion on transparent object regions using predicted normal vectors and boundaries, rather than applying a uniform approach across the entire scene, thereby improving local resolution and accuracy where needed.
Data Source
AI summary
A three-dimensional imaging method and apparatus includes: acquiring a target color image and original depth map corresponding to a target shooting scene, where the target shooting scene contains at least one transparent object; inputting the target color image into a preset image processing model and, according to an output of the preset image processing model, obtaining a transparent object segmentation result, a boundary prediction result and a normal vector prediction result; based on the transparent object segmentation result, performing cutting processing on the original depth map to obtain a first depth map without transparent object depth information; based on the first depth map, the transparent object segmentation result, the boundary prediction result and the normal vector prediction result, performing depth map global optimization to determine an optimized second depth map; and, based on the second depth map, determining a target three-dimensional image corresponding to the target shooting scene.