Depth Map Fusion Between Cameras for Missing Pixel Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera systems for robotic automation in environments like warehouses and manufacturing facilities face challenges in generating accurate and complete depth maps due to noise susceptibility and resolution differences between various cameras, leading to incomplete or inaccurate depth information.
Innovation Solution
A method and system that utilizes a communication interface to combine depth maps from different types of cameras, such as structured light and time-of-flight cameras, to update and enhance depth information by assigning depth values to empty pixels in one depth map based on corresponding values from another, thereby creating a more comprehensive and accurate depth map.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single type of depth-sensing camera is used, then the system complexity is reduced, but the measurement precision and reliability of depth maps deteriorate due to noise susceptibility and resolution limitations
Solution Approach 1:
The patent combines depth maps from multiple types of depth-sensing cameras (e.g., structured light camera and time-of-flight camera) to create a fused depth map. Each camera type has different strengths and weaknesses, and by merging their outputs, the system achieves higher measurement precision and reliability while compensating for individual camera limitations and noise susceptibility
2Measurement precision
If multiple types of depth-sensing cameras are used, then the measurement precision and completeness of depth maps improve, but the device complexity increases
Solution Approach 1:
The patent introduces a depth map fusion module as an intermediary that processes and integrates depth maps from multiple camera types. This mediator aligns, registers, and fuses the depth information, managing the complexity of multiple cameras while delivering improved measurement precision and complete depth coverage
3Loss of information
If depth maps from different camera types are combined, then the completeness of depth information improves by filling empty pixels, but the processing complexity increases
Solution Approach 1:
The patent applies local quality enhancement by identifying empty pixels or low-confidence regions in individual depth maps and selectively filling them using data from other camera types. This targeted approach improves depth information completeness where needed without unnecessarily processing all pixels, thereby managing processing complexity efficiently
Data Source
AI summary
A method and system for processing camera images is presented. The system receives a first depth map generated based on information sensed by a first type of depth-sensing camera, and receives a second depth map generated based on information sensed by a second type of depth-sensing camera. The first depth map includes a first set of pixels that indicate a first set of respective depth values. The second depth map includes a second set of pixels that indicate a second set of respective depth values. The system identifies a third set of pixels of the first depth map that correspond to the second set of pixels of the second depth map, identifies one or more empty pixels from the third set of pixels, and updates the first depth map by assigning to each empty pixel a respective depth value based on the second depth map.