Color-Guided Depth Map Completion for Low-Power LiDAR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Electronic devices, particularly those with limited power consumption, often provide depth images with insufficient accuracy due to inherent limitations in LiDAR systems and software complementation methods, resulting in unsatisfactory depth value precision.
Innovation Solution
A method involving a depth model comprising an encoder, downsampler, and decoder is used to enhance depth image accuracy by leveraging a color image to re-determine depth values for target areas within a depth image, utilizing a machine learning algorithm to train the model with marked depth image samples and expected output data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR system is used to generate depth images, then depth information can be obtained, but the accuracy of depth values is insufficient due to power consumption restrictions
Solution Approach 1:
The patent introduces a depth completion model as an intermediary between the low-accuracy LiDAR depth image and the final high-accuracy depth map. This model takes both the color image and initial depth image as inputs, processes them through encoder-decoder networks with skip connections, and outputs enhanced depth values that maintain detail while reducing noise, thereby achieving high accuracy without increasing power consumption of the LiDAR system
Solution Approach 2:
The patent combines multiple data sources (color image data and depth image data) into a composite processing framework. The depth completion model integrates features from both modalities through concatenation operations and multi-scale feature fusion, creating a composite representation that leverages the strengths of both input types to produce superior depth accuracy
2Measurement precision
If software is used to complement depth images, then depth accuracy can be improved, but the accuracy is still less satisfying
Solution Approach 1:
The patent replaces traditional mechanical/optical depth measurement systems with an intelligent software-based depth completion model. This neural network-based approach substitutes physical measurement limitations with computational intelligence, using learned patterns from training data to infer accurate depth values where direct measurement is insufficient, achieving higher accuracy without additional hardware complexity
3Measurement precision
If depth model with encoder and decoder is used, then depth image accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent segments the depth completion task into distinct functional modules: encoder networks for feature extraction, skip connections for multi-scale feature preservation, and decoder networks for depth map reconstruction. This segmentation allows each component to specialize in specific processing functions, improving overall accuracy while maintaining manageable complexity through modular design
Solution Approach 2:
The patent operates in multiple dimensional spaces simultaneously - processing features at different spatial scales through pyramid structures, combining color and depth modalities in joint feature spaces, and transforming data across different representations (raw pixels to feature maps to depth values). This multi-dimensional processing enhances accuracy by capturing relationships that single-dimensional approaches would miss
Data Source
AI summary
A method includes obtaining a color image and a first depth image; determining a target area of the first depth image, the target area of the first depth image having a first set of pixels with a first set of depth values; based on comparing an accuracy of the first set of depth values with a preset accuracy threshold: determining a target area of the color image that corresponds to the target area of the first depth image; determining, based on the target area of the color image, a second set of pixels with a second set of depth values for the target area of the first depth image; generating a second depth image that replaces the first set of pixels of the first depth image with the second set of pixels with the second set of depth values; outputting the second depth image as a three dimensional image.