Depth Image Upsampling via Color Intermediary Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth image acquisition technologies face challenges in achieving high resolution due to limitations in chip area and power consumption, resulting in low-resolution depth images that fail to meet user requirements for surface texture details and accuracy in applications like face recognition.
Innovation Solution
A method that maps a depth image of a first resolution into a color image of a second resolution, establishing an objective function to minimize errors between the processed image and the depth image, and solving this function to obtain a higher resolution depth image, incorporating neighborhood smoothing and non-local average regularization items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a depth acquisition component is used to obtain depth image, then depth information can be acquired, but the resolution is low due to chip area and power consumption limitations
Solution Approach 1:
The patent uses a color image as an intermediary to transfer high-resolution information from the color camera to the depth image. By mapping color image pixels to depth image pixels and using the color image's high resolution to guide the depth image reconstruction, the system achieves high-resolution depth imaging without requiring a large-depth sensor chip.
Solution Approach 2:
The patent changes the resolution parameter of the depth image by establishing an objective function that maps pixels from a high-resolution color image to a low-resolution depth image, then solves this function to generate a high-resolution depth image. This transforms the depth image resolution from being limited by hardware chip area to being determined by software processing.
2Measurement precision
If a depth acquisition component is used to obtain depth image, then depth information can be acquired, but power consumption increases with higher resolution requirements
Solution Approach 1:
The color image serves as a mediator that carries high-resolution information without requiring the depth sensor to increase its resolution. The system leverages the existing color camera's high resolution to enhance the depth image, avoiding the need for a high-power, high-resolution depth sensor.
Solution Approach 2:
The patent creates a high-resolution depth image by copying and mapping information from the high-resolution color image. Instead of directly capturing high-resolution depth data (which would require more power), the system copies structural and textural information from the color image and applies it to the depth image through the objective function.
3Area of stationary object
If low resolution depth image is used, then chip area and power consumption are reduced, but surface texture details and face recognition accuracy are insufficient
Solution Approach 1:
The color image acts as an intermediary that provides high-frequency texture information to the depth image. By mapping color image pixels to depth image pixels and using the color image's detailed information to guide the depth image reconstruction, the system achieves high surface texture detail accuracy without needing a large chip area.
Solution Approach 2:
The patent enhances the depth image by incorporating information from another dimension (the color image). By establishing pixel correspondence between the color image and depth image, and using the color image's high-resolution data to supplement the depth image, the system achieves high surface texture detail accuracy without increasing the depth sensor's physical dimensions.
4Device complexity
If low resolution depth image is used, then device complexity is reduced, but face recognition accuracy decreases
Solution Approach 1:
The color image serves as an intermediary that provides high-resolution facial features and textures to the depth processing system. By mapping color image information to the depth image through the objective function, the system achieves high face recognition accuracy without significantly increasing device complexity, as the color camera is already present in most devices.
Solution Approach 2:
The patent makes the color camera serve multiple functions: it captures both color information for visual display and structural information for enhancing the depth image. By using the color image to guide depth image reconstruction, the system achieves high face recognition accuracy without adding dedicated high-resolution depth sensing hardware, thus maintaining low device complexity.
Data Source
AI summary
The present disclosure provides a method, an electronic device, and a storage medium for obtaining a depth image. The method includes: mapping a depth image having a first resolution into a color image having a second resolution to obtain a processed image, the first resolution being smaller than the second resolution; establishing an objective function with respect to the processed image and a depth image having the second resolution, pixels in the processed image having one-to-one correspondence with pixels in the depth image having the second resolution, the objective function aiming to minimize an error with respect to the processed image and the depth image having the second resolution; and solving the objective function to obtain the depth image having the second resolution.