Multi-Depth Sensor Pixel Correction for Overlap Defects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When generating depth images using multiple depth sensors with overlapping fields of view, defects such as defective pixels occur due to mutual interference and lens distortion, leading to degraded image edges and abnormal depth values.
Innovation Solution
Utilize a trained machine learning model to correct defective pixels by processing depth images obtained from multiple sensors, identifying pixel correction positions, and applying corrections using neural networks to generate wide-angle depth images without defects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple depth sensors with overlapping fields of view are used to generate a wider angle depth image, then the field of view coverage is improved, but defects such as abnormal depth values and degraded image edges occur due to mutual interference and lens distortion
Solution Approach 1:
The patent divides the depth image into multiple regions including overlapping regions and non-overlapping regions. Different processing strategies are applied to each region: the overlapping region is processed using machine learning to identify and correct defective pixels, while the non-overlapping region is processed using conventional depth mapping. This segmentation allows the system to maintain wide field of view coverage while ensuring high depth map quality in critical regions.
Solution Approach 2:
The patent introduces a machine learning model as an intermediary component between the multiple depth sensors and the final depth image. This model processes the depth information from overlapping sensor fields, identifies defective pixels caused by mutual interference, and generates corrected depth values. The intermediary ML model acts as a mediator that reconciles the conflicting requirements of wide coverage and high precision.
2Manufacturing precision
If depth images from multiple sensors are combined to correct defective pixels, then the depth map quality is improved, but the processing complexity increases
Solution Approach 1:
The patent applies different processing qualities to different regions of the depth image. The machine learning-based correction is applied locally only to the overlapping region where defective pixels occur, while the non-overlapping region uses simpler processing. This local quality approach improves depth map quality where needed without unnecessarily increasing processing complexity across the entire image.
Solution Approach 2:
The patent performs partial correction by focusing machine learning processing only on the overlapping region rather than the entire depth image. This partial action approach corrects the defective pixels in the critical overlapping area while avoiding the excessive processing complexity that would result from applying the same correction method to the entire image, including regions that don't require correction.
Data Source
AI summary
The present disclosure relates to an information processing device, an information processing method, and a program capable of correcting a correction target pixel caused by a field of view of a sensor.Provided is an information processing device including a processing unit that performs processing using a trained model trained by machine learning on at least a part of a first image in which a subject acquired by a first sensor is indicated by depth information, a second image in which the subject acquired by a second sensor is indicated by depth information, and a third image obtained from the first image and the second image to correct a correction target pixel included in any of the images. The present disclosure can be applied to, for example, a device having a plurality of depth sensors.