Multi-Sensor Image Reconstruction for Obstruction-Free Scene Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing methods fail to perfectly remove obstructions, often leaving portions of the obstruction or object of interest missing, leading to reduced image quality and loss of information.
Innovation Solution
A method and electronic device using multiple image sensors and deep neural networks (DNNs) to capture and align images from different angles, determining parallax shifts, and reconstruct occluded portions using deep transfer techniques to enhance obstruction removal and image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single input image is used for obstruction removal, then the processing is simple and fast, but the obstruction removal is imperfect and information is lost
Solution Approach 1:
The patent transitions from single-image processing to multi-image processing by introducing a second input image captured from a different viewpoint. This dimensional change enables the system to access additional information about occluded regions, improving obstruction removal quality while maintaining reasonable system complexity through structured processing pipelines.
Solution Approach 2:
The patent introduces an intermediary processing stage that includes generating depth maps and segmentation masks from the input images. These intermediaries facilitate the obstruction removal process by providing spatial relationships and object boundaries, enabling more accurate reconstruction while organizing complexity into manageable processing stages.
2Loss of information
If multiple images from different angles are used, then occluded portions can be reconstructed, but the system complexity and processing time increase
Solution Approach 1:
The patent performs preliminary actions by pre-processing the input images to generate depth maps and segmentation masks before the main obstruction removal process. This preparation step organizes the multi-image data into structured representations that accelerate the subsequent reconstruction process, reducing overall processing time while maintaining high reconstruction quality.
Solution Approach 2:
The patent implements a continuous processing pipeline where depth maps, segmentation masks, and reconstructed images are generated in sequential stages. Each stage builds upon the previous one, ensuring that useful information is continuously extracted and utilized, which optimizes processing efficiency while maintaining comprehensive information recovery.
3Manufacturing precision
If deep neural networks are used for obstruction removal, then the removal quality improves, but the computational resources and processing time increase
Solution Approach 1:
The patent segments the image processing task into distinct modules: depth map generation, segmentation mask creation, and obstruction removal. Each module can be processed independently using appropriate computational resources, allowing the system to achieve high precision obstruction removal while optimizing energy consumption by applying deep neural networks only where necessary rather than uniformly across the entire process.
Data Source
AI summary
A method for obtaining a reconstructed image is provided. The method includes capturing, by the electronic device, a first sensor image and a second sensor image including a scene and an obstruction in the scene. The method includes generating, by the electronic device, an obstruction-free first image, a first obstruction template, an obstruction-free second image and a second obstruction template. Further, determining, a parallax shift between the obstruction-free first image and the obstruction-free second image and aligning the obstruction-free second image with respect to the obstruction-free first image and the second sensor image with respect to the first sensor image using the determined parallax shift. Further, the method includes determining occluded portions in the first sensor image and in the obstruction-free first image at corresponding locations in the aligned second sensor image and aligned obstruction-free second image respectively and obtaining the reconstructed obstruction-free first image.