Multipass Interference Correction via Patterned Illumination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D vision technologies, such as time-of-flight (ToF) sensors, suffer from multipass interference (MPI) that leads to inaccurate depth determination, shape distortion in images, and inability to detect mirrors, resulting in poor augmented reality (AR) performance and navigation errors due to complex computations and frame rate reduction.
Innovation Solution
A method using at least two light sources with structured light of differing spatial patterns, wavelengths, or polarizations, captured by an imaging sensor with a filter array, where each pixel calculates intensity values of direct and global components to correct MPI, allowing for real-time processing and detection of mirrors without significant hardware modifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple images (frames) are captured to obtain a corrected image, then MPI correction quality is improved, but frame rate is reduced causing motion artifacts
Solution Approach 1:
The patent applies preliminary action by using a depth map obtained from a previous frame to guide the MPI correction process in the current frame. Instead of capturing multiple frames for correction, the system proactively uses previously acquired depth information to identify and correct MPI-affected regions in real-time, thereby maintaining high frame rate while achieving effective MPI correction.
2Measurement precision
If conventional ToF or i-ToF sensors are used, then depth determination is performed, but MPI causes large errors (more than 10%) in depth determination
Solution Approach 1:
The patent introduces a depth map as an intermediary element that mediates between the raw ToF measurements and the final corrected depth image. The depth map from the previous frame serves as a reference to identify MPI-affected regions, allowing the system to selectively correct only those areas where MPI is present, thereby improving depth determination accuracy without requiring complete re-capture of multiple frames.
Solution Approach 2:
The patent replaces the traditional mechanical approach of capturing multiple physical frames with a computational method that uses a single frame combined with a depth map. Instead of relying on temporal redundancy through multiple captures, the system substitutes a computational correction algorithm that leverages depth information to identify and correct MPI artifacts, thereby maintaining real-time performance.
3Measurement precision
If complex computations are performed for MPI correction, then correction quality may be improved, but real-time processing cannot be achieved
Solution Approach 1:
The patent applies local quality by performing MPI correction only in regions where MPI is detected to be present, rather than processing the entire image uniformly. The system uses the depth map to identify MPI-affected local regions and applies correction algorithms selectively to those areas, thereby reducing overall computational load and enabling real-time processing while maintaining correction quality in problematic regions.
4Shape
If conventional imaging sensors are used, then image capture is performed, but MPI results in shape distortion of captured objects in 2D and 3D images
Solution Approach 1:
The patent uses a depth map as an intermediary to identify regions suffering from shape distortion due to MPI. By comparing the current frame's depth information with the previous frame's depth map, the system locates areas where shape distortion has occurred and applies targeted correction to restore accurate object shapes in both 2D and 3D representations.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides accurate depth determination, improved image quality, and enhanced AR performance with reduced computational costs and frame rate, enabling efficient MPI correction and mirror detection in real-time without complex hardware changes.
Implementation Method 1
capturing an image of the scene simultaneously illuminated by the at least two light sources by an imaging sensor through a filter array, wherein one pixel of the imaging sensor captures the image through one filter of the filter array
Implementation Method 2
capturing an image of the scene simultaneously illuminated by the at least two light sources by an imaging sensor through a filter array
Data Source
AI summary
The method includes simultaneously illuminating a scene by at least two light sources, each light source emitting structured light having a spatial pattern, a wavelength and/or a polarization, wherein the spatial pattern, the wavelength and/or the polarization of each structured light differ from each other, respectively, capturing an image of the scene simultaneously illuminated by the at least two light sources by an imaging sensor through a filter array, wherein one pixel of the imaging sensor captures the image through one filter of the filter array, calculating, for each pixel, intensity values of direct and global components of the light received by the pixel from a system of equations compiled for each joint pixel, and performing, for each pixel, image correction by assigning to each pixel its calculated intensity value of the direct component to obtain a corrected image.