Multiview Video Noise Reduction via Depth Map Pixel Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multiview video processing techniques fail to effectively address noise reduction across multiple camera views with different camera parameters and noise structures, leading to artifacts and detail loss in 3D displays, as they rely on single-camera based methods that do not utilize 3D structure information effectively.
Innovation Solution
A method that uses 3D structure estimation and dense depth maps to match pixels across multiple cameras, applying histogram equalization for color consistency, and median filtering to remove noise, considering occlusion and visibility, to achieve noise-free views with different camera parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If single-camera based noise reduction methods are used, then the processing complexity is low, but the noise reduction effectiveness is poor and artifacts occur in 3D displays
Solution Approach 1:
The patent merges multiple camera views into a unified processing framework. By combining observations from multiple cameras observing the same scene, the system creates a more robust noise reduction process that leverages the redundancy across views to achieve better denoising performance while maintaining processing feasibility through systematic integration of depth information and pixel matching.
Solution Approach 2:
The patent introduces depth maps as an intermediary element that facilitates the matching of corresponding pixels across multiple camera views. This depth-based intermediary enables the system to identify and compare pixels representing the same 3D point from different viewpoints, creating a bridge between multi-view observations that would otherwise be difficult to align and compare effectively.
2Reliability
If multiple camera views are processed independently, then the processing is simple, but additional observations from other cameras are not utilized
Solution Approach 1:
The patent merges multiple camera views into a unified processing framework. By combining observations from multiple cameras observing the same scene, the system creates a more robust noise reduction process that leverages the redundancy across views to achieve better denoising performance while maintaining processing feasibility through systematic integration of depth information and pixel matching.
Solution Approach 2:
The patent transitions from processing single 2D images to processing multiple 2D views in a 3D context. By introducing the depth dimension through depth maps and treating the problem as multi-view stereo processing, the system exploits the additional spatial dimension provided by multiple camera positions to gather more observations of the same scene points, thereby improving noise reduction effectiveness.
3Reliability
If depth-based pixel matching is applied, then color consistency and noise reduction improve, but processing time increases
Solution Approach 1:
The patent performs preliminary actions by first estimating depth maps for each camera view before proceeding with the noise reduction process. This preliminary depth estimation enables subsequent pixel matching operations to be performed more efficiently, as the depth information is already available to guide the correspondence search between views, avoiding the need for computationally expensive exhaustive matching during the actual denoising stage.
Solution Approach 2:
The patent applies local quality processing by performing pixel matching and noise reduction operations on a per-pixel or per-region basis rather than globally. By processing pixels in localized regions and using depth-based correspondence to identify matching points, the system achieves color consistency and effective noise reduction while reducing overall processing time through parallelizable local operations.
Data Source
Figure 1~3-c
Figure 4~5
Figure 6
AI summary
This invention relates to method for removing noise in multiview video by multiple cameras setup. This method comprises several steps which are normalizing the colour and intensity of the images; then choosing a reference image; reducing temporal noise for each channel motion compensation or frame averaging independently; mapping each pixel in the reference camera to the other camera views; determining the visibility of the corresponding pixel, after mapped to the other images by comparing the depth value; checking RGB range of the candidates with the corresponding pixel within the visible observations, then among stored RGB values from the visible regions of a pixel in the reference view, getting the median value and assigning this value to the reference pixel and all the other pixels matched to the reference pixel after mapping through depth map; and repeating the said steps until all of the pixels in each view are visited.