Multi-view Image Synthesis Using Sharpness-Based Pixel Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-view imaging technologies face challenges in handling motion blur, leading to inaccurate depth and texture data, which results in erroneous positioning of fast-moving objects in synthesized images due to differences in camera poses and exposure times.
Innovation Solution
A method that determines sharpness indications for each image to generate confidence scores, which are used to weight pixels during image blending, thereby reducing the impact of motion blur and improving the accuracy of depth estimation and image synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If images from multiple cameras with different poses and exposure times are used for multi-view synthesis, then the coverage and viewpoint diversity are improved, but motion blur varies across images causing inaccurate depth estimation and erroneous object positioning
Solution Approach 1:
The patent applies local quality by determining sharpness indications for different regions within images rather than treating entire images uniformly. By analyzing sharpness at pixel or region levels, the system can identify and weight sharp regions higher than blurred regions, allowing accurate depth estimation in sharp areas while tolerating blur in other areas. This resolves the contradiction by maintaining measurement precision locally where possible while preserving viewpoint diversity globally.
Solution Approach 2:
The patent changes the parameter of image weighting from uniform to variable based on sharpness indications. By introducing confidence scores derived from sharpness analysis, the system dynamically adjusts the weight of each image or image region in the multi-view synthesis process. This allows the system to adaptively prioritize images with less motion blur, thereby maintaining depth estimation accuracy while still utilizing multiple viewpoints for comprehensive scene coverage.
2Reliability
If fast-moving objects are captured from multiple viewpoints, then the completeness of object tracking is improved, but motion blur causes the objects to appear semi-transparent and lack texture, making depth determination difficult
Solution Approach 1:
The patent merges information from multiple images by combining their sharpness indications and confidence scores. When one image suffers from motion blur and loses texture information, the system combines it with other images that may have captured the same object with less blur. This merging process allows the system to maintain reliable object tracking by aggregating complementary information across multiple viewpoints, recovering texture details that are missing in individual blurred images.
Solution Approach 2:
The patent implements feedback by using sharpness indications to guide the image selection and weighting process. The system evaluates the sharpness of each image, feeds this information back into the synthesis process, and adjusts the contribution of each image accordingly. This feedback mechanism ensures that images with better sharpness (and thus more texture information) are given higher weights, maintaining object tracking reliability while compensating for information loss in blurred regions.
3Device complexity
If images with different degrees of motion blur are blended equally, then the processing simplicity is maintained, but the synthesized image contains wrongly positioned objects and artifacts
Solution Approach 1:
The patent applies preliminary action by determining sharpness indications and confidence scores for each image before performing the blending operation. This pre-processing step evaluates the quality of each image in advance, allowing the system to assign appropriate weights to each image before synthesis. By performing this quality assessment beforehand, the system avoids the need for complex iterative optimization during blending, maintaining processing simplicity while ensuring synthesis accuracy through informed weight selection.
Data Source
AI summary
A method for processing multi-view data of a scene. The method comprises obtaining at least two images of the scene from different cameras, determining a sharpness indication for each image and determining a confidence score for each image based on the sharpness indications. The confidence score is for use in the determination of weights when blending the images to synthesize a new virtual image.