Temporal Filtering Weight Maps for Motion-Aware Ghosting Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mixed-reality systems face challenges in temporal filtering due to ghosting effects and camera movement, leading to suboptimal image quality, especially in low light conditions.
Innovation Solution
A novel filtering weight computation method that combines motion compensation with inertial measurement unit data to generate a final weight map, adjusting the influence of images based on camera motion and object movement, thereby enhancing image quality and reducing ghosting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If multiple consecutive images are combined via temporal filtering to improve image quality in low light conditions, then image quality is improved, but ghosting effects occur when objects or cameras move during capture
Solution Approach 1:
The patent performs motion compensation on the first image before combining it with the second image. This preliminary action aligns the first image to the second image's coordinate system, ensuring that stationary objects occupy the same pixel locations. This resolves the ghosting problem while maintaining the image quality improvement benefits of temporal filtering in low light conditions.
Solution Approach 2:
The patent generates a weight map with different weight values for different regions of the image. Regions with significant motion differences receive lower weights, while regions with minimal motion receive higher weights. This local differentiation allows the system to combine images effectively in stationary regions while minimizing ghosting effects in moving regions.
2Illumination intensity
If temporal filtering combines multiple images to reduce noise and improve quality, then image quality improves, but computational complexity increases
Solution Approach 1:
The patent applies motion compensation and weight map generation selectively rather than uniformly across all images. By focusing computational resources on regions with significant motion differences and using the weight map to reduce processing in stable regions, the system achieves improved image quality while minimizing unnecessary computational overhead.
Solution Approach 2:
The patent divides the image processing into distinct segments: motion compensation, weight map generation, and weighted combination. This segmentation allows each computational step to be optimized independently and enables parallel processing of different image regions, reducing overall computational complexity while maintaining image quality improvements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques to temporally filter images via a filtering weight computation are disclosed. A first image having a first timestamp and a second image having a second timestamp are acquired. These images are generated by a camera, and the first timestamp is before the second timestamp. A motion compensation (MC) operation is performed on the first image to produce an MC image. A difference image is generated using the MC image and the second image. The difference image reflects differences in intensities that exist between the two images. A local weight map is generated based on those differences. A global weight map is generated based on certain IMU data. A final weight map is generated based on a combination of the local weight map and the global weight map. The final weight map is used to generate a filtered image.