Permutation-Invariant HDR Imaging via Symmetric Feature Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for high dynamic range (HDR) imaging face challenges in capturing detailed scenes with broad luminosity ranges, particularly in single-image HDR processing, which relies on hallucinating details that are under- or over-exposed, and multi-frame HDR methods struggle with accurate alignment and motion issues.
Innovation Solution
A permutation invariant deep learning model that performs symmetric pooling operations on feature representations from one or multiple low dynamic range (LDR) images, enabling denoising, dequantization, and reconstruction of under- and over-exposed regions, allowing for flexible input ordering and handling of images from different sensors with varying noise levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If multi-frame HDR fusion is used to capture broad luminosity ranges, then dynamic range coverage is improved, but alignment accuracy and processing reliability deteriorate due to motion and registration difficulties
Solution Approach 1:
The patent segments the multi-frame HDR problem into independent single-image HDR processing units. Each input image is processed separately through the neural network to generate feature representations, which are then combined through symmetric pooling. This segmentation eliminates the need for frame alignment while maintaining the benefit of multiple exposure values, resolving the contradiction between dynamic range coverage and alignment reliability.
2Reliability
If single-image HDR processing is used to avoid alignment issues, then processing reliability is improved, but luminosity range coverage deteriorates due to reliance on hallucinating details
Solution Approach 1:
The patent merges multiple single-image HDR processing results through symmetric pooling operations. The neural network processes each input image independently to maintain reliability, then the feature representations from multiple images are combined using symmetric pooling (max pooling, average pooling, or sum pooling). This merging allows the system to achieve broad luminosity range coverage by leveraging information from multiple exposures while maintaining the reliability of single-image processing.
3Adaptability or versatility
If permutation invariant processing is implemented to handle flexible input ordering, then adaptability is improved, but computational complexity increases due to symmetric pooling operations
Solution Approach 1:
The patent implements universal symmetric pooling operations that can process any number of input images in any order without requiring reconfiguration. The same pooling mechanism (max, average, or sum) works regardless of input count or sequence, providing permutation invariance. This universality achieves adaptability to flexible input ordering while keeping computational complexity manageable through efficient pooling implementations that scale well with the number of inputs.
Data Source
Figure 1(a)~2(b)
Figure 3
Figure 4(a)~4(b)
AI summary
Described herein is an image processing apparatus (1200) for forming an enhanced image, the apparatus comprising one or more processors (1204) configured to: receive (1101) one or more input images (301, 302, 404, 405, 406, 501, 502, 503, 601, 602, 603); form (1102) from each of the one or more input images a respective feature representation, each feature representation representing features of the respective input image; and subject (1103) the one or more feature representations to a symmetric pooling operation (305) to form an enhanced image (408, 505, 608) from at least some of the features of the one or more feature representations identified by the symmetric pooling operation. The apparatus may result in images with increased photoreceptive dynamic range, increased bit depth and signal-to-noise ratio, with less quantization error and richer colour representation.