Burst Image Restoration With Motion-Embedding Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image restoration methods struggle to effectively improve the quality of burst images captured in poor lighting conditions or with camera movement, leading to degraded image quality due to variations in ambient light and motion.
Innovation Solution
An image restoration apparatus utilizing a neural network-based approach that encodes individual images, performs motion-aware information exchange between feature representations, and decodes the fused motion-embedding feature representations to generate a high-quality restored image, considering camera and object motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional image restoration methods are used on burst images, then the processing speed is fast, but the image quality is degraded due to inability to handle motion and lighting variations
Solution Approach 1:
The patent segments the burst image set into individual images and processes them through separate feature extraction and motion-aware fusion operations. Each image is encoded into feature representations, motion information is extracted and embedded, then all feature representations are fused to generate the final restored image. This segmentation allows complex processing to be applied to each component independently while maintaining overall image quality.
Solution Approach 2:
The patent introduces feature representations as intermediary structures between the input burst images and the final restored image. These feature representations serve as a mediator that captures both image content and motion information, enabling the system to process motion-aware information exchange without directly manipulating the original images, thus improving quality while managing complexity.
2Manufacturing precision
If motion-aware processing is applied to burst images, then the restored image quality improves, but the computational time and energy consumption increase
Solution Approach 1:
The patent performs preliminary actions by extracting motion information and embedding it into feature representations before the final fusion step. Motion-aware feature extraction is performed in advance on individual images, and motion embedding is prepared beforehand. This preliminary processing of motion information allows the main fusion operation to proceed more efficiently, reducing overall computational energy while maintaining high restored image quality.
3Measurement precision
If deep learning-based neural networks are used for image restoration, then the mapping accuracy between input and output is high, but the training requirements and data needs are substantial
Solution Approach 1:
The patent implements self-service by using the burst image set itself as the training data source. The system extracts features and motion information from the burst images and uses these self-generated features for training the neural network, eliminating the need for external training datasets. This self-service approach enables high mapping accuracy while significantly reducing the quantity of external training data required.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and apparatus for image restoration based on burst images. The method includes generating a plurality of feature representations corresponding to individual images of a burst image set by encoding the individual images (810), determining a reference feature representation from among the plurality of feature representations (820), determining a first comparison pair including the reference feature representation and a first feature representation of the plurality of feature representations (830), generating a first motion-embedding feature representation of the first comparison pair based on a similarity score map of the reference feature representation and the first feature representation (840), generating a fusion result by fusing a plurality of motion-embedding feature representations including the first motion-embedding feature representation (850), and generating at least one restored image by decoding the fusion result (860).