Burst Image Deblurring Using Sensor-Guided Fourier Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current camera systems struggle to produce sharp and noise-reduced images in low-light environments due to camera shake, especially when using hand-held devices, as existing deblurring methods are computationally expensive and sensitive to blurring kernel estimation, and often introduce artifacts.
Innovation Solution
The method involves aggregating a burst of images by computing a weighted average of their Fourier coefficients, avoiding explicit blurring kernel computation and relying on the randomness of hand tremor, using gyroscope and accelerometer data for efficient alignment, and applying a post-processing filter to enhance image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multi-image deconvolution algorithms are used to remove blur, then image sharpness is improved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent segments the deblurring problem into two independent parts: (1) motion estimation using gyroscope/accelerometer data to obtain blur kernels, and (2) simple deconvolution using the estimated kernels. This segmentation avoids the computationally expensive joint estimation of motion and deconvolution, reducing complexity while maintaining image sharpness.
Solution Approach 2:
The patent performs motion estimation as a preliminary step before deconvolution, using available sensor data to predict blur kernels in advance. This preliminary action simplifies the subsequent deconvolution process, as the kernels are already estimated and don't require complex iterative optimization during the main processing stage.
2Manufacturing precision
If multi-image deconvolution algorithms are used to remove blur, then image sharpness is improved, but processing time increases by orders of magnitude
Solution Approach 1:
By separating motion estimation from deconvolution, the patent enables parallel processing where sensor data is processed independently from image data. This segmentation allows for optimized processing pipelines that reduce overall processing time while maintaining deblurring effectiveness.
Solution Approach 2:
The patent replaces complex iterative deconvolution mechanics with a simpler approach that uses sensor-derived motion models. Instead of relying on heavy computational mechanics to estimate blur, the system substitutes this with direct motion sensing from gyroscopes and accelerometers, significantly reducing processing time.
3Measurement precision
If explicit blurring kernel computation is performed, then deblurring accuracy is improved, but computational overhead and artifact introduction increase
Solution Approach 1:
The patent introduces sensor data (gyroscope and accelerometer readings) as an intermediary to estimate blur kernels. This intermediary provides physically grounded motion information that accurately reflects actual camera movement, leading to more accurate deblurring without the artifacts that arise from purely image-based kernel estimation methods.
Solution Approach 2:
The system uses the camera's own built-in sensors (gyroscope and accelerometer) to provide the motion information needed for deblurring. This self-service approach eliminates the need for external reference images or complex iterative kernel estimation, reducing computational overhead and avoiding artifacts while maintaining accuracy.
4Reliability
If prior assumptions on latent image are made, then deconvolution stability is improved, but image quality and naturalness deteriorate
Solution Approach 1:
The patent uses sensor data as an intermediary to provide physically grounded constraints on motion, replacing the need for image priors. This mediator provides stable deconvolution through actual measured motion rather than assumed image properties, maintaining both stability and image quality without artificial constraints.
Solution Approach 2:
The system relies on the camera's own sensor data rather than external priors or assumptions about the latent image. This self-service approach provides inherent stability through physical measurements while preserving image naturalness, as no artificial priors are imposed on the reconstruction process.
Data Source
AI summary
System and Method for automatically removing blur and noise in a plurality of digital images. The system comprises an electronic processor configured to receive the plurality of digital images, perform motion estimation and motion compensation to align the plurality of digital images, determine an alignment of the plurality of digital images with respect to a reference frame, generate a consistency map based on the alignment of the plurality of digital images with respect to the reference frame, combine the plurality of digital images aligned with respect to the reference frame in the Fourier domain using a quality of alignment information from the consistency map to generate an aggregated frame, and apply a post-processing filter to enhance the quality of the aggregated frame.