Continuous Kernel Neural Network for Burst Image Enhancement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional imaging systems face limitations in accuracy, flexibility, and efficiency due to rigid architectures, such as discrete kernels and Gaussian kernel forms, which lead to information loss and sub-optimal performance in handling noise and varying image contexts.
Innovation Solution
A continuous kernel neural network that learns continuous reconstruction kernels to merge digital image samples in local neighborhoods, allowing for flexible adaptation and efficient image enhancement without resampling, using neural implicits to represent kernels and condition them on local image information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If discrete kernels are used to combine digital images, then the system has a defined architecture for image processing, but information integrity is lost due to required resampling operations
Solution Approach 1:
The patent changes the fundamental parameter of kernel representation from discrete to continuous. Instead of using fixed discrete kernels that require resampling, the system employs continuous kernels defined by mathematical functions (e.g., Gaussian kernels) with adjustable parameters like sigma. This allows operating on original image data without resampling, preserving information integrity while maintaining a well-defined processing architecture.
2Ease of manufacture
If Gaussian kernels are used to combine digital images, then the system has a simple kernel form, but flexibility is reduced to adapt to different image contexts
Solution Approach 1:
The patent introduces dynamics by making kernel parameters adaptive rather than static. The sigma parameter of Gaussian kernels is adjusted dynamically based on local image characteristics such as gradient magnitude and orientation. This allows the same Gaussian kernel form to adapt to different image contexts (edges, smooth regions, textures) without requiring multiple fixed kernel types, thus maintaining simplicity while gaining flexibility.
3Measurement precision
If implicit neural representations are used to learn continuous scene functions, then the system can reconstruct continuous scenes, but computing resources and memory requirements increase significantly
Solution Approach 1:
The patent applies local quality by computing kernel parameters locally for each pixel or small neighborhood rather than learning a single global scene representation. Each pixel's kernel is determined by local image gradients and statistics, which are computationally inexpensive to calculate. This local approach achieves accurate continuous reconstruction without the heavy computational burden of global implicit neural representations, as each local computation is independent and simple.
Data Source
AI summary
The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize a continuous kernel neural network that learns continuous reconstruction kernels to merge digital image samples in local neighborhoods and generate enhanced digital images from a plurality of burst digital images. For example, the disclosed systems can utilize an alignment model to align image samples from burst digital images to a common coordinate system (e.g., without resampling). In some embodiments, the disclosed systems generate localized latent vector representations of kernel neighborhoods and determines continuous displacement vectors between the image samples and output pixels of the enhanced digital image. The disclosed systems can utilize the continuous kernel network together with the latent vector representations and continuous displacement vectors to generated learned kernel weights for combining the image samples and generating an enhanced digital image.