Dual-Field-of-View Image Fusion Using Coordinate-Guided Deep Learning
Solution Overview
Problem
Dual-camera systems in electronic devices, such as mobile phones, fail to capture high-quality images due to mismatched fields of view, resulting in inconsistent definition and poor stereoscopic effects when images are simply fused.
Innovation Solution
An image processing method that adds a reference coordinate image layer to one image with a different field of view, processed using a deep learning network model to enhance the image, incorporating denoising, demosaicing, and color fusion to improve image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If images from dual cameras with different fields of view are simply fused, then the processing is quick and simple, but the image quality is poor with inconsistent definition and fused boundaries
Solution Approach 1:
The patent segments the image processing into multiple stages: first processing each camera's image separately (denoising, demosaicing, color fusion) to obtain first enhanced images, then fusing these enhanced images through coordinate mapping to produce the final second enhanced image. This segmentation allows each stage to be optimized independently, resolving the contradiction between simple fusion and high quality output.
Solution Approach 2:
The patent applies preliminary processing actions to each camera's image before fusion. The first enhanced images are obtained through denoising, demosaicing, and color fusion operations performed separately on each camera's output. These preliminary enhancements ensure that when images are fused later, the definition is consistent and fused boundaries are eliminated, while still maintaining relatively quick processing.
2Device complexity
If images with different fields of view are directly fused, then the processing complexity is low, but the stereoscopic effect is poor and fused boundaries appear
Solution Approach 1:
The patent introduces coordinate mapping as an intermediary mechanism between images from different cameras. The coordinate mapping unit establishes correspondence relationships between pixels of different cameras based on their respective coordinate systems, enabling accurate alignment and fusion. This intermediary step ensures that the stereoscopic effect is natural and fused boundaries are eliminated, while adding only moderate processing complexity.
Solution Approach 2:
The patent transforms images from different coordinate systems into a unified coordinate system through coordinate mapping. By changing the parameter representation of image pixels according to their coordinate transformations, the patent enables accurate fusion of images with different fields of view, improving the stereoscopic effect without requiring overly complex processing.
3Speed
If simple fusion is used for dual camera images, then the processing is fast, but details are lost and image definition is inconsistent
Solution Approach 1:
The patent segments the processing into independent enhancement stages for each camera, applying denoising, demosaicing, and color fusion separately to obtain first enhanced images. This segmentation allows detailed processing to be applied efficiently to each image before fusion, preserving details and ensuring consistent definition while maintaining relatively fast processing through parallelizable operations.
Solution Approach 2:
The patent performs preliminary enhancement actions on each camera's image before the fusion step. By applying denoising, demosaicing, and color fusion operations in advance to generate first enhanced images, the patent ensures that when fusion occurs, the definition is consistent and details are preserved. This preliminary action approach maintains processing speed by preparing images in advance for efficient fusion.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
This application provides an image processing method and apparatus and an electronic device, and relates to the field of image technologies. The image processing method includes: acquiring a plurality of frames of original images; adding a reference coordinate image layer to a second field-of-view image; obtaining an image layer set according to a first field-of-view image, the second field-of-view image, and the reference coordinate image layer; processing, by using a deep learning network model, the image layer set to obtain a first enhanced image; and obtaining a second enhanced image according to the first enhanced image. In the method, since the added reference coordinate image layer reflects a mapping relationship between a field of view corresponding to the first field-of-view image and a field of view corresponding to the second field-of-view image, through the addition of the reference coordinate image layer, priori information can be added, so that different adjustments can be made subsequently according to different field-of-view relationships. Therefore, two images having different fields of view are more naturally fused, and a purpose of improving image quality is achieved.