Neural Image Restoring Circuit for Under-Display Camera Distortion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image restoration techniques fail to effectively address the broad range of issues, such as blurring, color shifting, and reduced light intensity, in images captured by under-display cameras due to light scattering and attenuation in display circuits, which exhibit spatially variable characteristics.
Innovation Solution
An image restoring circuit comprising a first and second restoring circuit with encoders and decoders, coupled by a coupling circuit, performs multiple restoration operations using neural networks to generate and refine output images, incorporating deformable convolution and frequency domain loss functions to enhance image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional image restoration techniques are used, then specific damages like noise or blur can be addressed, but they fail to simultaneously restore images with diverse types of damage including blurring, color shifting, and reduced light intensity
Solution Approach 1:
The image restoration task is segmented into multiple specialized sub-tasks, each handled by a dedicated restoring circuit. The first restoring circuit addresses certain types of damage while the second restoring circuit handles other types, allowing comprehensive restoration of diverse damage types including blurring, color shifting, and reduced light intensity without compromising restoration effectiveness
Solution Approach 2:
The system achieves universality by designing multiple restoring circuits that collectively handle all types of UDC image damage. Each circuit is specialized for particular damage types, but the combined system provides universal restoration capability across all damage categories, making the system adaptable to diverse restoration needs while maintaining high reliability
2Manufacturing precision
If a single restoring circuit is used, then the device complexity is low, but it cannot effectively address broad area issues with spatially variable characteristics
Solution Approach 1:
The restoration system is divided into multiple specialized circuits (first restoring circuit with first encoder-decoder, second restoring circuit with second encoder-decoder) that process different aspects of image damage. This segmentation enables high restoration quality by addressing spatially variable characteristics with targeted processing, while the modular structure keeps individual circuit complexities manageable
Solution Approach 2:
The system employs a nested architecture where the coupling circuit integrates outputs from the first encoder and first decoder, and feeds processed data to the second encoder. This nested structure allows complex restoration operations to be organized in hierarchical layers, improving manufacturing precision through systematic processing while organizing device complexity in a structured manner
3Manufacturing precision
If multiple restoration operations are performed through coupled circuits, then image quality metrics like PSNR and SSIM are significantly improved, but the device complexity increases
Solution Approach 1:
The coupling circuit is nested within the overall restoration system, receiving inputs from the first restoring circuit and feeding processed data to the second restoring circuit. This nested configuration enables multiple restoration operations to be performed in an organized sequence, significantly improving image quality metrics like PSNR and SSIM while containing device complexity within a structured hierarchical framework
Solution Approach 2:
The coupling circuit acts as an intermediary between the first and second restoring circuits, processing and transforming data from the first circuit before passing it to the second circuit. This intermediary role enables sophisticated multi-stage restoration operations that improve image quality metrics, while the dedicated intermediary structure helps manage overall device complexity by clearly defining data flow and processing stages
Data Source
AI summary
An image restoring circuit includes a first restoring circuit including a first encoder and a first decoder, and configured to generate a first output image and first tensor data by restoring an input image; a second restoring circuit including a second encoder and a second decoder, and configured to restore the input image by using an output of the first encoder, an output of the first decoder, and the first tensor data to thereby generate a second output image; and a coupling circuit configured to generate second tensor data based on the output of the first encoder and the output of the first decoder and provide the second tensor data to the second encoder.