Image Sensor Neural Network Processor Super-Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera technologies face challenges in achieving high-resolution images without increasing complexity and cost, particularly when photographing moving objects, and existing software-based solutions are difficult to implement in resource-constrained devices like smartphones and IoT devices due to high processing demands and expensive application processors.
Innovation Solution
A camera module and optical device that utilizes a separate processor on the image sensor to perform pre-processing of Bayer patterns, generating higher resolution images through deep learning algorithms, thereby reducing the burden on application processors and enabling efficient image processing without the need for expensive APs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical zoom is used to obtain high resolution images of distant objects, then image resolution is improved, but device complexity and manufacturing cost increase due to additional lens parts
Solution Approach 1:
The patent replaces the mechanical optical zoom system with a computational approach using a neural network processor that performs super-resolution processing on images captured by a fixed lens system. This substitution eliminates the need for moving lens parts while achieving high-resolution images of distant objects through software-based image enhancement algorithms.
2Measurement precision
If sensor shift technology or OIS technology is used to generate more pixel information, then image resolution is improved, but image quality deteriorates when photographing moving objects due to motion blur and artifacts
Solution Approach 1:
The patent replaces mechanical sensor shift or OIS technologies with a computational neural network-based super-resolution system. This approach processes images captured by a stationary sensor, eliminating motion-induced artifacts and blur while enhancing resolution through learned image priors and computational algorithms.
3Measurement precision
If software-based super resolution algorithms are implemented to achieve high resolution images, then image resolution is improved, but processing power requirements increase making it difficult to implement in resource-constrained devices
Solution Approach 1:
The patent extracts the computationally intensive super-resolution processing function from the main application processor and implements it in a dedicated neural network processor. This extraction allows complex image processing to be performed with lower power consumption and reduced computational burden on the main processor, enabling implementation in mobile and IoT devices.
Solution Approach 2:
The patent introduces a dedicated neural network processor as an intermediary component between the image sensor and the application processor. This intermediate processing unit handles the computationally demanding super-resolution tasks, allowing the main processor to focus on other functions while reducing overall power consumption and processing requirements.
4Measurement precision
If a separate image processor is mounted to implement super resolution algorithms, then image resolution is improved, but manufacturing cost increases
Solution Approach 1:
The patent merges the neural network processor with the image sensor to form an integrated image processing unit. This integration reduces the total component count and assembly complexity, thereby lowering manufacturing costs while still providing advanced super-resolution capabilities through dedicated hardware acceleration.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution allows for the economical manufacture of camera modules and optical devices with continuous zoom functionality, reduces power consumption, and optimizes image processing, enabling high-resolution images in resource-limited devices without the need for expensive hardware.
Implementation Method 1
an image sensor that converts an optical signal received from the outside into an electrical signal
Data Source
AI summary
An image sensor according to one embodiment may comprise: an image sensing unit which receives light and generates a first Bayer pattern having a first resolution; and a processor which receives a second Bayer pattern that is at least a portion of the first Bayer pattern from the image sensing unit, and then generates a third Bayer pattern having a higher resolution than the second alignment unit on the basis of the received second Bayer pattern.