Image Pickup Apparatus Noise Inference Using Learned Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image pickup apparatuses face challenges in accurately detecting and correcting noise, such as random telegraph noise (RTN) and pixel defects, which can lead to erroneous detection and reduced signal-to-noise ratio, especially when these defects are not registered in advance.
Innovation Solution
An image pickup apparatus that includes a solid-state image pickup device, a learning unit for generating a learned model using machine learning with first correction information, and an inference unit to infer noise in pixel signals, improving detection accuracy by using a neural network-based model to correct noise not accounted for in initial correction information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional noise detection methods using predetermined thresholds are used, then the detection process is simple and fast, but the detection accuracy is low and erroneous detection occurs for unknown noise types
Solution Approach 1:
A learned model serves as an intermediary between the input image data and the noise detection process. The model, trained on supervised data containing noise patterns, acts as a mediator that transforms raw pixel data into accurate noise identification, resolving the contradiction between simple detection and accurate detection.
Solution Approach 2:
The learned model is trained in advance using supervised data that includes various noise patterns and correction information. This preliminary training action enables the model to recognize different noise types accurately during actual operation, improving detection precision without increasing real-time complexity.
2Measurement precision
If a learned model trained on large amounts of diverse image data is used, then the noise detection accuracy is improved, but the data preparation and model training complexity increases
Solution Approach 1:
The system uses its own correction information and image data to generate supervised training data automatically. This self-service approach eliminates the need for external data collection and manual annotation, making the model training process easier while maintaining high detection accuracy.
Solution Approach 2:
The learned model is designed to handle multiple types of noise (salt-and-pepper noise, Gaussian noise, RTN) and various image conditions simultaneously. This multi-functionality allows a single model to achieve high detection accuracy across diverse scenarios without requiring separate models for each noise type.
3Productivity
If correction information is registered in advance for known defective pixels, then the correction process is efficient, but unknown noise and defects occurring over time cannot be corrected
Solution Approach 1:
The system implements a feedback mechanism where the learned model continuously analyzes image data and generates correction information for both known and unknown noise types. This feedback loop enables the system to adapt to new noise patterns over time while maintaining efficient correction processing through the use of learned patterns.
Solution Approach 2:
The correction system transitions from a static, pre-registered correction approach to a dynamic system that continuously learns and adapts. The learned model dynamically identifies and corrects various noise types based on current image data, enabling the system to handle both predetermined and emerging noise patterns effectively.
Data Source
AI summary
An image pickup apparatus includes a solid-state image pickup device, a learning unit configured to generate a learned model by performing machine learning using, as supervised data, first correction information for identifying a pixel signal for which noise is to be corrected among a plurality of pixels of the solid-state image pickup device and using, as input, an image acquired from the solid-state image pickup device and that has not been corrected based on the first correction information, and an inference unit configured to infer a pixel signal on which noise is superimposed by inputting an image corrected based on the first correction information to the learned model.