Solid-State Imaging Element Without Color Filter for Low-Light AI Colorization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional imaging systems using color filters suffer from light attenuation, leading to insufficient light reception and poor colorization in low-light conditions, making it difficult to capture clear and accurate color images, especially in dark environments such as night or dimly lit areas.
Innovation Solution
An imaging system that employs a solid-state imaging element without a color filter, utilizing a learning device with a neural network for AI-driven colorization of monochrome image data, which includes training data converted into the HSV color space and thinned based on saturation histograms to enhance color fidelity and visibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a color filter is used in the imaging element, then color images can be obtained, but light attenuation occurs and the amount of received light is reduced
Solution Approach 1:
The patent removes the color filter from the imaging element, extracting the light-blocking component that causes attenuation. This allows maximum light to reach the photoelectric conversion layer, solving the contradiction between color accuracy and light quantity by eliminating the filtering step entirely and replacing it with post-processing colorization using AI
Solution Approach 2:
The patent performs monochrome imaging first to capture maximum light information, then applies preliminary colorization processing using trained AI models. This preliminary action of capturing pure light data before color processing resolves the contradiction by separating light collection from color determination
2Loss of information
If the amount of image data for training is increased, then the model learns more features, but the colorization becomes insufficient and images become sepia or gray
Solution Approach 1:
The patent changes the parameter of training data quantity to an optimal range (100-10000 images) rather than using excessive data. It also changes the training approach by focusing on specific color information and using multiple trained models for different scenarios, resolving the contradiction by finding the optimal data quantity and quality balance
Solution Approach 2:
The patent introduces an intermediary selection process that chooses appropriate trained models based on input image characteristics. Instead of using one model trained on all possible data, it mediates between different specialized models to achieve optimal colorization quality without requiring excessive training data for each scenario
3Ease of operation
If conventional image sensors with color filters are used in low-light conditions, then imaging can be performed, but the received light is insufficient and clear color images cannot be captured
Solution Approach 1:
The patent extracts and removes the color filter that blocks light in low-light conditions. By eliminating the filter, the imaging element can capture maximum available light, and color information is recovered through AI colorization of the monochrome data, enabling imaging in conditions where conventional filtered sensors would fail
Solution Approach 2:
The patent replaces the mechanical/optical color filtering system with an AI-based computational colorization system. Instead of using physical filters to separate colors, it uses trained neural networks to infer and add color information to monochrome images, achieving better low-light performance by substituting physical light manipulation with computational processing
Data Source
AI summary
Color filters are used for color images obtained using imaging devices such as conventional image sensors. Imaging elements with color filters are sold, and an appropriate combination of the imaging element and a lens or the like is incorporated in an electronic device. Only providing a color filter to overlap a light-receiving region of an image sensor reduces the amount of light reaching the light-receiving region.An imaging system of the present invention includes a solid-state imaging element without a color filter, a storage device, and a learning device. As a selection standard for reducing the amount of learning data, in an HSV color space, saturation is used, and selection is performed so that the saturation has optimal distribution. When colorization disclosed in this specification is performed, the colorization and object highlight processing can be performed at the same time.