Image Sensor Pixel Storage Nodes for Flash-Matting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional flash-matting operations in imaging systems misidentify moving foreground objects as background objects due to the difference in pixel values between flash and no-flash images captured at different times, reducing their effectiveness.
Innovation Solution
The implementation of time-multiplexed flash-image and no-flash-image data capture using multiple storage nodes in image sensor pixels, where flash-image data is captured and stored during one phase and no-flash-image data during another, allowing for the generation of combined images that accurately distinguish foreground and background objects even with moving objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional flash-matting operations capture flash and no-flash images at different times, then the difference in pixel values can be used to distinguish foreground and background objects, but moving objects will appear at different locations in the two images causing misidentification
Solution Approach 1:
The patent applies periodic action by using time-multiplexed capture where the flash and no-flash images are acquired in alternating time slots within a single exposure period. This periodic switching allows both types of images to be captured simultaneously in time, eliminating motion-related misidentification while maintaining the ability to distinguish foreground from background through flash-induced brightness differences.
Solution Approach 2:
The patent introduces a temporal dimension to the image capture process by using multiple storage nodes (first and second storage nodes) to separate flash and no-flash image data in the time domain. This dimensional separation allows the system to maintain distinct image data streams without spatial interference, enabling accurate foreground-background distinction even when objects move during the exposure period.
2Ease of manufacture
If flash and no-flash images are captured at different times, then background objects can be identified through pixel value differences, but the capture time difference causes moving objects to be misidentified as background
Solution Approach 1:
The patent segments the image capture and storage process by dividing each pixel into multiple storage nodes (first storage node for flash images, second storage node for no-flash images). This segmentation allows independent storage and processing of flash and no-flash data, maintaining the simplicity of the flash-matting operation while improving object classification accuracy through proper temporal separation of image data.
3Reliability
If multiple storage nodes are used to capture time-multiplexed flash and no-flash images, then moving objects can be correctly identified, but the device complexity increases
Solution Approach 1:
The patent implements multi-functionality by designing image sensor pixels with multiple storage nodes that can serve different purposes. The same pixel structure with multiple storage nodes can capture both flash and no-flash images, perform flash-matting operations, and handle moving object detection, eliminating the need for separate dedicated hardware for each function and reducing overall system complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables accurate identification and removal or replacement of background objects in images, including moving objects, by ensuring that the difference in image data is solely due to flash intensity, thereby enhancing the effectiveness of flash-matting operations.
Implementation Method 1
a first image is captured and readout using the flash
Implementation Method 2
image sensor pixels, each having multiple charge storage regions coupled to the image sensor pixels respective photosensitive elements
Data Source
AI summary
Electronic devices may be provided with image sensors and light sources. The image sensors may include image pixels each having a photosensitive element, first and second storage nodes, and first and second transfer transistors coupled between the photosensitive element and the first and second storage nodes. The first and second transfer transistors may be synchronized with the light source so that charges generated by the photosensitive element of each image pixel when the light source is on are transferred to the first storage node of that pixel and charges generated by the photosensitive element of each image pixel when the light source is off are transferred to the second storage node of that pixel. The light source may be an oscillating light source that is configured to turn on and off multiple times during an image exposure. The generated charges may be used in flash-matting operations.