In-Pixel Background Subtraction for Foreground Motion Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image sensors face challenges in effectively distinguishing objects in the foreground from the background and detecting motion in external scenes, especially under varying lighting conditions, which limits their application in scenarios like driver monitoring and augmented/virtual reality.
Innovation Solution
A voltage domain global shutter image sensor system that uses an infrared illumination source to capture two sequential images, one without and one with illumination, allowing for subtraction to highlight foreground objects and detect motion, utilizing a CMOS image sensor with sample and hold circuits and full column differential amplifiers to process the images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image sensors are used to capture images under varying lighting conditions, then the images contain both background and foreground information, but it becomes difficult to distinguish foreground objects from background and detect motion
Solution Approach 1:
The patent divides the imaging process into two separate capture phases: one with infrared illumination and one without. This segmentation allows the system to separate foreground objects (which reflect infrared light) from the background (which does not), thereby improving foreground detection accuracy while eliminating background interference.
Solution Approach 2:
The patent introduces an infrared illumination source as an intermediary element that selectively illuminates foreground objects. This intermediary enables the differentiation between foreground and background by creating a contrast based on infrared reflectivity, solving the problem of distinguishing objects under varying lighting conditions.
2Measurement precision
If infrared illumination is used to highlight foreground objects, then foreground detection is improved, but the system complexity increases due to additional illumination source and sequential imaging requirements
Solution Approach 1:
The patent makes the image sensor system multi-functional by enabling it to perform both standard visible light imaging and infrared-based foreground detection using the same sensor array. The sensor can operate in different modes (with or without infrared illumination) without requiring separate dedicated sensors, thereby reducing overall system complexity while maintaining high detection accuracy.
Solution Approach 2:
The patent employs periodic alternating illumination where the infrared illumination source is turned on and off in sequence with the image capture. This periodic action allows the system to capture both illuminated and unilluminated frames alternately, enabling foreground detection through differential processing without requiring continuous infrared illumination, thus managing power consumption and system complexity.
3Measurement precision
If sequential images are captured with and without infrared illumination, then motion detection capability is enhanced, but the time required to process and compare images increases
Solution Approach 1:
The patent performs preliminary processing by capturing and storing both illuminated and unilluminated reference frames before the actual motion detection is needed. These pre-captured frames are held in memory and can be quickly compared against new incoming frames, reducing the real-time processing time required for motion detection while maintaining high accuracy.
Solution Approach 2:
The patent replaces complex mechanical or hardware-based motion detection mechanisms with software-based differential image processing. By using computational algorithms to subtract and compare image frames, the system achieves accurate motion detection without requiring additional mechanical sensors or complex hardware circuits, thereby reducing processing time and system complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the ability to identify objects and detect motion in the foreground by distinguishing between illuminated and unilluminated images, providing clear and continuous monitoring without distracting the user, suitable for applications like vehicle cameras and AR/VR systems.
Implementation Method 1
an infrared illumination source 120 configured to illuminate the external scene
Implementation Method 2
pixels having photosensitive elements (e.g., photodiodes) that absorb a portion of the incident image light and generate image charge upon absorption of the image light
Data Source
AI summary
An imaging system includes a pixel array configured to generate image charge voltage signals in response to incident light received from an external scene. An infrared illumination source is deactivated during the capture of a first image of the external scene and activated during the capture of a second image of the external scene. An array of sample and hold circuits is coupled to the pixel array. Each sample and hold circuit is coupled to a respective pixel of the pixel array and includes first and second capacitors to store first and second image charge voltage signals of the captured first and second images, respectively. A column voltage domain differential amplifier is coupled to the first and second capacitors to determine a difference between the first and second image charge voltage signals to identify an object in a foreground of the external scene.