RGB-IR Sensor Timing Mechanism for Non-Contaminated Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D sensing systems using a single RGB-IR sensor face contamination issues due to structured light patterns affecting RGB images, necessitating a mechanism to derive a non-contaminated video stream.
Innovation Solution
A timing mechanism is implemented using an RGB-IR image sensor and a structured light projector, controlled by a circuit to capture sequences of images with and without structured light patterns, allowing for separate processing of RGB and IR data to prevent contamination and enable features like depth information generation and facial recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single RGB-IR sensor is used to capture both RGB and IR images, then system complexity and cost are reduced, but the RGB image becomes contaminated by the IR structured light pattern
Solution Approach 1:
The patent implements periodic switching between capturing images with structured light and without structured light. The control circuit alternates the structured light projector on/off and synchronizes the RGB-IR sensor exposure accordingly, creating a periodic capture sequence that separates contaminated and non-contaminated frames for different processing purposes.
Solution Approach 2:
The patent segments the image capture process into distinct phases: one phase captures images with structured light for depth information, another phase captures images without structured light for clean RGB video. This segmentation allows each capture phase to serve its specific function without interference.
2Measurement precision
If the structured light projector is turned on continuously to provide depth information, then depth sensing capability is improved, but the RGB video stream becomes contaminated
Solution Approach 1:
The structured light projector operates periodically rather than continuously. During specific time intervals, it projects structured light patterns for depth measurement. During other intervals, it remains off to allow clean RGB image capture. This periodic operation ensures both depth information and clean video streams are obtained.
Solution Approach 2:
The control circuit pre-coordinates the structured light projector activation with the sensor exposure timing. Before capturing RGB video frames, the system ensures the structured light is turned off. Before capturing depth information, the system activates the structured light projector, ensuring each capture phase has the appropriate lighting conditions.
3Illumination intensity
If the shutter exposure time is extended to capture sufficient light, then image quality is improved, but the contamination from structured light pattern increases
Solution Approach 1:
The system uses periodic switching of the structured light projector during the exposure sequence. By controlling when the projector is on or off during the capture sequence, the system can extend total exposure time for sufficient light capture while ensuring that RGB video frames are captured during off-periods when no contamination occurs.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for the generation of non-contaminated video streams, enhancing the accuracy of depth information and facial recognition while reducing system complexity and cost, and enabling advanced security features like liveness determination and access control.
Implementation Method 1
the structured light projector comprises an array of vertical-cavity surface-emitting lasers. In some embodiments where the structured light projector comprises an array of vertical-cavity surface-emitting lasers, the array of vertical-cavity surface-emitting lasers emits light having a wavelength in an 800 to 1000 nanometers range
Implementation Method 2
the structured light projector further comprises a lens configured to decompose a laser light pattern from the array of vertical-cavity surface-emitting lasers to a dense dot pattern array
Implementation Method 3
Conventional 3D sensing systems include two image sensors in order to acquire an infrared (IR) structured light pattern and an RGB image for liveness detection
Data Source
AI summary
An apparatus includes an interface circuit and a control circuit. The interface circuit may be configured to receive pixel data corresponding to a field of view of a camera. The control circuit may be configured to process the pixel data arranged as video frames and control an exposure time for capturing the pixel data and a turn on time of a structured light pattern to obtain a sequence of images comprising at least one image including the structured light pattern and at least one image where the structured light pattern is absent. The control circuit may be further configured to perform a depth analysis to generate depth information using the at least one image including the structured light pattern. The control circuit may store and execute an artificial neural network trained to (i) discern whether a face is at least one of a real face and a fake face, and (ii) make a liveness determination utilizing the depth information.