Multi-View Cabin Sensing Using Reflectance to Reduce Glare
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Occupant Monitoring Systems (OMS) face challenges in accurately detecting occupant conditions due to low-light conditions, reflections, depth ambiguities, and occlusions, which degrade sensor data quality and downstream task accuracy.
Innovation Solution
Distribute optical sensors and IR illuminators throughout the cabin, positioning them to avoid direct glare and synchronize illumination patterns to extract reflectance data and reconstruct 3D geometry, using techniques like BRDF and stereo vision to generate high dynamic range images and improve detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If conventional IR illumination techniques are used to illuminate the scene, then the camera can capture images in low-light conditions, but the IR light distributes unevenly within the scene and omits some monitored areas
Solution Approach 1:
The patent divides the single IR illuminator into multiple distributed IR illuminators positioned at different locations throughout the cabin (dashboard, rearview mirror, door panels, seats). This segmentation allows each illuminator to cover specific zones, collectively providing uniform illumination across the entire monitored space including previously omitted areas like rear seats and footwells.
Solution Approach 2:
The patent combines multiple distributed IR illuminators with multiple optical sensors (RGB cameras, IR cameras, depth sensors) positioned throughout the cabin. This merging of multiple illumination sources and sensing elements creates a coordinated multi-view sensing system that achieves comprehensive and uniform coverage of the entire cabin environment.
2Device complexity
If IR illuminators are integrated directly into the camera module, then the system structure is simplified, but direct glare is introduced into the optical sensor
Solution Approach 1:
The patent separates the IR illuminators from the optical sensors, placing them at different locations throughout the cabin rather than integrating them into the same module. This spatial segmentation eliminates direct glare into the optical sensors while maintaining the ability to illuminate the scene effectively.
Solution Approach 2:
The patent introduces a distributed array of IR illuminators as intermediaries between the light source and the optical sensors. These illuminators are positioned to illuminate the scene indirectly, with light reflecting off surfaces rather than shining directly into the camera lenses, thereby eliminating glare while maintaining illumination effectiveness.
3Adaptability or versatility
If multiple optical sensors are used to capture sensor data, then the system can perform comprehensive occupant monitoring tasks, but the sensor data quality is degraded by reflections and occlusions
Solution Approach 1:
The patent transitions from single-view 2D imaging to multi-view 3D sensing by distributing optical sensors throughout the cabin at different positions and angles. This dimensional change from monocular to stereo/multocular vision enables the system to capture depth information and multiple perspectives simultaneously, resolving ambiguities caused by reflections and occlusions in any single view.
Solution Approach 2:
The patent implements a feedback mechanism where sensor data from multiple views is continuously processed to identify and compensate for reflections and occlusions. The system uses depth information and multi-perspective data to detect inconsistent regions that indicate reflections or occlusions, then adjusts or corrects the sensor data to improve overall accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The described techniques enhance sensor data accuracy and detail, reducing glare and occlusions, enabling improved occupant monitoring and detection tasks such as gaze detection, pose estimation, and child presence detection.
Implementation Method 1
IR light-emitting diodes (LEDs) or vertical-cavity surface-emitting lasers (VCSELs) are integrated directly into the camera module to illuminate the camera's field of view
Implementation Method 2
IR light-emitting diodes (LEDs) are integrated directly into the camera module
Implementation Method 3
vertical-cavity surface-emitting lasers (VCSELs) are integrated directly into the camera module
Implementation Method 4
Reflections from surfaces like windows or shiny interior materials can distort sensor readings
Implementation Method 5
extract reflectance data and/or three-dimensional (3D) surface(s) in the target region
Data Source
AI summary
Optical sensors (e.g., cameras) and (e.g., IR) illumination sources may be distributed in an environment (e.g., an interior space such as a cabin or cockpit of an ego-machine) and synchronized to generate frames of sensor data. By positioning the optical sensors and assigning them corresponding frequency ranges, the resulting sensor data (e.g., images from different perspectives and with different illumination patterns) may be used to extract reflectance data, the reflectance data may be used to generate more accurate sensor data (e.g., HDR images, images re-rendered using an extracted bidirectional reflectance distribution function), and the resulting sensor data may be used in one or more downstream tasks, such as operator or occupant monitoring or detection tasks (e.g., gaze detection, pose detection, attentiveness or fatigue assessment, facial recognition, gesture recognition, occupant presence detection, child presence detection, seat belt detection, hands-on-wheel detection, etc.), generating visualizations (e.g., video conference calls), and/or otherwise.