In-Cabin Sensor Calibration Using a Shared 3D Coordinate Frame
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-cabin monitoring systems face challenges in correlating optical sensor data and depth-perception sensor data due to a lack of techniques for calibrating these sensors with respect to their extrinsic parameters, particularly for interior applications where depth-perception sensors are used inwardly, leading to inconsistencies in feature detection and classification.
Innovation Solution
A framework is established to create a shared three-dimensional intermediary coordinate system within a vehicle interior using hybrid calibration targets, allowing for the calibration of depth sensors and optical sensors by determining extrinsic calibration parameters, enabling the correlation of features detected by both sensor types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If optical sensors are used for line-of-sight sensing in in-cabin monitoring, then motion detection capability is improved, but field of view is obstructed by intervening objects
Solution Approach 1:
The patent combines optical sensors (cameras) with depth-perception sensors (RADAR, LiDAR, ultrasonic sensors) into a unified in-cabin monitoring system. This merging allows the system to leverage both the high-resolution motion detection of optical sensors and the penetrating capability of depth sensors that can see through intervening objects like blankets and car seats, thereby resolving the field of view obstruction problem while maintaining motion detection accuracy.
Solution Approach 2:
The patent creates a multi-functional monitoring system that uses multiple sensor types (optical, depth-perception) to perform various monitoring tasks including gaze direction identification, attentiveness monitoring, and safety event detection. This universal approach allows the system to adapt to different monitoring needs and overcome the limitations of any single sensor type.
2Device complexity
If monocular camera systems are used for optical sensing, then system complexity is reduced, but depth information precision is insufficient
Solution Approach 1:
The patent combines monocular camera systems with depth-perception sensors to create a hybrid system. The optical sensor provides wide-field imaging with low complexity, while the depth sensor (RADAR, LiDAR, or ultrasonic) provides precise depth measurements. By merging these complementary technologies, the system achieves accurate depth information without the complexity of stereoscopic vision systems.
Solution Approach 2:
The patent introduces a shared three-dimensional intermediary coordinate system as a mediator to integrate data from optical sensors and depth-perception sensors. This coordinate system serves as a common reference frame that allows features detected by either sensor type to be correlated and combined, enabling precise depth information extraction while maintaining the simplicity of monocular cameras.
3Difficulty of detecting and measuring
If depth-perception sensors are used for in-cabin monitoring, then ability to detect through intervening objects is improved, but difficulty in correlating sensor data increases
Solution Approach 1:
The patent introduces a shared three-dimensional intermediary coordinate system as a mediator to integrate data from optical sensors and depth-perception sensors. This coordinate system serves as a common reference frame that allows features detected by either sensor type to be correlated and combined, enabling precise depth information extraction while maintaining the simplicity of monocular cameras.
Solution Approach 2:
The patent transforms sensor data from different coordinate systems into a unified three-dimensional intermediary coordinate system by determining extrinsic calibration parameters (rotation and translation). This parameter transformation allows coherent integration of optical and depth sensor data, reducing the complexity of data correlation while maintaining the ability to detect through intervening objects.
4Measurement precision
If extrinsic calibration parameters are determined for depth and optical sensors, then feature correlation accuracy is improved, but calibration process complexity increases
Solution Approach 1:
The patent performs extrinsic calibration of sensors beforehand to establish a shared three-dimensional intermediary coordinate system. By determining the rotation and translation parameters in advance, the system creates a pre-configured reference frame that simplifies real-time feature correlation. This preliminary calibration action improves measurement precision while the calibration complexity is confined to the setup phase rather than ongoing operation.
Data Source
AI summary
In various examples, calibration techniques for interior depth sensors and image sensors for in-cabin monitoring systems and applications are provided. An intermediary coordinate system may be generated using calibration targets distributed within an interior space to reference 3D positions of features detected by both depth-perception and optical image sensors. Rotation-translation transforms may be determined to compute a first transform (H1) between the depth-perception sensor's 3D coordinate system and the 3D intermediary coordinate system, and a second transform (H2) between the optical image sensor's 2D coordinate system and the intermediary coordinate system. A third transform (H3) between the depth-perception sensor's 3D coordinate system and the optical image sensor's 2D coordinate system can be computed as a function of H1 and H2. The calibration targets may comprise a structural substrate that includes one or more fiducial point markers and one or more motion targets.