Vehicle Image Sensor Localization via Driver Gaze Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera-based systems in vehicles face challenges in accurately localizing the image sensor relative to a vehicle coordinate system, especially in vehicles with mechanical steering wheel adjustments, where precise positional data is not available, leading to unreliable spatial data determination.
Innovation Solution
A method utilizing an environment sensor to detect changes in the driver's viewing direction and correlate them with external objects in the vehicle's surroundings, allowing for precise calibration of the image sensor's location based on spatial information from these objects, thereby enhancing localization accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the image sensor is mounted on the steering wheel column to achieve good driver monitoring, then the camera can be adjusted with the steering wheel position, but the camera position becomes variable and cannot be precisely localized in the vehicle coordinate system
Solution Approach 1:
The patent introduces a calibration object with known position and a processing unit as intermediaries. The calibration object serves as a reference mediator between the variable camera position and the vehicle coordinate system, enabling precise localization through image analysis and coordinate transformation calculations.
Solution Approach 2:
The system uses feedback from the calibration object detection to determine and store the camera's position and orientation. This feedback mechanism allows the system to adapt to steering wheel adjustments and maintain accurate localization by continuously updating camera parameters based on calibration object observations.
2Measurement precision
If electronic monitoring of steering wheel adjustments is implemented to track camera position, then camera position can be calculated from reference position, but such information is not available in many cars
Solution Approach 1:
The system performs self-calibration by autonomously detecting the calibration object and calculating its own position and orientation parameters. The processing unit executes coordinate transformations and determines camera localization without requiring external vehicle system data, making the solution universally applicable.
Solution Approach 2:
The patent replaces the electronic monitoring approach with an optical-mechanical calibration method. Instead of relying on electronic steering angle sensors and CAN bus data, the system uses image capture and geometric calculations based on the calibration object to determine camera position, eliminating dependency on vehicle-specific electronic systems.
3Device complexity
If the camera position is not known, then spatial data can only be determined relative to the camera, but the viewing direction cannot be precisely matched with elements inside or outside the car
Solution Approach 1:
The patent transforms the problem from 2D image coordinates to 3D vehicle coordinate system by introducing depth information through the calibration object's known spatial position. The processing unit performs coordinate transformation that maps camera-relative coordinates to vehicle-relative coordinates, enabling precise spatial matching in three-dimensional space.
Solution Approach 2:
The calibration object serves as a spatial mediator that connects the camera coordinate system with the vehicle coordinate system. By detecting the calibration object's known position and orientation, the system establishes a transformation relationship that enables accurate mapping between different coordinate systems.
Data Source
Figure 1
Figure 2
Figure 3a~3d
AI summary
A method for localizing an image sensor mounted on a vehicle comprises: detecting, by means of the image sensor, at least one change of a passenger's viewing direction from a first viewing direction to a second viewing direction; detecting, by means of an environment sensor, at least one external object in an external environment of the vehicle and determining whether the at least one detected change in the passenger's viewing direction and the at least one detected external object correlate in time. The method further comprises for each detected external object for which a time correlation with a change in the passenger's viewing direction is identified: retrieving a spatial information of the external object and localizing the image sensor on the basis of the retrieved spatial information and the second viewing direction corresponding to the respective correlated change in the passenger's viewing direction.