In-Vehicle Object Identification Using Gaze and Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Occupants in moving vehicles face difficulties in accurately identifying objects of interest due to vehicle movement and obstructing objects, and they often need to use external devices like smartphones, which can be distracting and inconvenient.
Innovation Solution
The system utilizes gaze data combined with map data and vehicle sensor data to identify objects of interest within the vehicle's surroundings, allowing occupants to request information about the objects without needing to use external devices, thus reducing distractions and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If occupants use mobile devices to capture images for object identification, then object identification capability is improved, but driving safety deteriorates due to distraction
Solution Approach 1:
The system enables self-service by automatically capturing images using the vehicle's existing camera infrastructure and autonomously processing object identification through integrated algorithms, eliminating the need for driver intervention with mobile devices while maintaining identification capability
Solution Approach 2:
The system merges the vehicle's sensor systems (cameras, GPS, map data) with object identification functionality into a unified system, allowing the vehicle itself to perform tasks previously requiring external mobile devices, thereby improving safety by keeping the driver's attention on the road
2Loss of information
If occupants manually capture images using external devices, then object information can be obtained, but operation convenience deteriorates due to handling difficulties during vehicle movement
Solution Approach 1:
The system performs automatic image capture and object identification without requiring manual operation by occupants, using the vehicle's integrated sensors and processing systems to obtain object information autonomously
Solution Approach 2:
The system captures images and processes object identification in advance before the occupant needs the information, so that when the occupant requests object information, the system has already prepared the data, eliminating the need for manual capture operations
3Measurement precision
If the system continuously monitors and processes environmental data for object identification, then object identification accuracy is improved, but energy consumption increases
Solution Approach 1:
The system uses periodic action by triggering comprehensive environmental monitoring and processing only when specific conditions are met, such as when the vehicle is stationary or when an occupant indicates interest in an object, rather than continuously monitoring all the time
Solution Approach 2:
The system performs preliminary actions by pre-processing and storing environmental data when conditions are favorable (low motion, good visibility), so that when object identification is needed, the processed data is already available, reducing the need for intensive real-time processing
Data Source
AI summary
A method of obtaining object information includes receiving a request initiation from an occupant of a vehicle. The method includes receiving a request from the occupant. The method includes determining a content of the request from the occupant. The method further includes detecting a gaze location of the occupant. The method includes receiving information related to an environment surrounding the vehicle based on data collected by a sensor attached to the vehicle. The method includes identifying a region of interest (ROI) outside of the vehicle based on the detected gaze location and the information related to the environment surrounding the vehicle. The method includes generating log data based on the ROI and the content of the request. The method includes transmitting the log data to an external device. The method includes receiving information related to an object within the ROI, wherein the information satisfies the content of the request.