Eye-Tracking Guided Object Detection for Autonomous Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in efficiently detecting and classifying objects of interest, such as pedestrians and traffic signs, due to the computational intensity of processing multiple sensor data streams, which can lead to reduced navigation efficiency and increased computational burden.
Innovation Solution
A deep neural network trained using eye-tracking data from a wearable eye-tracking system that mimics human visual behavior by focusing on objects that receive human attention, reducing the number of region proposals and improving computational efficiency by using image data alone, rather than combining it with LIDAR or radar data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensor data streams (including LIDAR and radar) are processed for object detection, then detection accuracy is improved, but computational burden increases
Solution Approach 1:
The patent extracts and utilizes eye-tracking data from the driver as a selective attention signal to identify which regions of the sensor data stream are most important for detection. This allows the system to process only the relevant portions of the full sensor data, reducing computational burden while maintaining detection accuracy for objects that matter to the driver.
Solution Approach 2:
The system changes the parameter of data selection by using eye-gaze coordinates to dynamically adjust which regions are processed. Instead of uniformly processing all sensor data, the system modifies the processing parameters to focus computational resources on regions where the driver is looking, thereby reducing overall computational burden while preserving accuracy for critical objects.
2Measurement precision
If comprehensive sensor data processing is performed, then object detection accuracy is improved, but navigation efficiency decreases
Solution Approach 1:
The system performs preliminary action by using eye-tracking data to pre-identify regions of interest before full object detection processing. The eye-gaze information provides advance guidance on where objects are most likely to be located, allowing the system to prepare and prioritize processing of these regions, thereby improving navigation efficiency without sacrificing detection accuracy.
Solution Approach 2:
The patent applies partial action by processing only the portions of sensor data that correspond to regions where the driver is looking, rather than processing the entire data stream. This selective partial processing maintains detection accuracy for relevant objects while significantly improving navigation efficiency by avoiding unnecessary processing of irrelevant regions.
3Reliability
If all region proposals are processed for object detection, then detection completeness is improved, but computational time increases
Solution Approach 1:
The patent segments the object detection process into two stages: first, eye-tracking data is used to identify and segment out the most important regions of interest; second, full detection is applied only to these segmented regions rather than the entire image. This segmentation maintains detection completeness for critical objects while reducing computational time by excluding irrelevant regions from intensive processing.
4Measurement precision
If deep neural networks process multiple sensor streams, then object classification accuracy is improved, but onboard computational requirements increase
Solution Approach 1:
The patent introduces eye-tracking data as an intermediary that mediates between the driver's attention and the object detection system. This intermediary provides a simplified selection criterion that reduces the complexity of onboard computational requirements by filtering which regions need deep neural network processing, while still maintaining classification accuracy for the objects that matter most to the driver.
Data Source
AI summary
A training system for a deep neural network and method of training is disclosed. The system and/or method may comprise: receiving, from an eye-tracking system associated with a sensor, an image frame captured while an operator is controlling a vehicle; receiving, from the eye-tracking system, eyeball gaze data corresponding to the image frame; and iteratively training the deep neural network to determine an object of interest depicted within the image frame based on the eyeball gaze data. The deep neural network generates at least one feature map and determine a proposed region corresponding to the object of interest within the at least one feature map based on the eyeball gaze data.