Driver Awareness Heat Maps from Noisy Gaze and Scene Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Estimating a person's awareness of their environment from noisy gaze measurements is challenging, particularly in applications like automated driver assistance systems, where accurate estimation of attended awareness is crucial but has been elusive due to noise and calibration errors in sensor systems.
Innovation Solution
A system comprising a camera, a monitoring system, and a computing device that implements a convolutional encoder-decoder neural network to process image data and inject gaze sequences, generating a gaze probability density heat map and an attended awareness heat map, which refines noisy gaze signals using visual saliency to improve awareness estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gaze sequences are used to estimate attended awareness, then awareness estimation is achieved, but measurement precision deteriorates due to noise and calibration errors
Solution Approach 1:
The patent introduces an intermediary computational model (convolutional encoder-decoder neural network) that processes noisy gaze sequences and image data to produce a refined attended awareness heat map. This intermediary model acts as a mediator between the unreliable gaze measurements and the final awareness estimation, filtering out noise and calibration errors while preserving the essential attention information.
Solution Approach 2:
The system implements feedback by using the computational model to generate attended awareness heat maps that can be fed back into the driver assistance system. This feedback loop allows the system to continuously refine its understanding of driver attention based on processed gaze data, improving measurement precision over time through iterative refinement.
2Measurement precision
If computational models are implemented to process gaze data, then attended awareness estimation is improved, but device complexity increases
Solution Approach 1:
The computational model is segmented into distinct functional components: an encoder that processes image data and extracts visual features, a decoder that processes gaze sequences and reconstructs attention patterns, and a integration mechanism that combines these to generate the attended awareness heat map. This segmentation makes the complex system more manageable and implementable.
Solution Approach 2:
The convolutional encoder-decoder neural network serves multiple functions: it processes both image data and gaze sequences, performs noise filtering, generates probability density heat maps, and produces final attended awareness estimates. This multi-functionality reduces the need for separate specialized components, managing device complexity while maintaining measurement precision.
Data Source
AI summary
A system includes a camera configured to capture image data of an environment, a monitoring system configured to generate a gaze sequences of a subject, and a computing device communicatively coupled to the camera and the monitoring system. The computing device is configured to receive the image data from the camera and the gaze sequences from the monitoring system, implement a machine learning model comprising a convolutional encoder-decoder neural network configured to process the image data and a side-channel configured to inject the gaze sequences into a decoder stage of the convolutional encoder-decoder neural network, generate, with the machine learning model, a gaze probability density heat map, and generate, with the machine learning model, an attended awareness heat map.