Event Camera Driving Assist for Low-Latency Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional vehicle cameras for driving assist systems face limitations in dynamic range, motion blur, and latency, particularly in high dynamic conditions, which affect object detection and driver monitoring accuracy.
Innovation Solution
Employing event cameras with high temporal resolution, dynamic range, and asynchronous event-based image capture, integrated with neuromorphic computing architectures, to process data from multiple sensors for enhanced object detection and driver monitoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional cameras are used for driving assist systems, then the system can capture images of objects and scenes, but the dynamic range is limited and motion blur occurs in high dynamic conditions
Solution Approach 1:
The patent changes the fundamental operating parameters of the camera system by using event cameras that operate asynchronously and report only changes in brightness. This allows the system to handle extreme dynamic ranges (at least 100 dB) by adjusting exposure and gain parameters dynamically for each pixel independently, rather than using fixed exposure settings across the entire sensor array.
Solution Approach 2:
The system transitions from static frame-based capture to dynamic event-based capture where each pixel independently responds to brightness changes in real-time. This dynamic approach allows the camera to adapt to rapidly changing lighting conditions and capture motion without blur, as events are recorded asynchronously based on actual changes rather than fixed time intervals.
2Measurement precision
If conventional cameras are used for driver monitoring, then the system can track driver position, but latency is high and eye tracking precision is reduced
Solution Approach 1:
The patent replaces the traditional mechanical/frame-based image capture system with an event-based sensing system that directly detects and reports brightness changes. This substitution eliminates the need for continuous frame capture and processing, reducing latency to less than 10,000 microseconds while improving measurement precision for eye tracking applications through direct neural network processing of event streams.
Solution Approach 2:
The system performs preliminary processing by having each pixel independently detect and report brightness changes as they occur, rather than waiting for frame-based processing. This preliminary action at the sensor level enables real-time eye tracking with minimal latency, as the data is prepared and transmitted asynchronously as events rather than being processed in batches after frame capture.
3Productivity
If event cameras with high temporal resolution are used, then motion blur is reduced and object detection is improved, but the system complexity increases with neuromorphic computing architecture
Solution Approach 1:
The patent segments the image processing task by having each pixel in the event camera operate independently to detect brightness changes. This segmentation allows parallel processing of visual information across the sensor array, improving object detection speed while managing system complexity through distributed rather than centralized processing. The neural network processes segmented event streams from different regions of the scene simultaneously.
Solution Approach 2:
The patent introduces an event camera as an intermediary between the physical scene and the neural network processing system. This intermediary converts complex visual information into simplified event streams that contain only relevant changes, reducing the complexity of data that needs to be processed by the neural network while maintaining high temporal resolution for fast object detection.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Improves object detection accuracy and reduces motion blur, enabling low-latency eye tracking and driver monitoring under dynamic lighting conditions, enhancing vehicle control and safety.
Implementation Method 1
The event camera outputs data responsive to changes in brightness at respective pixels of the photosensing array
Implementation Method 2
a near-infrared light emitter disposed behind the reflective element and emitting light at a wavelength of about 940 nanometers
Data Source
AI summary
A vehicular driving assist system includes an event camera disposed at an in-cabin side of a windshield of a vehicle and viewing forward of the vehicle through the vehicle windshield. Data output by the event camera is provided to an electronic control unit (ECU). The vehicular driving assist system, responsive to processing of data output by the event camera, detects at least one object present exterior of the vehicle. Vehicle kinematic data is provided to the ECU as the vehicle is driven along a road. The vehicle kinematic data includes speed data and (i) steering angle data, (ii) yaw data, (iii) pitch data and/or (iv) roll data. Responsive at least in part to processing of data output by the event camera and to processing at the ECU of vehicle kinematic data, (i) steering of the equipped vehicle is controlled and/or (ii) braking of the equipped vehicle is controlled.