ROLO Engine Sensor Fusion for ADAS Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current advanced driver assistance systems (ADAS) face limitations in providing accurate and comprehensive road situation analysis due to the limitations of individual sensors such as image sensors, lidar, and radar, which often result in incomplete, robust, and inaccurate data, necessitating the need for improved sensor fusion and object detection techniques.
Innovation Solution
A vision-centric deep-learning-based method and system utilizing a Recurrent You Only Look Once (ROLO) engine with convolutional neural networks (CNN) and long short-term memory units (LSTMs) for real-time traffic environment analysis, integrating camera inputs with lidar and infrared sensors to detect and track objects, predict future object status, and determine timely navigation warnings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image sensors are used for object detection, then discrimination ability is improved, but depth sensing ability deteriorates
Solution Approach 1:
The patent combines multiple sensor types (image sensors, lidar, radar, ultrasonic sensors) into a unified sensor fusion system. This merging allows the system to leverage the high discrimination ability of image sensors while compensating for their poor depth sensing with lidar and radar data, thereby resolving the contradiction between discrimination and depth perception.
Solution Approach 2:
The patent introduces deep learning-based object detection and tracking algorithms as intermediaries that process and integrate data from multiple sensor sources. These algorithms act as mediators that fuse information from image sensors, lidar, and radar to produce comprehensive object detection results that overcome the limitations of individual sensors.
2Length of stationary object
If radar is used for detection, then detection range is improved, but lateral spatial information deteriorates
Solution Approach 1:
The patent merges radar data with image sensor and lidar data in a sensor fusion framework. This combination allows the system to maintain radar's long detection range while supplementing it with the rich lateral spatial information provided by image sensors and lidar, thereby resolving the contradiction between range and spatial detail.
3Area of stationary object
If lidar is used for detection, then field of view is improved, but discrimination ability deteriorates
Solution Approach 1:
The patent combines lidar data with image sensor data in a sensor fusion system. This merging allows the system to leverage lidar's wide field of view for comprehensive scene coverage while using image sensors to provide high discrimination ability for detailed object recognition and classification.
4Measurement precision
If multiple sensor types are used, then detection accuracy is improved, but system complexity deteriorates
Solution Approach 1:
The patent implements a universal deep learning-based object detection and tracking system that processes data from multiple sensor types through a unified algorithmic framework. This multi-functional approach allows the same core algorithms to handle data from image sensors, lidar, radar, and ultrasonic sensors, thereby improving detection accuracy while managing system complexity through algorithmic universality.
Data Source
AI summary
In accordance with various embodiments of the disclosed subject matter, a method and a system for vision-centric deep-learning-based road situation analysis are provided. The method can include: receiving real-time traffic environment visual input from a camera; determining, using a ROLO engine, at least one initial region of interest from the real-time traffic environment visual input by using a CNN training method; verifying the at least one initial region of interest to determine if a detected object in the at least one initial region of interest is a candidate object to be tracked; using LSTMs to track the detected object based on the real-time traffic environment visual input, and predicting a future status of the detected object by using the CNN training method; and determining if a warning signal is to be presented to a driver of a vehicle based on the predicted future status of the detected object.