Multi-Modal Driving Event Classification With Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for classifying driving events in vehicles consume excessive computing and networking resources due to inaccurate event classification, leading to improper driver feedback and false alarms, and fail to efficiently identify safety-critical events from a vast amount of non-critical data.
Innovation Solution
A multi-modal neural architecture utilizing object detection and tracking models, combined with spatiotemporal neural networks, processes video and sensor data to accurately classify driving events into categories like normal, near-crash, or crash events, optimizing resource usage and enhancing real-time analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current techniques for classifying driving events are used, then computing and networking resources are consumed, but the classification accuracy is poor leading to false alarms and improper driver feedback
Solution Approach 1:
The system segments the classification process into multiple stages: initial filtering of non-critical events, intermediate analysis of potentially critical events, and detailed examination of safety-critical events. This hierarchical segmentation allows the system to apply different levels of computational resources appropriately, improving classification accuracy for critical events while conserving resources on routine events.
Solution Approach 2:
The patent introduces intermediary processing layers between raw data collection and final classification, including feature extraction modules and preliminary classification filters. These intermediaries preprocess data to identify patterns and reduce the volume of data requiring intensive computation, thereby improving overall classification accuracy while reducing resource consumption.
2Measurement precision
If comprehensive video and sensor data are analyzed to identify safety-critical events, then measurement precision improves, but the quantity of data to be processed increases significantly
Solution Approach 1:
The system extracts only the most relevant features and data elements from comprehensive video and sensor streams, rather than processing all raw data. By identifying and extracting key indicators of safety-critical events, the system maintains high detection accuracy while dramatically reducing the volume of data that requires intensive processing.
Solution Approach 2:
The patent applies partial processing to the full dataset by initially analyzing only a subset of data streams or features that are most indicative of critical events. This selective partial action allows the system to achieve sufficient detection accuracy without the computational burden of processing every detail of the comprehensive data set.
Data Source
AI summary
A device may receive video data associated with a vehicle experiencing an event, and may utilize an object detection model and an object tracking model to determine object data identifying bounding boxes, tracks, and classes for objects depicted in the video data. The device may process the object data, with an object backbone of a spatiotemporal multi-modal (ST-MM) neural network model, to determine object features associated with dynamics of the objects depicted in the video data, and may determine vehicle features associated with dynamics of the vehicle. The device may process the object features and the vehicle features, with a recurrent neural network of the ST-MM neural network model, to classify the event into a category, and may perform one or more actions based on the category of the event.