Multimodal Data Filtering for Autonomous Driving Scenarios
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data filtering mechanisms for autonomous driving are inefficient due to the need for separate filtering rules for different scenarios, leading to increased workload and costs, as open-source datasets are not directly applicable and are difficult to reuse across varying vehicle scenarios.
Innovation Solution
A method and system that utilize multimodal fusion, feature extraction, and encoding with a transformer neural network to select and retain perception data based on similarity matching with target encoded information for specific scenarios, such as overtaking or lane line missing, by receiving data from various signal sources like image sensors and LiDAR, and determining retention based on similarity thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate filtering rules are designed for different scenarios, then data filtering accuracy is improved, but device complexity and workload increase significantly
Solution Approach 1:
The patent creates a universal filtering mechanism using scenario encoding and similarity matching that can handle multiple different scenarios (overtaking, avoiding, accelerating, decelerating, changing lanes, emergency stop, lane line missing) with a single system. The encoder generates scenario-specific encoded information, and the similarity matching module compares incoming data against these encoded scenarios, allowing one filtering system to serve multiple scenarios without requiring separate filtering rules for each scenario type.
2Measurement precision
If separate filtering rules are designed for different scenarios, then data filtering accuracy is improved, but workload increases significantly
Solution Approach 1:
The filtering system processes all scenario types through a unified pipeline: encoding the scenario type, generating encoded information, performing similarity matching, and making retention decisions. This universal approach maintains high filtering accuracy for each scenario while reducing overall workload compared to maintaining and executing separate filtering rules for each scenario.
Solution Approach 2:
The system transforms the filtering approach by changing from rule-based filtering to encoding-based similarity matching. The scenario type is encoded into a compact representation, and filtering decisions are made by comparing similarity metrics rather than evaluating complex rules, significantly reducing computational workload while maintaining accuracy.
3Quantity of substance
If open-source datasets are used directly, then data collection costs are reduced, but data applicability to autonomous driving scenarios decreases
Solution Approach 1:
The patent introduces scenario encoded information as an intermediary between open-source datasets and autonomous driving algorithms. The encoding module creates scenario-specific representations that bridge the gap between generic open-source data and specialized autonomous driving requirements. This intermediary layer enables effective use of open-source datasets by transforming them into scenario-appropriate formats through encoding and similarity matching.
Data Source
Figure 1~2
Figure 3~4
AI summary
The invention relates to a method and system for selecting data for a predetermined scenario by filtering. The method includes: receiving a set of perception data from one or more signal sources; performing multimodal fusion on the set of perception data to generate multimodal serialized data; extracting features from the multimodal serialized data and encoding them to generate encoded information; receiving target encoded information, wherein the target encoded information is generated based on perception data for the predetermined scenario; and performing similarity matching between the encoded information and the target encoded information, and determining, based on a matching result, whether to retain the set of perception data.