Autonomous Vehicle Test Run Retrieval Using Saliency Events
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for testing autonomous vehicle systems struggle to efficiently identify salient scenarios, as determining the saliency of test runs in real or simulated environments is challenging due to the large volume of data and the difficulty in manually reviewing run data to determine its significance, which is compounded by the fact that salient parts may be a small fraction of the total run.
Innovation Solution
A computer system and method for identifying salient test runs by generating time-indexed events and decision indicators based on predefined driving action assessment rules, allowing for the automatic retrieval and visualization of scenarios that meet predefined safety and performance criteria, thereby facilitating the identification of salient scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review of run data is performed to determine saliency, then measurement precision of scenario significance is improved, but loss of time and productivity deteriorate due to large volume of data
Solution Approach 1:
The patent replaces manual mechanical review of run data with an automated computer-implemented system that processes trajectory data, detects events, and generates saliency scores automatically. This substitution eliminates the time-consuming manual review process while maintaining or improving the accuracy of saliency determination through systematic algorithmic analysis of trajectory deviations and event detection.
Solution Approach 2:
The system enables self-service by automatically analyzing run data without human intervention. The computer system autonomously processes trajectory data, detects events based on predefined criteria, calculates saliency scores, and identifies salient scenarios, allowing the testing process to serve itself rather than requiring manual review of each dataset.
2Reliability
If comprehensive testing of all run data is performed, then reliability of safety assessment is improved, but productivity deteriorates due to processing large volumes of data
Solution Approach 1:
The patent extracts only the most relevant and salient portions of run data for detailed analysis. By calculating saliency scores and identifying scenarios with highest significance, the system extracts critical safety-related events from the comprehensive dataset, maintaining reliable safety assessment while reducing the volume of data requiring full processing and review.
Solution Approach 2:
The system applies local quality by focusing detailed analysis resources on specific high-salency portions of the data rather than uniform processing. Salient scenarios identified through automated scoring receive prioritized attention, allowing reliable safety assessment of critical events while improving overall testing productivity through selective deep analysis.
3Measurement precision
If detailed analysis of all trajectory data is performed, then measurement precision of performance assessment is improved, but use of energy and computational resources worsens
Solution Approach 1:
The patent applies partial action by performing detailed analysis only on salient scenarios rather than all trajectory data. The system calculates saliency scores to identify which portions of the data warrant detailed examination, applying computational resources selectively to high-value cases. This maintains measurement precision for critical assessments while reducing overall energy and computational resource consumption.
Data Source
AI summary
The disclosure provides systems and methods for identifying salient test runs involving an autonomous vehicle system. A processor receives sets of run data, each set representative of a driving scenario. For each set, an output set is generated, the output set comprising time-indexed events generated in response to a detected behaviour of at least one challenger agent, and a sequence of decision indicators indicating whether a driving action by an ego agent would be permissible. A data retrieval component is coupled to a results database and retrieves output sets based on the time-indexed events and the sequence of decision indicators. The processor generates the sequence of decision indicators by generating a planned trajectory of the ego agent, and determining whether the predefined driving action by the ego agent would be permissible.