Sensor Fidelity Scoring for Realistic AV Simulation Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Autonomous vehicle (AV) training and testing in simulated environments face challenges in accurately recreating real-world scenarios due to divergences between real-world and simulated sensor data, primarily caused by sensor noise and errors in asset material and sensor modeling.

Innovation Solution

A method for quantifying sensor fidelity using a fidelity score, which evaluates the realism of simulated sensor data by comparing it to real-world data through machine-learning models, and identifies contributing sensor parameters through perturbation algorithms, enabling the assessment and improvement of simulated environments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If simulated environments are used for AV training and testing, then productivity and data collection efficiency are improved, but measurement precision and reliability of sensor data deteriorate due to divergences from real-world scenarios

Engineering Contradiction:

Improvetraining and testing efficiencyVSAvoidsensor data fidelity

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent implements a feedback mechanism by training a machine learning model to distinguish between real and simulated sensor data, then using this model to calculate fidelity scores that feed back into the simulation system to identify and correct divergences from real-world scenarios

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent replaces direct physical measurement of sensor fidelity with a computational approach, using machine learning models and fidelity score calculations to assess and improve simulation realism without requiring continuous real-world physical testing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Device complexity

If sensor modeling is simplified to improve ease of manufacture and operation, then device complexity is reduced, but measurement precision and reliability of sensor data deteriorate due to increased noise and errors

Engineering Contradiction:

Improvesensor modeling complexityVSAvoidsensor data fidelity

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent changes the parameter being measured from raw sensor data characteristics to a derived fidelity score that quantifies the difference between real and simulated data, allowing assessment of modeling accuracy without increasing modeling complexity

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20240166222A1Measuring simulation realism

Publication Date: 2024.05.23 GM CRUISE HOLDINGS LLC

AI summary

The disclosed technology provides solutions for measuring sensor realism (or fidelity) and in particular, provides methods for quantitatively evaluating the fidelity of sensor data collected using a simulated (virtual) environment. In some aspects, a process of the disclosed technology includes steps for receiving simulation training data, providing the simulation training data to an encoder neural-network to generate a plurality of simulation embeddings, and generating a first cluster based on the plurality of simulation embeddings. The process can further include steps for receiving real-world training data, providing the real-world training data to the encoder neural-network to generate a plurality of real-world embeddings, and generating a second cluster based on the plurality of real-world embeddings. Systems and machine-readable media are also provided.