Computation Graphs for Non-Monotonic STL Test Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems for autonomous vehicles rely on computationally expensive algorithms that can only handle monotonic Signal Temporal Logic (STL) formulas, limiting their ability to effectively generate diverse driving scenarios and certify safety in complex environments.

Innovation Solution

The development of a method to translate STL formulas into computation graphs, allowing for the efficient generation of parametric STL formulas that can be solved using machine learning frameworks, enabling the creation of robustness traces and automatic test case generation for autonomous vehicles.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If computationally expensive algorithms are used to handle monotonic STL formulas, then the system can maintain simplicity in the algorithm design, but the system cannot handle non-monotonic STL formulas and generates limited driving scenarios

Engineering Contradiction:

Improveability to handle non-monotonic STL formulasVSAvoidcomplexity of algorithm

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a computation graph as an intermediary representation that bridges STL formulas and machine learning frameworks. The computation graph translates temporal logic formulas into a differentiable computational structure, enabling the use of standard ML optimization tools to handle both monotonic and non-monotonic STL formulas uniformly, thus resolving the contradiction between handling complex formulas and maintaining algorithm simplicity

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent transforms the handling of STL formulas by changing the parameter representation from traditional monotonic constraints to parametric forms that can capture non-monotonic behavior. By representing formulas with parameters that can be optimized continuously, the system can handle a broader class of STL formulas including non-monotonic ones, expanding adaptability without requiring fundamentally new algorithmic approaches

Inventive Principle:

Principle #35Parameter changes

2Productivity

If computationally expensive algorithms are used, then the system can ensure accurate STL formula evaluation, but the system becomes inefficient for generating diverse driving scenarios

Engineering Contradiction:

Improveefficiency of test case generationVSAvoidaccuracy of STL formula evaluation

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent replaces traditional mechanical/computational algorithms with a machine learning-based approach. By substituting conventional algorithmic evaluation with neural network-based inference trained on temporal logic patterns, the system achieves both high accuracy in STL formula evaluation and improved productivity in generating diverse driving scenarios, as the ML models can efficiently handle complex temporal reasoning tasks

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent uses computation graphs that can be copied and reused across different STL formula evaluations. Once a computation graph is built for a given formula structure, it can be efficiently instantiated multiple times for different scenarios, avoiding the need to recompute from scratch and thus improving generation efficiency while maintaining evaluation accuracy through the preserved graph structure

Inventive Principle:

Principle #26Copying

3Adaptability or versatility

If monotonic STL formulas are used, then the algorithm design remains simple, but the system cannot effectively generate diverse driving scenarios with varying difficulty levels

Engineering Contradiction:

Improvediversity of driving scenariosVSAvoidcomplexity of formula handling

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent creates a universal computation graph framework that can handle multiple types of STL formulas (monotonic and non-monotonic) within a single unified structure. This multi-functional approach allows the same computational infrastructure to generate diverse driving scenarios with varying difficulty levels by simply changing the formula parameters, without requiring separate handling mechanisms for different formula types, thus achieving diversity without proportional increase in complexity

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11829275B2Systems and methods for automatic test generation

Publication Date: 2023.11.28 TOYOTA JIDOSHA KK

AI summary

Systems and methods for generating and evaluating driving scenarios with varying difficulty levels is provided. The disclosed systems and methods may be used to develop a suite of regression tests that track the progress of an autonomous driving stack. A robustness trace of a temporal logic formula may be computed from an always-eventually fragment using a computation graph. The robustness trace may be approximated by a smoothly differentiable computation graph, which can be implemented in existing machine learning programming frameworks. The systems and methods provided herein may be useful in automatic test case generation for autonomous or semi-autonomous vehicles.