Autonomous Vehicle Safety Architecture with Runtime Invariant Monitors

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current software-safety engineering techniques are insufficient to ensure the safety of autonomous software in self-driving vehicles, particularly for advanced autonomy functions like machine learning, which cannot be validated using traditional testing methods, and runtime invariant monitors have been mainly deployed in remote-controlled or teleoperated unmanned vehicles, not effectively addressing safety risks in autonomous functions such as planning and control.

Innovation Solution

A safety architecture for autonomous vehicles that allows autonomy components to fail arbitrarily while higher-integrity 'safety gate' components uphold safety requirements, using a multi-channel approach with primary and secondary 'doer/checker' pairs to ensure continued operation despite component failures, and incorporating a reusable architectural pattern for mapping, planning, and executing safe trajectories.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional software testing methods are used to validate autonomy software, then validation process is simple and familiar, but they cannot adequately validate advanced autonomy functions like machine learning

Engineering Contradiction:
Improvevalidation effectivenessVSAvoidvalidation complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent introduces runtime invariant monitors as intermediary components that bridge the gap between complex autonomy software and traditional safety validation methods. These monitors act as mediators that continuously check the output of autonomy components against predefined safety constraints, enabling validation of machine learning systems without requiring complete reform of traditional testing approaches.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The safety architecture segments the validation function into two parts: (1) runtime invariant monitors that perform continuous safety checks on autonomy component outputs, and (2) traditional software testing that validates the monitors themselves. This segmentation allows advanced autonomy functions to be validated through the monitor layer without requiring direct application of traditional testing methods to the complex autonomy algorithms.

Inventive Principle:
Principle #1Segmentation

2Reliability

If runtime invariant monitors are deployed to firewall safety criticality, then resource-intensive software-safety engineering can focus on monitoring components, but autonomy components cannot be validated using traditional testing methods

Engineering Contradiction:
Improvesafety assuranceVSAvoidvalidation ease
Core Design Contradiction:
ReliabilityVSEase of manufacture

Solution Approach 1:

Runtime invariant monitors serve as intermediaries that isolate the validation burden from complex autonomy components. By placing safety checks at the monitor layer, the patent enables autonomy components to be developed and tested with more flexible methods while the monitors provide the formal safety assurance required by standards.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent extracts the safety-critical validation function from the autonomy components themselves and places it in separate runtime invariant monitor components. This extraction allows autonomy software to be developed using advanced techniques that are difficult to validate traditionally, while the extracted monitor components can be validated using conventional rigorous methods.

Inventive Principle:
Principle #2Taking out (Extraction)

3Reliability

If a single doer/checker pair is used, then the system structure is simple, but the system cannot continue operation if the autonomy component fails

Engineering Contradiction:
Improvefail-operational capabilityVSAvoidsystem complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies local quality by creating specialized doer and checker components with distinct roles. The doer components (autonomy software) perform complex autonomous functions while the checker components (runtime invariant monitors) perform specialized safety verification. This local specialization enables the system to maintain operation by having checkers continuously verify doer output even when doers fail.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The runtime invariant monitors perform preliminary safety checks on the output of autonomy components before the output is executed. By performing this safety verification in advance, the system can detect and respond to autonomy component failures before they cause harmful effects, enabling fail-operational behavior through immediate switching to safe states.

Inventive Principle:
Principle #10Preliminary action

4Reliability

If safety gate components are designed with high integrity, then safety requirements are upheld, but the complexity of the safety architecture increases

Engineering Contradiction:
Improvesafety integrityVSAvoidarchitecture complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent extracts safety-critical functions into separate runtime invariant monitor components that can be designed and validated with high integrity using traditional methods. By taking out the safety verification function from the complex autonomy software, the patent enables high-integrity safety gates without requiring the entire autonomy system to be redesigned with equally high complexity.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

High-integrity safety gate components act as intermediaries between the complex autonomy software and the controlled system. These mediator components provide the rigorous safety assurance required by standards while allowing the autonomy components to maintain their inherent complexity and use advanced techniques that would be difficult to validate directly.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS10962972B2Safety architecture for autonomous vehicles
Publication Date: 2021.03.30 CARNEGIE MELLON UNIV
  • US10962972B2 patent drawing
  • US10962972B2 patent drawing
  • US10962972B2 patent drawing

AI summary

A safety architecture system includes, in one aspect, a first stage comprising a primary unit that generates primary data for performing normal system functionality; a secondary unit that generates secondary data for performing alternative system functionality; a primary safety gate coupled to the primary unit, with the primary safety gate providing the primary data as a primary output responsive to a determination of validity of the primary data; and a secondary safety gate coupled to the secondary unit, with the secondary safety gate providing the secondary data as a secondary output responsive to a determination of validity of the secondary data. The system also includes an output selector that is coupled to both the primary safety gate and the secondary safety gate of the first stage, with the output selector providing a system output responsive to the determinations of the validities of the primary data and the secondary data.