Prognostic Algorithm Verification Using Confidence Intervals

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Prognostic algorithms for predicting the remaining useful life of components in air vehicles and integrated systems face challenges in verifying their performance, particularly in characterizing uncertainty and ensuring that they capture a specified percentage of failures, due to difficulties in working with probability distributions and the lack of available data for verification.

Innovation Solution

A method and system for verifying the requirements of prognostic algorithms using field maintenance data, which involves providing a template for algorithm requirements, determining a probability density function, and calculating a confidence value to assess whether the algorithm meets the specified failure avoidance requirements, applicable to components with multiple failure modes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If prognostic algorithms are used to predict time-to-maintenance, then availability is increased and logistics footprint is reduced, but verification of algorithm performance becomes difficult due to lack of field data and uncertainty characterization

Engineering Contradiction:

Improvecomponent reliabilityVSAvoidverification complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent introduces confidence intervals as an intermediary statistical tool that mediates between the prognostic algorithm's predictions and the verification process. By calculating confidence intervals around time-to-maintenance predictions, the system creates a verifiable metric that accounts for uncertainty without requiring complex probabilistic modeling, thus resolving the verification difficulty while maintaining reliability improvements

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the traditional mechanical verification approach (direct observation of algorithm accuracy) with a statistical verification method using confidence intervals and hypothesis testing. This substitution allows verification using existing field maintenance data without requiring complex real-time monitoring systems, reducing verification complexity while maintaining reliability assessment capability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If traditional maintenance procedures are used, then verification is straightforward, but maintenance effectiveness is reduced due to overly conservative replacement schedules

Engineering Contradiction:

Improvemaintenance efficiencyVSAvoidfailure prediction precision

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent makes maintenance schedules dynamic by using confidence intervals to adapt replacement timing based on actual component behavior and prediction uncertainty. Instead of fixed conservative schedules, the system dynamically adjusts maintenance timing within confidence bounds, improving maintenance efficiency while maintaining sufficient prediction precision for safety-critical applications

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter used for maintenance scheduling from fixed time intervals to confidence-based time windows. By using the upper and lower bounds of confidence intervals as flexible scheduling parameters, the system improves maintenance efficiency by avoiding premature replacements while maintaining precision through statistical guarantees on prediction reliability

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If confidence intervals are calculated using t-distribution, then verification is possible with limited field data, but accuracy decreases when sample size is small

Engineering Contradiction:

Improveprediction accuracyVSAvoidfield data quantity

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent applies partial action by using confidence intervals with acceptable (rather than optimal) precision when field data is limited. Instead of requiring large sample sizes for high precision, the system uses available data to calculate confidence intervals that provide sufficient verification capability, accepting reduced precision as a trade-off for enabling verification with limited data

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS9400960B2Methods for verifying satisfaction of prognostic algorithm requirements for a component having multiple failure modes

Publication Date: 2016.07.26 LOCKHEED MARTIN CORP

AI summary

Methods for verifying satisfaction of prognostic algorithm requirements for a component of a certain device of interest are provided. A method according to an example of an embodiment of the invention can include providing a prognostic algorithm requirements statement for a preselected component contained in each of a plurality of a certain type device, receiving field data indicating a number of premature component failures and a total number of replacements including both due to premature failures and scheduled maintenance, determining a probability density function providing a probability of failing to replace the prematurely failed components, determining a confidence value indicating a level of confidence that failure avoidance requirements provided in the prognostic algorithm statement are being met, and verifying whether or not the prognostic algorithm requirements provided in the prognostic algorithm requirements statement are being satisfied according to a preselected minimum level of confidence.