Method to use a receiver operator characteristics curve for model comparison in machine condition monitoring

Abstract

A method to use a receiver operator characteristics curve for model comparison in machine condition monitoring. The method and systems of using this method may be used to evaluate different monitoring models. These models may be used to monitor a variety of different systems such as power plant systems or magnetic resonance imaging systems. The methods use training data and designate one or more points in the data as a false negative, thereby permitting a receiver operator characteristics analysis to be performed. Multiple receiver operator characteristics analyses may be performed either on different models or on different points within a single model, thereby permitting the receiver operator characteristics analyses to be used to select a beneficial model for monitoring a particular system.

Description

FIELD OF THE INVENTION[0001]This invention is directed generally to model evaluation, and more particularly to evaluating models for monitoring systems.BACKGROUND[0002]Receiver Operating Characteristics (ROC) curves have long been used to evaluate classifier performance in many fields (e.g. signal detection and machine learning). An ROC curve provides information on the tradeoff between the hit rate (true positives) and the false alarm rates (false positives). In order to draw the ROC curve both positive and negative examples are needed.[0003]In some applications, for example, machine condition monitoring, there are plenty of negative examples. However the positive examples are either rare, or do not fully describe the overall set of the possible positive examples. Instead of the positive examples, some rules about the positive examples are known. For example, if a sensor drifts off from the set of observed states by a certain amount, a fault has occurred.[0004]In the prior art, ROC...

AI Technical Summary

Benefits of technology

[0006]This present invention provides a method of using a receiver operator characteristics curve for model comparison in machine condition. The method and systems of using this method may be used to evaluate different monitoring models. These models may be used to monitor a variety of different systems such as power plant systems or magnetic resonance imaging systems. The methods use training data and designate one or more points in the data as a false negative, thereby permitting a receiver operator characteristics analysis to be performed. Multiple receiver operator characteristics analyses may be performed either on different models or on different points within a single model, thereby permitting the receiver operator characteristics analyses to be used to select a beneficial model for monitoring a particular system.

Problems solved by technology

However the positive examples are either rare, or do not fully describe the overall set of the possible positive examples.

For example, if a sensor drifts off from the set of observed states by a certain amount, a fault has occurred.

Nevertheless, to date, no prior art references have taught or suggested the use of ROC curves as a means for comparing different operating models, especially those operating models used for monitoring power plants.

Images