Overall averaging of retained extremums for ML anomaly detection

Abstract

Systems, methods, and other embodiments associated with preserving signal extremums for ML model training when integrally averaging time series signals for ML anomaly detection are described. In one embodiment, a method includes identifying a location and a value of an extremum in a training signal; integrally averaging the training signal to generate an averaged training signal; placing the value of the extremum in the averaged training signal at the respective position of the extremum to produce an averaged training signal retaining the extremum; placing the value of the extremum in the averaged training signal at the respective position of the extremum to produce an averaged training signal retaining the extremum; and training a machine learning model to detect anomalies in the signal using the training signal with the averaged extremum reserved.

Description

Technical Field

[0001] The systems and methods described herein relate to the training of machine learning models for anomaly detection. Background Art

[0002] A large number of sensors can be used to monitor the operation of various assets such as data centers, passenger aircraft, and refineries. Time series data or signals from sensors can be used for machine learning (ML) time series prognostic monitoring to detect incipient failures of monitored assets before failures occur. This makes it possible to take corrective actions before the monitored assets fail.

[0003] The accumulated volume of time series data is outstripping the processing capabilities of the cloud. The computational cost (in terms of processor time and memory) of analyzing sample-intensive time series signals can be prohibitively expensive. For large-scale machine learning (ML) use cases involving hundreds of thousands of time series signals, the processing requirements for training ML models can exceed the capacity of cloud computing systems. Summary of the invention

[0004] In one embodiment, a computer-implemented method is provided. The method includes identifying the location and value of an extremum in a training signal. The method includes averaging the training signal as a whole to produce an averaged training signal. The method includes placing the value of the extremum into the averaged training signal at the corresponding location of the extremum to produce an averaged training signal that retains the extremum. And, the method includes training a machine learning model with the averaged training signal that retains the extremum to detect anomalies in the signal.

[0005] In one embodiment, a computer-implemented method is provided. The method includes identifying the location and value of an extremum in a training signal. The method includes averaging the training signal as a whole to produce an averaged training signal. The method includes placing the value of the extremum into the averaged training signal at the corresponding location of the extremum to produce an averaged training signal that preserves the extremum. And, the method includes training a machine learning model with the averaged training signal that preserves the extremum to facilitate detecting one or more anomalies in one or more monitoring signals from one or more sensors so as to detect an incipient or actual failure of a device monitored by the one or more sensors.

[0006] In one embodiment, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium includes computer-executable instructions stored thereon, which, when executed by at least one processor of a computer, cause the computer to perform operations or steps of a method. The instructions cause the computer to identify the location and value of (1) a minimum value and the location and value of (2) a maximum value of a training signal. The instructions cause the computer to retain the minimum value and the maximum value. The instructions cause the computer to perform an overall average of the training signals in the training set to produce an averaged training signal. The overall average eliminates the minimum and the maximum values ​​in the averaged training signal. The instructions cause the computer to place the value of the minimum value in the averaged training signal at the location of the minimum value and place the value of the maximum value in the averaged signal at the location of the maximum value to produce an averaged training signal with extreme value retention. Furthermore, the instructions cause the computer to train a machine learning model with the averaged training signal with extreme value retention to detect anomalies in the signal.

[0007] In one embodiment, a computing system is provided. The computing system includes at least one processor, at least one memory operably connected to the processor, and one or more non-transitory computer-readable media. The computer-readable medium includes instructions stored thereon, which, when executed by at least the processor, cause the computing system to perform the operations or steps of the method. The computing system is caused to identify the location and value of extreme values ​​in the training signal. The computing system is caused to retain the extreme values. The computing system is caused to average the training signal to produce an averaged training signal. The averaging eliminates the extreme values. The computing system is caused to generate an averaged training signal that retains the extreme values ​​by placing the retained values ​​of the extreme values ​​into the averaged training signal at the corresponding positions of the extreme values. And, the computing system is caused to detect anomalies in other averaged signals based at least in part on the generated averaged training signal that retains the extreme values.

[0008] In one embodiment, a computer-implemented method is presented. The method includes identifying the location and value of an extremum in a training signal obtained from a sensor for monitoring a first physical state of an asset. The method includes averaging the training signal as a whole to remove noise from the training signal to produce an averaged training signal. The method includes placing the value of the extremum into the averaged training signal at the corresponding location of the extremum to produce an averaged training signal that retains the extremum. And, the method includes training a machine learning model with the averaged training signal that retains the extremum to detect anomalies in a signal obtained from a sensor for monitoring a second physical state of the asset, wherein the anomaly indicates a degradation that has occurred in the second physical state of the asset. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings that are incorporated into the specification and form a part thereof illustrate various systems, methods and other embodiments of the present disclosure. It will be appreciated that the element boundaries (e.g., boxes, groups of boxes or other shapes) shown in the figures represent an embodiment of boundaries. In some embodiments, an element can be implemented as multiple elements, or multiple elements can be implemented as one element. In some embodiments, an element that is shown as an internal component of another element can be implemented as an external component, and vice versa. In addition, elements may not be drawn to scale.

[0010] Figure 1 An embodiment of an ensemble averaging system for retaining extreme values ​​associated with retaining signal extreme values ​​for ML model training when ensemble averaging a time series signal for ML anomaly detection is illustrated.

[0011] Figure 2 An embodiment of an ensemble averaging method for retaining extreme values ​​associated with retaining signal extreme values ​​for ML model training when ensemble averaging a time series signal for ML anomaly detection is illustrated.

[0012] Figure 3 Another embodiment of an ensemble averaging method for retaining extreme values ​​associated with retaining signal extreme values ​​for ML model training when ensemble averaging a time series signal for ML anomaly detection is illustrated.

[0013] Figure 4 A signal graph showing an example application of an example method of ensemble averaging for preserving extreme values ​​to a time series signal is illustrated.

[0014] Figure 5 Illustrated are signal graphs comparing an example time series signal before and after applying one embodiment of extreme value-preserving ensemble averaging.

[0015] Figure 6 An embodiment of a computing system configured with the disclosed example systems and / or methods is illustrated. DETAILED DESCRIPTION

[0016] Systems, methods, and other embodiments are described herein for preserving signal extreme values ​​when ensemble averaging a time series signal to train an ML model. In one embodiment, an ensemble averaging system that preserves extreme values ​​preserves the observed minimum and maximum values ​​when ensemble averaging a time series signal used to train an ML model. Preserving these extreme values ​​at the locations where they occur and replacing the averaged values ​​of these locations with them during model training significantly reduces the false positive probability (FAP) in the ML model.

[0017] In one embodiment, an extremum-preserving ensemble averaging system identifies the locations of extrema in a training signal. The training signal is then ensemble averaged to produce an averaged signal, which may lose extrema. The extrema are then placed back into the averaged signal at the corresponding locations where they occurred. A machine learning model is then trained to detect anomalies using this extremum-preserving averaged signal.

[0018] In one embodiment, as used herein, the term "extremes" refers to minimum and maximum values ​​over a portion of a signal. In one embodiment, as used herein, the term "ensemble averaging" refers to generating an average value for a block of adjacent time steps in a signal, rather than generating an average value across multiple signals.

[0019] As used herein, the term "time series signal" refers to a data structure that indexes a series of data points (such as observations or sampled values) in time order. In one embodiment, the data points of a time series signal may be indexed by a timestamp and / or an observation number. In one embodiment, the data points of a time series signal repeat at uniform or consistent intervals.

[0020] As used herein, the term "time series database" refers to a data structure that includes one or more time series signals that share a common index, such as a series of timestamps or observation numbers.

[0021] — Example of an extreme value-preserving ensemble averaging system —

[0022] Figure 1 An embodiment of an ensemble averaging system 100 for preserving extrema associated with preserving signal extrema for ML model training when ensemble averaging a time series signal for ML anomaly detection is illustrated. The ensemble averaging system 100 for preserving extrema includes an extrema locator 105, an ensemble averager 110, an extrema replacer 115, and an ML model trainer 120. In one embodiment, each of these components 105, 110, 115, and 120 of the ensemble averaging system 100 for preserving extrema can be implemented as a module.

[0023] In one embodiment, the extremum locator 105 is configured to identify the location of the extremum for the training signal 125. In one embodiment, the extremum locator 105 is configured to receive (or retrieve from a memory or storage device) the training signal 125. In one embodiment, the training signal 125 is a time series signal. In one embodiment, the training signal 125 is one of a plurality of signals in a training database, in which case the extremum locator 105 is configured to distinguish the training signal 125 from the other signals in the training database. In one embodiment, the training database is a time series database.

[0024] In one embodiment, the extrema locator 105 is configured to parse the values ​​of the training signal 125 to identify the locations of the extrema in the training signal 125. In one embodiment, the extrema are the maximum values ​​in the training signal 125 and the minimum values ​​in the training signal 125. In one embodiment, the extrema locator 105 is configured to detect (and record) the maximum values ​​of the training signal 125 and record the index position of the maximum values. In one embodiment, the extrema locator 105 is configured to detect (and record) the minimum values ​​of the training signal 125 and record the index position of the minimum values. The locations 130 of the extrema (minimum and maximum values) and their associated values ​​can then be provided to the extrema replacer 115.

[0025] In one embodiment, the ensemble averager 110 is configured to ensemble average the training signal to produce an averaged training signal. In one embodiment, the ensemble averager 110 is configured to receive (or retrieve from a memory or storage device) a training signal 125. In one embodiment, the ensemble averager 110 is configured to divide the values ​​of the training signal 125 into segments suitable for an averaging window. The ensemble averager 110 is configured to determine the average of the values ​​in the segment, thereby producing an ensemble average of the segment. The ensemble averager 110 is configured to append the determined ensemble average to the averaged training signal 135. Once the ensemble averaged value is determined for each segment and appended to the averaged training signal 135, the averaged training signal 135 is complete and can then be provided to the extreme value replacer 115.

[0026] In one embodiment, the extreme value replacer 115 is configured to place the extreme values ​​in the averaged training signal at the corresponding positions of the extreme values ​​to produce an averaged training signal that retains the extreme values. In one embodiment, the extreme value replacer 115 is configured to receive (or retrieve from a memory or storage device) the averaged training signal 135 and the positions 130 of the extreme values ​​and the values ​​of the extreme values. In one embodiment, the extreme value replacer 115 is configured to determine (i) the minimum position in the averaged training signal 135 corresponding to the position where the minimum value appears in the training signal 125, and (ii) the maximum position in the averaged training signal 135 corresponding to the position where the maximum value appears in the training signal 125. In one embodiment, the extreme value replacer 115 is configured to place (i) the minimum value from the training signal 125 at the minimum position in the averaged training signal 135, and (ii) the maximum value from the training signal 125 at the maximum position in the averaged training signal 135. Thus, in one embodiment, the extrema replacer is configured to replace the averaged values ​​with extrema at locations in the averaged training signal 135 corresponding to locations where extrema occur in the training signal 125. Thus, in one embodiment, the extrema replacer 115 generates an averaged training signal 140 that retains the extrema. The averaged training signal 140 that retains the extrema can be used to train an ML model to detect anomalies in the overall averaged signal.

[0027] In one embodiment, the ML model trainer 120 is configured to train a machine learning model to detect anomalies in a signal using the average training signal that retains extrema. In one embodiment, the ML model trainer 120 is configured to receive (or retrieve from a memory or storage device) the average signal 140 that retains extrema. In one embodiment, the ML model trainer 120 is configured to adjust the parameters of the ML anomaly detection model 145 so that the ML anomaly detection model 145 is configured to predict the value of the average training signal 140 that retains extrema within a predetermined level of accuracy.

[0028] In one embodiment, upon completion of training, the ML anomaly detection model 145 is configured to detect anomalies in the monitoring signal 150. In one embodiment, the ML anomaly detection model 145 is configured to generate an alert 155 in response to detection of an anomaly in the monitoring signal 150.

[0029] Further details about the ensemble averaging system 100 for preserving extreme values ​​are provided herein. Figure 2 and Figure 3 The operation of the ensemble averaging system 100 for preserving extreme values ​​is described by referring to the example method for ensemble averaging for preserving extreme values ​​shown in FIG. Figure 4 and Figure 5The signal diagrams shown in FIG. 1 are used to describe the operation of the extremum locator 105, the ensemble averager 110, and the extremum replacer 115 on the time series signal. Further details on the operation of the ensemble averaging system 100 preserving extremums will be explained herein in the context of experimental verification of the effectiveness and accuracy of the ensemble averaging system 100 preserving extremums.

[0030] —Example of the ensemble averaging method that retains extreme values—

[0031] Figure 2 An embodiment of an ensemble averaging method 200 for retaining extreme values ​​associated with retaining signal extreme values ​​for ML model training when ensemble averaging a time series signal for ML anomaly detection is illustrated. In one embodiment, the ensemble averaging method 200 for retaining extreme values ​​is a method for retaining minimum and maximum values ​​for ML model training when ensemble averaging signal values ​​to reduce signal granularity and smooth the signal.

[0032] In summary, in one embodiment, the ensemble averaging method 200 that retains extreme values ​​first identifies the locations of extreme values ​​(i.e., minimum signal values ​​and maximum signal values) in the training signal. Then, the ensemble averaging method 200 that retains extreme values ​​performs ensemble averaging on the training signal to generate an averaged training signal. Then, the ensemble averaging method 200 that retains extreme values ​​places the extreme values ​​into the averaged training signal at the corresponding locations of the extreme values. For example, in one embodiment, the minimum value is placed into the averaged training signal at the location of the minimum value, and the maximum value is placed into the averaged signal at the location of the maximum value. Thereby generating an averaged training signal that retains extreme values. Then, the ensemble averaging method 200 that retains extreme values ​​uses the averaged training signal that retains extreme values ​​to train a machine learning model to detect anomalies in the signal.

[0033] In one embodiment, the ensemble averaging method 200 for preserving extrema is initiated at the start block 205 in response to a processor of a computer determining one or more of the following: (i) a training signal (e.g., the training signal 125) or a signal database including one or more training signals has been provided to the ensemble averaging system 100 for preserving extrema; (ii) a user or administrator of the ensemble averaging system 100 for preserving extrema has initiated the method 200; (iii) it is currently the time when the method 200 is scheduled to run; or (iv) the method 200 should be initiated in response to the occurrence of some other condition. In one embodiment, the computer is configured by computer executable instructions to perform the functions of the ensemble averaging system 100 for preserving extrema. After initiation at the start block 205, the method 200 continues to the processing block 210.

[0034] At processing block 210, the processor identifies the location and value of an extremum in the training signal. In one embodiment, the processor examines the signal values ​​of the training signal to determine which value is an extremum, and then records the signal value and the location of the extremum in the training signal. In one embodiment, the processor parses the training signal to find two extremums: a maximum or maximum signal value in the training signal, and a minimum or minimum signal value in the training signal. Once an extremum is found, the processor can record the value of the extremum, and an index value indicating the placement of the extremum in the training signal.

[0035] In one embodiment, the extreme values ​​in the training signal are the maximum signal value and the minimum signal value within the training signal. In one embodiment, as used herein, the "position" of the extreme value (or other value) within the signal refers to the timestamp or time position at which the value appears. In one embodiment, the position of the extreme value (or other value) in the training signal can be referenced by an index such as a timestamp or an observation number (from which an approximate timestamp can be derived). In one embodiment, the value of the extreme value (or other signal value) is an amplitude or other quantity representing a sensor's reading of a physical phenomenon. In one embodiment, the training signal is a time series of readings describing a physical phenomenon. In one embodiment, the training signal is a time series signal belonging to a time series database of one or more time series signals. In one embodiment, the time series signal is a training set that includes less than all observations of the time series signal.

[0036] In one embodiment, the location of the extreme values ​​can be identified by parsing the observed values ​​and indexes in the training signal. In one embodiment, the processor compares the observed values ​​to each other (e.g., as described in more detail herein with respect to processing blocks 335 and 340) to find which has a maximum or largest value and which has a minimum or smallest value. The processor then stores the value and index (location within the training signal) of the maximum value and the value and index (location within the training signal) of the minimum value. The values ​​and indexes of these extreme values ​​can be retrieved from a storage device for subsequent processing. Therefore, the values ​​(minimum and maximum values) of the extreme values ​​are retained. In one embodiment, the values ​​and indexes (locations) of the extreme values ​​are identified for multiple training signals in a training database.

[0037] In one embodiment, the functionality of processing block 210 may be performed by extrema locator 105 or other components of system 100. Processing block 210 is completed, and method 200 continues at processing block 215. At the completion of processing block 210, the processor has identified the extrema in the training and recorded their corresponding values ​​and positions in the training signal. The values ​​and positions of the minimum and maximum values ​​are retained for subsequent reinsertion into the overall averaged signal, as described below.

[0038] At processing block 215, the processor performs overall averaging of the training signal to produce an averaged training signal. In one embodiment, the processor subdivides the training signal into averaging windows, calculates an average of the training signal values ​​in the averaging windows, and creates an averaged training signal based on the calculated average. In one embodiment, the processor. In one embodiment, the average is the mean of the values. In one embodiment, the average is the median of the values.

[0039] In one embodiment, ensemble averaging as used herein refers to a series of averages of values ​​generated in successive, non-overlapping signal segments of constant length. In one embodiment, ensemble averaging as used herein is a form of moving averaging in which the averaging windows do not overlap and are continuous. In one embodiment, the average of the training signal values ​​within the averaging window becomes the value of the averaged training signal. Therefore, in one embodiment, the averaged training signal is a time series of the average of the training signal values.

[0040] The averaging of signal values ​​can eliminate extreme values. For example, averaging across an average window causes each signal value in the window to be replaced by the average value of the signal value in the window, thereby eliminating any extreme values ​​that occur in the window. Due to the influence of averaging the values ​​in the window, averaging eliminates the minimum and maximum values ​​in the averaged training signal. When the maximum value exists in a window with other lower signal values, the average value of the signal value in the window will be less than the maximum value, thereby eliminating the maximum value. When the minimum value exists in a window with other higher signal values, the average value of the signal value in the window will be greater than the minimum value, thereby eliminating the minimum value. Therefore, when generating an averaged signal from the average value of the signal value, averaging causes the extreme values ​​in the signal to be deleted, replaced, lost or otherwise eliminated. In order to keep these extreme values ​​from being eliminated due to averaging, the position and value of the extreme values ​​can be stored or saved as a data structure so as to be put back into the averaged signal. Therefore, the average value of the signal value falls between the extreme values. In one embodiment, overall averaging as used herein refers to any algorithm for averaging the values ​​of the signal, which also eliminates the extreme values ​​due to averaging.

[0041] In one embodiment, the processor parses the training signal to assign the start and end positions of the averaging window in the training signal. For example, the processor counts the fragments of the training signal, which are the length (WS) of the averaging window. The length WS of the averaging window can be expressed with the time step of a signal (such as a training signal) or the number of observations. Then, the processor averages the values ​​of the training signal that appear in the averaging window to generate the averaged value of the window. The averaged value is then stored in the averaged signal. In one embodiment, the averaged value is found and added to the averaged signal in the order of the index value of the training signal. In this way, the processor can generate the averaged training signal as a data structure. For example, the averaged training signal is created as a time series signal, wherein the averaged value is its data point. The averaged value in the training signal can be indexed with a timestamp or observation number corresponding to the averaging window from which the averaged value is generated.

[0042] In one embodiment, due to the overall averaging, the observations in the training signal are condensed into fewer observations made at longer sampling intervals in the averaged training signal. For example, the sampling interval in the averaged training signal is the length of the averaging window. Thus, in one embodiment, the length of the averaged training signal (L A ) is the length of the training signal (L) divided by the size of the averaging window (WS).

[0043] In one embodiment, each value in the averaged training signal corresponds to an averaging window applied to the training signal. In one embodiment, the value in the averaged training signal is the average of the values ​​of the training signal occurring within the averaging window. Thus, in one embodiment, the value in the averaged training signal resulting from averaging the training values ​​within the averaging window corresponds to the averaging window.

[0044] In one embodiment, the averaged training signal values ​​share the same order as the training signal. For example, a first averaged training signal value averaging a first WS training signal value is followed by a subsequent second averaged training signal value averaging the next WS training signal value after the first WS training signal value, and so on.

[0045] refer to Figure 4An example of ensemble averaging is shown and described elsewhere herein. In one embodiment, the functionality of processing block 215 may be performed by ensemble averager 110 or other components of system 100. Processing block 215 is then completed and method 200 continues at processing block 220. In one embodiment, at the completion of processing block 215, an ensemble averaged signal of averaged values ​​has been created from the training signal. In one embodiment, the averaged value is the average of a range of values ​​in the training signal (i.e., an averaging window). Ensemble averaging filters or smoothes noise within the training signal. Relative to an ML model trained on the training signal, the removal of noise enhances the accuracy of the ML model trained on the averaged training signal. Ensemble averaging also reduces the size of the training signal by a ratio of 1 / WS. The reduction in the size of the training signal reduces the memory footprint of training the ML model on the averaged training signal compared to training the ML model on the training signal.

[0046] At processing block 220, the processor places the value of the extremum at the corresponding position of the extremum into the averaged training signal to produce an averaged training signal that retains the extremum. In one embodiment, the processor extracts the corresponding position of the extremum. For example, the corresponding position of the extremum is the position of the minimum value in the training signal, and the position of the maximum value in the training signal. In one embodiment, the processor extracts the corresponding value of the extremum. For example, the corresponding value of the extremum is the minimum value in the training signal and the maximum value in the training signal.

[0047] In this way, the processor can generate the average training signal retaining the extreme value as a data structure, such as a time series signal. The processor can generate the average training signal retaining the extreme value by modifying the data structure of the average training signal. The processor can replace the value in the average training signal with the value of the extreme value to convert the average training signal into the average training signal retaining the extreme value. For example, the processor can locate the position corresponding to the position of the maximum value in the data structure of the average training signal, and replace the average value at the position with the value of the maximum value. And, the processor can locate the position corresponding to the position of the minimum value in the data structure of the average training signal, and replace the average value at the position with the value of the minimum value. By performing these replacements, the processor can modify the data structure of the average training signal to the data structure of the average training signal retaining the extreme value.

[0048] In one embodiment, when the training signal is converted into an averaged training signal, the length of the signal is changed from L to L A, and the observation number can change. For example, when the observation number of the minimum value in the training signal is #20, the observation number of the position of the minimum value can become #4 in the averaged training signal with a WS of 5. Therefore, in one embodiment, as discussed above, the position of the extreme value is based on the time position of the timestamp. Even if the observation index number and granularity do change, the position of the extreme value based on the timestamp does not change the time position between the training signal and the averaged training signal. In one embodiment, the position can be an approximate time position to allow for the granularity of the signal. Therefore, in one embodiment, the position of the extreme value remains consistent in time within the training signal and the averaged training signal, regardless of the change in granularity or observation index number between the signals.

[0049] In one embodiment, the processor then determines the overall averaging window in which the extreme value occurs. For example, the processor may divide the position of the extreme value by the window size (WS) and round up to the nearest integer. The resulting number indicates the position of the averaging window in which the extreme value occurs. Because the position of the overall averaging window corresponds to the position of the averaged values ​​of these windows within the averaged signal, the resulting number also indicates the index of the observation within the averaged signal that corresponds to the overall averaging window.

[0050] In one embodiment, the processor then substitutes the overall average of the observations within the averaged signal corresponding to the overall averaging window with the value of the extreme value. In one embodiment, the processor identifies the index of the observation. In one embodiment, the processor then overwrites, replaces or substitutes the average of the observations at that index with the value of the extreme value. Thus, in one embodiment, the value of the extreme value is placed in the averaged training signal at the position that the extreme value occupied in the training signal. Once the extreme value returns to the averaged training signal, the averaged training signal becomes the averaged training signal that retains the extreme value.

[0051] Thus, in one embodiment, the value of the extremum is repositioned or reinserted into the averaged signal at a position corresponding to its original position in the training signal. In one embodiment, the position corresponds to a timestamp. In one embodiment, the value of the extremum occupies an approximate time position in the averaged training signal that retains the extremum as that occupied by the extremum in the training signal. An example of such repositioning of the extremum into the averaged training signal is shown in FIG. Figure 4 In one embodiment, the functionality of processing block 220 may be performed by extrema replacer 115 or other components of system 100. Processing block 220 is then complete and method 200 continues at processing block 225.

[0052] In one embodiment, upon completion of processing block 220, an average training signal preserving extrema has been generated that preserves the values ​​of the extrema at their respective locations. The average training signal preserving extrema can then be used to train an ML model to detect anomalies. In one embodiment, an ML model trained on the average training signal preserving extrema exhibits higher prediction accuracy than an ML model trained on the average training signal or an unprocessed training signal. For example, reinserting or repositioning extrema in the signal used to train the ML model eliminates the possibility that a value in the monitoring signal monitored by the ML model triggers a false alarm due to being out of the range of values ​​in the training signal used to train the ML model.

[0053] At processing block 225, the processor trains a machine learning model with the averaged training signal retaining extreme values ​​to detect anomalies in the signal. In one embodiment, the processor trains the ML model to predict values ​​in the signal based at least in part on the averaged training signal retaining extreme values. In one embodiment, anomalies are detected by deviations from the predicted values.

[0054] For general context, the ML model for anomaly detection operates to estimate or predict the expected value of an individual time series signal based on the input value of one or more other signals. The ML model can be trained to make accurate predictions of the expected value of each signal. A training time series database is provided as an input to the ML model. The training time series database includes the training portion of the individual signal and one or more other signals as component time series signals. The training process involves iteratively optimizing the configuration of the ML model until the ML model consistently predicts that the expected value of the training portion of the individual signal matches the actual value of the training portion of the individual signal (within an acceptable tolerance). When the training is completed, the ML model can be used to monitor or monitor whether the individual time series signal has anomalies. A monitoring time series database including the monitoring portion of the individual signal and one or more other signals is provided as an input to the ML model. The ML model predicts the expected value of the monitoring portion of the individual signal based on the value of the monitoring portion of the one or more other signals. During monitoring, the expected value predicted by the ML model for the individual signal can be compared with the actual value or observed value (e.g., the value received from the sensor) of the individual signal. The difference between the actual value of the individual signal and the expected value of the individual signal predicted by the model can indicate anomalies in the actual individual signal. Therefore, ML models can be trained to detect anomalies in individual signals.

[0055] In one embodiment, the averaged signal retaining the extreme values ​​is used to train the ML model so that the ML model detects anomalies in another overall averaged signal with fewer false alarms. In one embodiment, the averaged signal retaining the extreme values ​​is included in a training database along with another overall averaged signal that the ML model is to be trained to detect anomalies. The training database is provided as an input to the ML model for training.

[0056] In one embodiment, the average signal retaining the extreme values ​​is a component signal of a training database, which is provided as an input for training the ML model. In the training database, component signals other than the average signal retaining the extreme values ​​are also averaged as a whole using the averaging window size WS as described above. Moreover, one or more of the component signals other than the average signal retaining the extreme values ​​may also have their extreme values ​​retained as described above. In one embodiment, each component signal of the training database is averaged as a whole using the averaging window size WS, and the extreme values ​​are retained.

[0057] Once the training database is provided to the ML model, the ML model is trained (in a manner similar to that discussed above for context) to detect anomalies in another ensemble averaged signal based on the extreme value-retained averaged signal. Both the other ensemble averaged signal and the extreme value-retained averaged signal are included in the training database provided to the ML model. In particular, the ML model is trained to predict expected values ​​of another averaged signal in the training database based on the values ​​of the extreme value-retained averaged signal in the database. For example, the configuration of the ML model is iteratively adjusted until the ML model predicts an expected value of another averaged signal that matches the actual value of the other averaged signal. In this way, the extreme value-retained averaged signal in the signal database is used to train the ML model to detect anomalies in another individual signal in the signal database.

[0058] When training is complete, the ML model can be used to detect anomalies in the monitored portion of another averaged signal where there is a difference between the expected value of the ML model and the actual value. Because the ML model is trained to detect anomalies using an averaged signal that retains extreme values ​​(which retains the maximum and minimum values ​​of the training signal), the ML model detects anomalies in the other averaged signal and reduces false positives. False positives are reduced because the averaged values ​​of the averaged signal that does not exhibit anomalies should generally be between the minimum and maximum values ​​of the training signal.

[0059] The trained ML model is then stored for subsequent use in monitoring the overall averaged signal (such as other averaged signals) for anomalies. For example, the ML model may monitor a monitoring portion of a signal database, of which the training database is also a part. The monitoring portion of the signal database may include monitoring portions of other averaged signals, which averaged signals may also be referred to herein as monitoring signals.

[0060] In one embodiment, the functionality of processing block 225 may be performed by ML model trainer 120 or other components of system 100. Processing block 225 is then completed and method 200 proceeds to end block 230 where method 200 is completed. At the completion of method 200, the ML model has been trained to detect abnormal behavior in the ensemble-averaged signal. Training the ML model using an ensemble-averaged training signal that retains extreme values ​​ensures that the ML model is trained to accommodate the widest amplitude range of signal values. In one embodiment, an ML model trained with an ensemble-averaged training signal that retains extreme values ​​as described herein will generate few, if any, false alarms due to monitoring values ​​outside the training range. Thus, in one embodiment, a reduction in memory footprint and processing burden is obtained by ensemble-averaging the signal while retaining extreme values ​​prevents the loss of AI prognostic accuracy that may occur due to ensemble averaging.

[0061] — Further Embodiment of the Global Averaging Method Preserving Extreme Values—

[0062] In another embodiment of the method 200, anomalies in the averaged signal are detected based on the averaged training signal retaining the extreme values. In one embodiment, the processing block 210 may include identifying the location and value of the extreme values ​​in the training signal and retaining the extreme values. For example, the extreme values ​​may be retained by storing them as a data structure in a storage device or memory.

[0063] In one embodiment, processing block 215 may include averaging the training signal to produce an averaged training signal. The training signal may be averaged by selecting consecutive, non-overlapping segments of the signal of constant length. (Signal segments may also be referred to as "windows," "moving windows," "averaging windows," or "global averaging windows," as described below with reference to Figure 3 and Figure 4215). The average of the values ​​in the signal segments can then be calculated. The average of the values ​​can then be placed as a data point in a time series signal data structure of an averaged training signal. The average of the values ​​is placed in the data structure at the index corresponding to the training signal segment from which the average was calculated. Once the average of the values ​​is placed in the data structure of the averaged training signal, an averaged training signal is generated. As discussed in further detail above with reference to processing block 215, averaging eliminates extreme values ​​in the averaged signal. The average of the signal values ​​is located between the extreme values.

[0064] In one embodiment, processing block 220 may include generating an averaged training signal that retains the extreme values ​​by placing the retained values ​​of the extreme values ​​into the averaged training signal at the corresponding positions of the extreme values. For example, the processor may modify the data structure of the averaged training signal by replacing two averaged values ​​with the retained extreme values ​​(retained minimum and maximum values). In particular, the averaged values ​​at the index corresponding to the training signal segment where the extreme value occurs are replaced with the retained values ​​of the extreme value.

[0065] In one embodiment, processing block 225 may include detecting anomalies in other averaged signals based at least in part on the generated averaged training signal that retains extreme values. For example, the other averaged signal is a monitoring signal. The processor may train an ML model with the averaged training signal that retains extreme values, and use the trained ML model to predict or estimate the value of the monitoring signal. The difference between the predicted value and the actual value of the monitoring signal may indicate an anomaly in the monitoring signal. Because the predicted value of the monitoring signal is estimated by the ML model trained with the averaged training signal that retains extreme values, the anomaly detected in the monitoring signal is detected based at least in part on the generated averaged training signal that retains extreme values.

[0066] In one embodiment, placing the values ​​of the extreme values ​​in the averaged training signal at the corresponding positions of the extreme values, as discussed above with reference to processing block 220, may also include determining an overall average window in which one of the extreme values ​​appears. The overall average window is a signal segment of the training signal for which the average value of the signal value in the segment is calculated, as discussed below with reference to Figure 3 and Figure 4 The average of the values ​​in the ensemble averaging window is used to represent the value of the training signal in the averaged training signal. Extreme values ​​appear in the ensemble averaging window, where the signal value within the ensemble averaging window includes the minimum or maximum value of the training signal.

[0067] Once the ensemble averaging window in which the extreme value occurs is determined, the value of the ensemble average corresponding to the ensemble averaging window is replaced by the value of one of the extreme values. The value of the ensemble average "corresponds" to the ensemble averaging window because it is the value in the averaged training signal that is obtained by averaging the values ​​of the training signal within the ensemble averaging window, as described herein with reference to Figure 4 This correspondence can also be based on the temporal position of the values ​​within the overall average window (as discussed above) and the temporal extent of the overall average window. Thus, the position within the averaged training signal where the values ​​of the extreme values ​​in the training signal should be retained is determined.

[0068] In one embodiment, as discussed above with reference to processing block 220, placing the values ​​of the minimum and maximum values ​​in the averaged training signal at the corresponding positions of the minimum and maximum values ​​can also include determining whether the minimum value occurs within a first overall averaging window. Then replacing the first overall averaged value corresponding to the first overall averaging window in which the minimum value occurs with the value of the minimum value, so that the value of the minimum value replaces the first overall averaged value in the averaged training signal. It is also determined whether the maximum value occurs within a second overall averaging window. Then replacing the second overall averaged value corresponding to the second overall averaging window in which the maximum value occurs with the value of the maximum value, so that the value of the maximum value replaces the second overall averaged value in the averaged training signal.

[0069] In one embodiment of processing block 215, ensemble averaging the training signal may include selecting the length of the ensemble averaging window or window size (WS), such as described herein with reference to Figure 3 The processor may then determine the number (M) of ensemble averaging windows covering the length (L) of the training signal, e.g., as discussed herein with reference to processing block 320. For the number (M) of ensemble averaging windows, the processor (i) averages the signal values ​​within the ensemble averaging windows to create an average signal value, e.g., as discussed herein with reference to processing blocks 215 and 355, (ii) appends the averaged signal value to the averaged training signal, and (iii) shifts the ensemble averaging window by the length of the ensemble averaging window, e.g., as discussed herein with reference to processing block 380. Thus, in one embodiment, the processor adds a new ensemble averaged signal value to the end of the averaged training signal for each ensemble averaging window position on the training signal. In one embodiment, as used herein, the term "append" refers to adding an item to the end of an ordered data structure, e.g., adding an observation to a time series at an index or position that follows (one or more) other observations in the time series.

[0070] In one embodiment, method 200 may also include monitoring other averaged signals with the trained machine learning model. As used herein, a monitoring signal is a signal to be monitored for anomalies. The monitoring signal to be monitored for anomalies by the trained machine learning model should first be averaged as a whole, for example in a manner similar to that described above with reference to processing block 215 for the training signal. Thus, the monitoring signal is averaged as a whole to produce an averaged monitoring signal. The extreme values ​​of the monitoring signal are not retained or maintained.

[0071] The processor then uses the trained machine learning model to monitor the average monitoring signal for anomalies. In one embodiment, the processor uses the trained machine learning model to monitor the average monitoring signal for anomalies by predicting the value of the average monitoring signal and comparing the predicted value of the average monitoring signal with the actual value. In order to monitor the average monitoring signal, the average monitoring signal is provided as an input signal to the trained ML model. The trained ML model is executed to predict or estimate the value of the average monitoring signal, and the residual (absolute value of the difference) between the observed value of the average monitoring signal and the value estimated by the ML model of the average monitoring signal is calculated. When the residual becomes larger, an anomaly can be detected in the average monitoring signal, as discussed below.

[0072] The absolute value of the difference between the predicted input signal value and the actual signal value is called the residual. Analysis of the difference or residual can indicate the presence of abnormal behavior in the average monitoring signal. In one embodiment, the processor executes a fault detection model (such as a sequential probability ratio test (SPRT)) to determine whether the behavior of the average monitoring signal is abnormal based on the residual. In one embodiment, the processor detects an anomaly in the average monitoring signal. For example, the processor detects the anomaly because the analysis of the residual meets a threshold indicating the presence of an anomaly. Therefore, the anomaly in the average signal is detected based on the difference between the predicted value and the actual value. Detection of an anomaly in the average monitoring signal indicates the presence of an anomaly in the monitoring signal.

[0073] In one embodiment, in response to detecting an anomaly, an alarm may be sounded. In one embodiment, an electronic alarm message is composed and transmitted indicating that abnormal behavior has been detected in the averaged monitoring signal. The alarm message may be transmitted for display on a display device, or transmitted to other components of the ensemble averaging system 100 or a broader time series data service to automatically initiate corrective action.

[0074] In one embodiment, during monitoring, monitoring signals are received in real time as a monitoring data stream from the sensor. In one embodiment, although the monitoring signals arrive in the form of a stream, the monitoring signals are averaged as a whole as the monitoring signals arrive. In one embodiment, as used herein, "real time" refers to substantially real-time operation, and the availability of the output depends on the delay between actions that is acceptable in the context of the monitoring application.

[0075] In one embodiment, the values ​​of the training signal are parsed in a first pass to identify the locations and values ​​of the extreme values. Thus, in one embodiment, the values ​​of the training signal are parsed in a first pass to identify the locations and values ​​of the minimum values ​​and the locations and values ​​of the maximum values, and these locations and values ​​are retained. The values ​​of the training signal are then parsed again in a second pass to average the training signal as a whole.

[0076] In one embodiment, the machine learning model is a multivariate machine learning model. In one embodiment, the machine learning model is a univariate machine learning model.

[0077] In one embodiment, the detection of anomalies provides an early warning of an impending failure of a monitored asset. Furthermore, the detection of anomalies may also indicate that a monitored asset has failed. Therefore, in one embodiment, the ensemble averaging method 200 with extreme value retention may also include detecting an impending or actual failure of a device based on the detection of one or more anomalies in one or more monitoring signals from one or more sensors.

[0078] In one embodiment, detecting one or more anomalies in one or more monitoring signals includes averaging the monitoring signals in the one or more monitoring signals as a whole to produce an average monitoring signal. Detection also includes executing a trained machine learning model to estimate the value of the average monitoring signal. Detection also includes calculating the residual between the observed value of the average monitoring signal and the value estimated by the machine learning model of the average monitoring signal. And, detection also includes analyzing the residual to detect at least one anomaly.

[0079] In one embodiment, an alert about an anomaly may initiate a response of a corrective action. For example, a corrective action may mitigate, remove, or otherwise correct one or more behaviors of the monitored device that generated the anomaly. In one embodiment, the holistic averaging method 200 retaining extreme values ​​may also include taking an action in response to detecting an incipient or actual failure of the monitored device (or asset). The action is at least one of: issuing an alert or taking a corrective action.

[0080] In one embodiment, the ensemble averaging method 200 retaining extreme values ​​can be applied to various assets or devices. Thus, in one embodiment, the monitored device is one of the following or a portion thereof: a computer, a server installation, a disk drive, a tape drive, a data center, a cooling system, an aircraft, a ship or vessel, a land vehicle, an industrial installation, a utility installation, a refinery, a motor, an engine, a turbine, a pump, a machine tool, a production line, or a robot.

[0081] In one embodiment, one or more non-transitory computer readable media store computer executable instructions (also referred to as program instructions). The computer executable instructions are configured to cause one or more computers to perform operations including the operations of the extreme value-retaining ensemble averaging method 200 (or other methods described herein) when executing the computer executable instructions.

[0082] In one embodiment, the computing system includes one or more computers. The computing system is configured by computer-executable instructions to perform operations including the operations of the extreme value-preserving ensemble averaging method 200 (or other methods described herein).

[0083] In one embodiment, a computer program product includes a computer program. When the computer program is executed by at least one processor of a computer, the processor performs operations including the operations of the global averaging method 200 (or other methods described herein) that retains extreme values. As an example, the computer program may include one or more computer executable instructions that cause the computer to perform operations. Therefore, an embodiment can be implemented by using a computer program product, which includes a computer program or computer executable instructions that, when executed by at least one processor of a computer system, cause the processor and / or computing system to perform one or more of the methods described herein.

[0084] —Improving performance by averaging the entire population while preserving extreme values—

[0085] Computational challenges, such as limitations on processing power and memory footprint, have become significant limiting factors for streaming multivariate machine learning (ML) anomaly detection. For large-scale streaming or batch ML anomaly detection use cases, the sensor count and sampling rate combined can exceed the capacity of cloud containers (configuration of CPU, GPU, and available RAM). Both sensor count and sensor sampling rate grow exponentially, causing the computational burden (shown by the number of sensors multiplied by the sampling rate) to grow doubly exponentially, far exceeding the available CPU / GPU processing rate and memory capacity.

[0086] To date, multivariate ML anomaly detection involves a trade-off between computational burden and artificial intelligence (AI) performance: achieving better AI performance involves higher computational burden, and conversely, reducing computational burden reduces AI performance. In this context, “AI performance” covers metrics such as: signal prediction accuracy, false positive probability (FAP) (a measure of the likelihood that the system generates an alert when the alert is not plausible), false negative probability (MAP) (a measure of the likelihood that the system does not generate an alert when the alert is plausible), and time to detection (TTD) (a measure of the early warning time to detect an incipient anomaly).

[0087] In one embodiment, the extremum-retaining ensemble averaging system and method described herein reduces, minimizes, or even eliminates the tradeoff between AI performance and computational burden. In one embodiment, the extremum-retaining ensemble averaging system and method as shown and described herein reduces the false positive probability (FAP) while reducing computational cost and memory footprint without hampering the capacity for degraded detection. In one embodiment, the extremum-retaining ensemble averaging system and method described herein thus reduces the computational burden of large-scale streaming or batch ML anomaly detection without reducing AI performance.

[0088] In one embodiment, the novel processing explained herein cleverly handles adjacent time steps, not with naive averaging of adjacent measured points, but with a novel "extreme value preserving" two-way data flow framework. This new extremum preserving ensemble averaging (EPEA) framework greatly reduces the memory footprint of ML training operations, while reducing the computational cost (e.g., measured in processor cycles or operations) of multivariate ML anomaly detection. In one embodiment, by applying the extremum preserving ensemble averaging system and method shown and described herein, the computational cost is linearly reduced during monitoring or anomaly detection operations. In one embodiment, by applying the extremum preserving ensemble averaging system and method shown and described herein, the computational cost is superlinearly reduced during more computationally intensive ML model training operations. This has the effect of significantly increasing throughput and reducing latency for ML analysis.

[0089] Furthermore, in one embodiment, the extrema-preserving ensemble averaging system and method as shown and described herein reduces false positives and improves model accuracy by preserving global extrema in the ensemble-averaged data, thereby eliminating small statistical changes between multivariate training and test ranges and thereby reducing the noise content of the data.

[0090] In one embodiment, the extremum-preserving ensemble averaging system and method as shown and described herein does not require hardware upgrades to computing or sensor hardware. Therefore, in one embodiment, the extremum-preserving ensemble averaging implementation can be immediately backward compatible with existing data centers and sensor systems.

[0091] — Additional example of extreme value-retaining ensemble averaging method —

[0092] Figure 3 Another embodiment of an overall averaging method 300 for retaining extreme values ​​associated with retaining signal extreme values ​​for ML model training when overall averaging time series signals for ML anomaly detection is illustrated. In one embodiment, method 300 shows an overall averaging algorithm for retaining extreme values ​​in a two-way process. In one embodiment, in an initial processing pass through a training set of an example signal database, for each of the N signals, the minimum and maximum values ​​are identified, and the indexes (timestamps) of the minimum and maximum values ​​are stored together with the values ​​of the minimum and maximum values. In one embodiment, in a second processing pass through the example signal database, for each of the N signals, the entire training set is overall averaged. In one embodiment, overall averaging is achieved by deploying a moving window of size WS every WS adjacent time steps and averaging the values ​​appearing in the moving window. The moving window may also be referred to as a "window", "averaging window" or "overall averaging window" in this article. The window size WS may also be referred to as the "length" of the window. The window size or window length can be expressed in terms of the number of time steps or observations of the time series signal. The ensemble averaging process is modified so that the maximum value of all N signals is substituted at the location where a maximum value is identified in the first pass, and the minimum value of all N signals is substituted at the location where a minimum value is identified in the first pass.

[0093] In one embodiment, the extrema-preserving ensemble averaging method 300 is initiated at a start block 305 in response to a processor of a computer determining one or more of the following: (i) a training signal (e.g., training signal 125) or a signal database including N training signals has been provided to the extrema-preserving ensemble averaging system 100; (ii) a user or administrator of the extrema-preserving ensemble averaging system 100 has initiated the method 300; (iii) it is currently the time when the method 300 is scheduled to run; or (iv) the method 300 should be initiated in response to the occurrence of some other condition. After initiation at the start block 305, the method 300 continues to the processing block 310.

[0094] At processing block 310 , the processor initializes a signal database comprising N signals. In one embodiment, the signal database is initialized by retrieving the signal database from a memory or storage device and making the component signals of the database available to the processor. The method 300 then continues at processing block 315 .

[0095] At processing block 315, the processor selects the length of the overall averaging window or window size WS. In one embodiment, the processor determines the window size based on the target maximum memory usage during ML model training. In one embodiment, the processor calculates the minimum window size that causes the maximum memory utilization during ML to remain below the target maximum. The target maximum can be predetermined based on the remaining amount of available memory capacity of the (one or more) cloud containers on which the ML model training is to be performed. In one embodiment, the processor accepts user input of the window size. In one embodiment, the processor retrieves the predetermined window size from a memory or storage device. Then, processing block 315 is completed and method 300 continues at processing block 320.

[0096] At processing block 320, the processor divides the length L of the signal database (e.g., the length of the training set of the signal database) by the window size WS to obtain the number of windows M covering the length L of the signal database. In one embodiment, the processor rounds the resulting number of windows M up to the next integer, thereby not allowing partial windows. In one embodiment, partial windows are allowed. Then, processing block 320 is completed, and method 300 continues at processing block 325.

[0097] At processing block 325, the processor initializes the signal index i to an initial value of 1, and the method 300 proceeds to decision block 330. Decision block 330 is located at the head of the outer while loop for processing each of the N signals in the time series database. At decision block 330, the processor compares the value of the signal index i with the number of signals N to determine whether the signal index i is less than or equal to (i.e., not greater than) the number of signals N. When the signal index i is less than or equal to the number of signals N (330: Yes), the signal is retained for processing, and the method 300 continues at processing block 335.

[0098] At processing block 335, the processor finds the maximum value Max of signal i and the corresponding index of the maximum value. At processing block 340, the processor finds the minimum value Min of signal i and the corresponding index of the minimum value. In one embodiment, processing blocks 225 and 340 can be performed together in a single pass of signal i. In one embodiment, the functions of processing blocks 335 and 340 are performed by extreme value locator 105. In one embodiment, the processor initializes the value of the maximum value Max to a value lower than the range of signal i, and initializes the value of the minimum value Min to a value higher than the range of signal i. For each observation of signal i, the processor compares the signal value v at the observation with the values ​​of Max and Min. When the signal value v exceeds the maximum value Max, the processor updates the value of the maximum value Max to the signal value v, and records the index of v (e.g., timestamp) as the index of the maximum value Max. When the signal value v is less than the minimum value Min, the processor updates the value of the minimum value Min to the signal value v, and records the index of v (e.g., timestamp) as the index of the minimum value Min. In this manner, once each observation of signal i has been processed, the maximum value Max is the global maximum value of signal i, the minimum value Min is the global minimum value of signal i, and the indices of these values ​​are recorded. Method 300 then continues at processing block 345 .

[0099] At processing block 345, the processor initializes the window index j to an initial value of 1, and the method 300 proceeds to decision block 350. Decision block 350 is located at the head of the inner while loop for processing each of the M windows covering the signal. At decision block 350, the processor compares the value of the window index j with the number of windows M to determine whether the window index j is less than or equal to (i.e., not greater than) the number of windows M. When the window index j is less than or equal to the number of windows M (block 350: yes), the window is kept for processing, and the method 300 continues at processing block 355.

[0100] At processing block 355, the processor averages the time steps (observations) in window j. In one embodiment, the functionality of processing block 355 is performed by overall averager 110. In one embodiment, the processor finds the mean of all observations with indices within window j. In one embodiment, the processor finds the median of all observations with indices within window j. In one embodiment, window j is a continuous block of observations between a first index (inclusive) and a second index (exclusive). In one embodiment, the lower end of window j includes observations at the lower end, while the upper end of window j does not include observations at the upper end. In one embodiment, the lower end of window j does not include observations at the lower end, while the upper end of window j includes observations at the upper end. Then, method 300 continues at decision block 360.

[0101] At processing block 360, the processor determines whether the index of the maximum value in signal N is within window j. For example, the processor determines the index of the observation that appears within window j. The processor then compares the index of the observation in window j with the index of the maximum value in signal N. If a match is found, e.g., the index of the maximum value is equal to one of the indices of the observations in window j, then the index of the maximum value is in window j (block 360: yes), and method 300 continues at processing block 365. If no match is found, then the index of the maximum value is not in window j (block 360: no), and method 300 continues at decision block 370.

[0102] At processing block 365, the processor replaces the average value generated at processing block 355 with the maximum value Max. In one embodiment, the processor overwrites the average value with the maximum value Max. In one embodiment, the functions of processing blocks 360 and 365 are performed by extreme value replacer 115. Then, method 300 continues at processing block 380.

[0103] At processing block 360, the processor determines whether the index of the minimum value in signal N is within window j. For example, the processor determines the index of the observation that appears within window j. The processor then compares the index of the observation in window j with the index of the minimum value in signal N. If a match is found, e.g., the index of the minimum value is equal to one of the indices of the observations in window j, then the index of the minimum value is in window j (block 370: yes), and method 300 continues at processing block 375. If no match is found, then the index of the maximum value is not in window j (block 370: no), and method 300 continues at processing block 380.

[0104] At processing block 375, the processor replaces the average value generated at processing block 355 with the minimum value Min. In one embodiment, the processor overwrites the average value with the minimum value Min. In one embodiment, the functions of processing blocks 370 and 375 are performed by extreme value replacer 115. Then, method 300 continues at processing block 380.

[0105] At processing block 380, the processor shifts window j by WS time steps (or observations). In one embodiment, the processor advances window j by its length WS so that it covers the next block of time steps (or observations). When the window is shifted by the length of window WS, the window positions do not overlap. Then, method 300 continues at processing block 385.

[0106] At processing block 385, the processor increments the value of window index j by 1. This indicates that processing has moved to the next overall averaging window. Method 300 then returns to decision block 350. While window index j remains less than or equal to the number of windows M (block 350: yes), the window remains for processing and the inner loop iterates again. When window index j is no longer less than or equal to the number of windows M (block 350: no), no window remains for processing, the inner loop terminates, and method 300 continues at processing block 390.

[0107] At processing block 390, the processor increments the value of signal index i by 1. This indicates that processing has averaged all windows of the signal as a whole and moved on to the next signal in the signal database. The method 300 then returns to decision block 330. While the signal index i remains less than or equal to the number of signals N (block 330: yes), the signal remains for processing and the outer loop iterates again. When the signal index i is no longer less than or equal to the number of signals N (block 330: no), no signal remains for processing and the outer loop terminates. The method 300 continues to end block 395, where processing is complete.

[0108] — Example Effect of Ensemble Averaging on Preserving Extreme Values ​​of a Signal —

[0109] Figure 4 A signal graph 400 showing an example application of an example method of ensemble averaging for preserving extreme values ​​to a time series signal is illustrated. An upper subgraph 403 of example raw, unaveraged signal data is compared to a lower subgraph 450 of example signal data after an embodiment of ensemble averaging for preserving extreme values. The upper subgraph 403 shows raw, unaveraged values ​​405 of an example time series signal 410. The unaveraged values ​​405 are plotted relative to an observation axis 415 and a signal value axis 420. The lower subgraph 450 shows an ensemble average 455 of the unaveraged values ​​405, preserving extreme values, plotted relative to an observation axis 417 and a signal value axis 420.

[0110] The length of the example time series signal is 30 observations (L=30). The observations of the example time series signal 410 are subdivided into six (M=6) overall averaging windows: the first overall averaging window 425, the second overall averaging window 426, the third overall averaging window 427, the fourth overall averaging window 428, the fifth overall averaging window 429, and the sixth overall averaging window 430. The window size of the overall averaging windows 425-430 is 5 (WS=5). Therefore, the overall averaging window includes 5 observations. In this example, the overall averaging windows 425-430 include the lower limit value of their range, but do not include the upper limit value of their range.

[0111] In one embodiment, the ensemble average 455 of the retained extrema constitutes the value of the example ensemble averaged signal 460 of retained extrema. Because the ensemble averaged signal 460 of retained extrema consists of the average over an ensemble average window of length 5, the example ensemble averaged signal 460 of retained extrema has one observation for every five observations in the example time series signal 410. Thus, in this example, the ensemble averaged signal 460 will have a length of six observations (L A =6). More generally, in one embodiment, the extremum-preserving ensemble-averaged signal has one observation for every WS observations in the raw, un-averaged time series signal. In one embodiment, the observations of the extremum-preserving ensemble-averaged signal correspond to ensemble-averaged windows of the raw, un-averaged time series signal. In one embodiment, the order of the observations in the extremum-preserving ensemble-averaged signal is the order of the ensemble-averaged windows of the raw, un-averaged time series signal.

[0112] The maximum value 435 of the example time series signal 410 has a value of 3.75 and occurs at observation 7. The minimum value 440 of the example time series signal 410 has a value of 1 and occurs at observation 14. As discussed above, the average value of the raw signal value within the ensemble averaging window becomes the value corresponding to the ensemble averaging window in the ensemble averaged signal retaining the extreme value, unless an extreme value (maximum value or minimum value) occurs within the ensemble averaging window. When an extreme value occurs within the ensemble averaging window, the value of the extreme value becomes the value corresponding to the ensemble averaging window in the ensemble averaged signal retaining the extreme value.

[0113] Therefore, in one embodiment, the mean of the ensemble averaging window is placed in the ensemble averaging signal retaining extreme values ​​at a location where no extreme value appears in the ensemble averaging window. For example, the mean of the unaveraged value 405 in the values ​​in the first ensemble averaging window 425 (the first window mean 445) is 2. Since neither the maximum value 435 nor the minimum value 440 appears in the first ensemble averaging window 425, the first window mean 445 is placed in the ensemble averaging signal 460 retaining the example extreme values ​​at a location corresponding to the first ensemble averaging window 425.

[0114] In one embodiment, the maximum value of the time series signal is placed where the maximum value of the ensemble averaged signal retaining extreme values ​​appears within the ensemble average window. For example, the average value (second window average value 446) of the unaveraged values ​​405 in the values ​​in the second ensemble averaged window 426 is 3. Because the maximum value 435 appears at observation 7, within the second ensemble averaged window 426, the second window average value 446 is replaced by the value of the maximum value 435 (3.75), as shown at the maximum value 465. The value of the maximum value 435 is placed in the example ensemble averaged signal retaining extreme values ​​460 at a position corresponding to the second ensemble averaged window 426. Therefore, in one embodiment, the maximum value 465 is placed in the ensemble averaged signal 460 at a time position corresponding in time to the time position of the maximum value 435 in the example time series signal 410. In other words, in one embodiment, the value of the maximum value is placed in the averaged training signal at the position of the maximum value in the training signal.

[0115] In one embodiment, the minimum value of the time series signal is placed at the location in the ensemble averaged signal of the retained extreme values ​​where the minimum value appears within the ensemble average window. For example, the mean of the unaveraged value 405 in the values ​​in the third ensemble averaged window 427 (third window mean 447) is 2. Because the minimum value 440 appears at observation 14, within the third ensemble averaged window 427, the third window mean 447 is replaced by the value (1) of the minimum value 440, as shown at the minimum value 470. The value of the minimum value 440 is placed at the location corresponding to the third ensemble averaged window 427 in the example ensemble averaged signal of the retained extreme values ​​460. Therefore, in one embodiment, the minimum value 470 is placed at the time location in the ensemble averaged signal 460 that corresponds in time to the time location of the minimum value 440 in the example time series 410. In other words, in one embodiment, the value of the minimum value is placed at the location of the minimum value in the training signal in the averaged training signal.

[0116] — Sample Test Data —

[0117] In order to illustrate the overall averaging system and method for retaining extreme values ​​as shown and described herein, an example time series database including 20 example time series signals is generated. In this example, the example time series signal is synthesized. In this example, the initial step of synthesis is to generate 20 different synthetic waves composed of 3 sine waves with periods of 199, 383 and 547. These periods provide known (basic facts) correlations and patterns in individual time series signals. In one example, 20 example time series signals of the example time series database are synthesized to represent the correct behavior of the example device or device. Therefore, it is known that the 20 example time series signals do not represent abnormal activity or degradation. Then random measurement noise is superimposed on the individual composite signal to generate the example time series signal. The random measurement noise makes the behavior of the example time series signal very similar to the actual sensor signal. The noise ratio (NR) referred to herein indicates the standard deviation of the Gaussian noise added. In the example time series signal, NR is 1.5. The example time series database (and each example time series signal therein) is synthesized to have 5000 observations, sampled once per second. Thus, the example time series signal contains 5000 observations, incremented by 1 second.

[0118] Figure 5 A signal graph 500 of an example time series signal from the above-described example time series database is illustrated. The signal graph 500 presents a comparison of an example time series signal before and after one embodiment of applying ensemble averaging that preserves extreme values. The top subgraph 505 shows raw, unaveraged signal data 510, which is plotted relative to a time axis 515 and a signal value axis 520. The raw, unaveraged signal data 510 preserves the original, fine-grained details of the example time series signal. The bottom subgraph 525 shows averaged signal data 530 after applying ensemble averaging that preserves extreme values, which is plotted relative to a time axis 515 and a signal value axis 520. The averaged signal data 530 shows a noise-reduced, coarser-grained signal resulting from ensemble averaging that preserves extreme values.

[0119] A portion of the observations in the time series signal can be designated as a training signal. The training signal is used to train the ML model to predict or estimate the behavior of (one or more) component time series signals of the time series signal database. For example, the signal diagram 500 shows a training signal 540 of the raw, unaveraged signal data 510 and the averaged signal data 530. In this example, the first half of the signal data is used as the training signal 540.

[0120] Another portion of the observations in the time series signal can be designated as a monitoring signal and analyzed with the trained ML model to generate predicted or estimated signal values ​​for the component time series signal(s) of the time series signal database. For example, signal diagram 500 shows a monitoring signal 545 of the raw, unaveraged signal data 510 and the averaged signal data 530. In this example, the second half of the signal data is used as the monitoring signal 545.

[0121] In one embodiment, as discussed above, the minimum 550 in the training signal 540 and the maximum 555 in the training signal 540 are identified in the fine-grained, unaveraged signal data 510. In the bottom sub-graph 525, it is observed that most of the averaged signal data 530 have a lower noise level than the unaveraged signal data 510. This is due to the overall averaging filter. However, in the training signal 540, the fine-grained extreme values ​​still exist: the maximum value in the training signal 540 remains at the maximum point 560 in the averaged signal data 530, and the minimum value in the training signal 540 remains at the minimum point 565 in the averaged signal data 530.

[0122] — Improved experimental validation —

[0123] In one embodiment, extreme values ​​are retained for the training signal portion of the time series signal database, but not for the monitoring signal portion of the time series database. In one embodiment, the monitoring set of the time series database is averaged as a whole without retaining extreme values. Therefore, extreme values ​​are not retained in the monitoring signal 545 of the averaged signal data 530.

[0124] A set of baseline results was established. A Multivariate State Estimation Technique (MSET) model was generated using 20 example time series signals. The model was trained on the first half of the data (first 2500 seconds) and then deployed to monitor the second half of the data (2501 to 5000 seconds).

[0125] A second set of results of applying an embodiment of the system and method for retaining the overall averaging of extreme values ​​to an example time series database of 20 example time series signals is also obtained to verify the improvement. The 20 example time series signals are used to generate the MSET model after first performing the overall averaging of the retained extreme values. In this example, 5 is selected as the window size (WS). After the overall averaging process of retaining the extreme values, the 20 example time series signals now have signals with a sampling interval of 5 seconds, thereby reducing the observations from a total of 5000 to a total of 1000. The MSET model is trained on the first half of the data (the first 500 observations cover the first 2500 seconds) and then deployed to monitor the second half of the data (the 501st observation to the 1000th observation, covering the 2501st second to the 5000th second).

[0126] Comparison of the results reveals that, in one embodiment, the memory footprint and computational cost of training the MSET model and monitoring are reduced by an ensemble averaging system and method that retains extreme values. This reduction in memory footprint and computational cost is due to a reduction in the number of observations in the example time series database.

[0127] Comparison of the results also reveals additional benefits: In one embodiment, the ensemble averaging system and method that retains extreme values ​​significantly reduces the false positive probability (FAP). When the MSET model is trained on and monitors the raw signals of the example time series database, several sequential probability ratio test (SPRT) alarms appear. Because the raw signals are known as ground truth to represent correct behavior and not abnormal activity, these SPRT alarms are false positives. However, in one embodiment, when the MSET model is trained on and monitors the ensemble averaged signal that retains extreme values, the SPRT alarm count, and thus the FAP, drops to near zero.

[0128] The following metrics summarize the results for the example time series database. The metrics tracked allow for a reasonable comparison of the performance between models trained and monitored on the raw signal and models trained and monitored on the ensemble averaged signal with extreme values ​​preserved. Alerts and empirical FAP are tracked to compare the predictive performance of the models, while kurtosis, skewness, and residual mean between the signal values ​​and the values ​​predicted by the model are tracked as proxies for accuracy.

[0129] As discussed above, the signals of the time series database do not include any anomalies or degradations and have a noise ratio of 1.5. To facilitate viewing a snapshot of the results, the metrics are averaged over all 20 signals in the example time series database. For the MSET model using the unaltered (raw signals) dataset, the average alert count was 7.8 alerts, the average empirical FAP was 0.0031, the average kurtosis was 6.09, the average absolute skewness was 0.290, and the average absolute mean was 0.061. For the MSET model using the ensemble averaged dataset of the example retained extreme values, the average alert count was 1.0, the average empirical FAP was 0.0020, the average kurtosis was 4.10, the average absolute skewness was 0.235, and the average absolute mean was 0.050.

[0130] In addition to an approximate 33% improvement in the mean empirical FAP and kurtosis of the residuals, the results of the MSET model using the example extremum-retaining ensemble averaging data set show a significant improvement in the average alarm count. The performance gains of the processed signal of the example extremum-retaining ensemble averaging in the remaining metrics (skewness and bias of the residuals) are smaller, but still meaningful. Using data processed according to an embodiment of ensemble averaging with retained extrema as described herein, there is no deterioration in the metrics reflecting the effectiveness of anomaly detection. Instead, all metrics are improved. Therefore, in one embodiment, by implementing ensemble averaging with retained extrema as described herein, both the predictive performance and accuracy of ML anomaly detection are improved.

[0131] Further analysis determined how different training schemes affect MSET and its ability to detect degradation. Analysis was performed to evaluate the impact of the ensemble averaging system and method that retains extreme values ​​on anomaly detection. The effect was illustrated by injecting synthetic faults into one or more of the time series signals in the example database. This technique demonstrated the effectiveness of anomaly detection because when the fault is injected into the test signal database, there is ground truth knowledge about the presence of the fault, which is used to quantify the probability of false negative (MAP).

[0132] To simulate a slow drift in the signal, a small ramp was injected into the example time series signal. The ramp simulates a common sensor degradation pattern known as linear decalibration bias. The ramp starts at time step 3750 of the example time series signal and ends at time step 5000. Therefore, the simulated degradation is entirely within the monitored portion of the signal. After the degradation is inserted into the signal database, the MSET model is trained on the first half of the raw signal and then used to monitor the second half of the raw signal. Furthermore, after the degradation is inserted into the signal database, the MSET model is trained on the ensemble average of the retained extreme values ​​in the first half of the signal and then used to monitor the ensemble average of the second half of the signal.

[0133] The following metrics summarize the results for the example time series database. One metric that is useful for determining the detection of performance degradation is the time to detection (TTD). Another metric, alert density, is an indicator of the uncertainty in detecting anomalies. Comparing the TTD of the two models shows that the MSET model generated from the unchanged data slightly outperforms the MSET model generated from the data processed by ensemble averaging with extreme values. (This is due to the very slight delay introduced by the average of the values ​​in the ensemble averaging window being kept incomplete until all values ​​in the ensemble are available.) But more importantly, the uncertainty in detecting anomalies is significantly reduced in the ensemble averaged results, as is evident by the higher alert density starting approximately at observation 900 (approximately 4500 seconds after the ensemble averaging process with extreme values ​​in the data set).

[0134] Clearly, in one embodiment, the extremum-preserving ensemble averaging approach reduces FAP, improves model accuracy, and reduces detection uncertainty while only slightly changing detection time. These improvements are a direct result of combining extremum preservation and ensemble averaging of adjacent time steps.

[0135] Extreme value preservation reduces false positives caused by statistical noise variations between the training range and the test range of signal values. When using raw signal data, the training measurements can cover all relevant operating modes to produce quality estimates within a reasonable range for the device or device, but the test range can still exceed the training range boundaries due to noise in the signal. In this case, false positives are likely to be triggered. However, by preserving the global extreme values ​​of the signal during training, the possibility that the test range will exceed the training range and trigger a false positive is eliminated.

[0136] The second reason for the reduction in FAP is a product of ensemble averaging. By averaging adjacent values, the resulting signal exhibits an increased signal-to-noise ratio without changing the time dependence of the measurement. Moreover, as mentioned earlier, ensemble averaging also reduces the sampling rate, which reduces the memory consumption and computational cost of the ML analysis.

[0137] The reduction in memory usage and computational cost achieved by one embodiment of ensemble averaging with preserved extrema is empirically validated. The first experiment tracks the maximum memory usage as the window size WS of the ensemble averaging window increases during the training phase of MSET. It is shown that the memory consumption decreases monotonically as the ensemble averaging window size increases. This suggests that memory consumption will always decrease as the ensemble averaging window increases. Furthermore, the reduction in memory footprint decreases cubically, so the benefits of using ensemble averaging with preserved extrema are quickly realized.

[0138] The second experiment tracks the time to complete the task as the overall average window size WS increases during the training phase of MSET. The time to complete the task represents the computational cost. It shows that the time to complete the task (and therefore the computational cost) decreases superlinearly as the overall average window size increases.

[0139] The improvements discussed herein are achieved using an equivalent (or identical) baseline computer configuration without changing the hardware. Thus, both computer performance and ML anomaly detection are improved by the systems and methods described herein. These improvements are not caused by brute force application of computing power, but rather by the innovative systems and methods for holistic averaging that preserve extreme values ​​shown herein.

[0140] For illustration purposes, MSET is used herein as an example ML anomaly detection model. In one embodiment, the ensemble averaging system and method for preserving extreme values ​​shown and described herein similarly improves nonlinear nonparametric (NLNP) regression algorithms for multivariate anomaly detection. Such NLNP regression algorithms include neural networks (NN), support vector machines (SVM), autoassociated kernel regression (AAKR), and similarity-based modeling (SBM), such as multivariate state estimation technology (MSET) (including Oracle's proprietary multivariate state estimation technology (MSET2)).

[0141] — Cloud or Enterprise Implementation —

[0142] In one embodiment, the system (such as the global averaging system 100 retaining extreme values) is a computing / data processing system that includes a collection of computing applications or distributed computing applications for access and use by other client computing devices that communicate with the system over a network. In one embodiment, the global averaging system 100 retaining extreme values ​​is a component of a time series data service that is configured to collect, provide, and perform operations on time series data. The applications and computing systems can be configured to operate with or be implemented as a cloud-based network computing system, infrastructure as a service (IAAS), platform as a service (PAAS), or software as a service (SAAS) architecture or other types of networked computing solutions. In one embodiment, the system provides at least one or more of the functions disclosed herein and a graphical user interface for accessing and operating the functions. In one embodiment, the ensemble averaging system 100 preserving extrema is a centralized server-side application that provides at least the functionality disclosed herein and is accessed by many users via computing devices / terminals that communicate with computers of the ensemble averaging system 100 preserving extrema (acting as one or more servers) over a computer network.

[0143] In one embodiment, the components of the extreme value-preserving overall averaging system 100 communicate with each other through electronic messages or signals. These electronic messages or signals can be configured as calls to functions or processes that access the features or data of the components, such as, for example, application programming interface (API) calls. In one embodiment, these electronic messages or signals are sent between hosts in a format compatible with the Transmission Control Protocol / Internet Protocol (TCP / IP) or other computer networking protocols. The components of the system 100 can (i) generate or write electronic messages or signals to issue commands or requests to another component, (ii) transmit the message or signal to other components of the computing system 100, (iii) parse the content of the received electronic message or signal to identify the command or request that the component can execute, and (iv) automatically perform or execute the command or request in response to identifying the command or request. The electronic message or signal can include a query for a database. The query can be written and executed in a query language compatible with the database and executed in a runtime environment compatible with the query language.

[0144] In one embodiment, a remote computing system can access information or applications provided by the ensemble averaging system 100 for preserving extreme values, such as through a web interface server. In one embodiment, a remote computing system can send requests to the ensemble averaging system 100 for preserving extreme values ​​and receive responses therefrom. In one example, access to information or applications can be achieved by using a web browser on a personal computer or mobile device. In one example, messages exchanged with the ensemble averaging system 100 for preserving extreme values ​​can take the form of remote representational state transfer (REST) ​​requests, such as using JavaScript Object Notation (JSON) as a data exchange format, or sending simple object access protocol (SOAP) requests to and from an XML server. REST or SOAP requests can include API calls to components of the ensemble averaging system 100 for preserving extreme values.

[0145] —Computing Device Embodiment—

[0146] Figure 6An example computing system 600 is illustrated, configured and / or programmed as a special purpose computing device having one or more of the example systems and methods described herein and / or equivalents. In one embodiment, the example computing system 600 includes an example computer or computing device 605. The example computing device 605 may include at least one hardware processor 610, a memory 615, and an input / output port 620, which are operably connected via a bus 625. In one example, the computer 605 may include ensemble averaging logic 630 that retains extreme values, which is configured to facilitate retaining signal extreme values ​​for ML model training when ensemble averaging a time series signal for ML anomaly detection, similar to reference Figure 1-Figure 5 The logic, systems, and methods shown and described.

[0147] In different examples, the logic 630 can be implemented in hardware, a non-transitory computer-readable medium 637 having stored instructions, firmware, and / or a combination thereof. Although the logic 630 is shown as a hardware component attached to the bus 625, it should be appreciated that in other embodiments, the logic 630 can be implemented in the processor 610, stored in the memory 615, or stored in the disk 635.

[0148] In one embodiment, logic 630 or computer is a means (i.e., structure: hardware, non-transitory computer-readable medium, firmware) for performing the described actions. In some embodiments, computing device 605 may be a server operating in a cloud computing system, a server configured in a software as a service (SaaS) architecture, a smart phone, a laptop computer, a tablet computing device, etc.

[0149] For example, the component can be implemented as an ASIC programmed to facilitate retaining signal extreme values ​​for ML model training when ensemble averaging time series signals for ML anomaly detection. The component can also be implemented as stored computer executable instructions that are presented to the computer 605 as data 640, which are temporarily stored in the memory 615 and then executed by the processor 610.

[0150] The logic 630 may also provide means (e.g., hardware, non-transitory computer-readable medium storing executable instructions, firmware) for retaining signal extreme values ​​for ML model training when ensemble averaging the time series signal for ML anomaly detection.

[0151] Generally describing an example configuration of computer 605, processor 610 can be a variety of processors, including dual microprocessors and other multi-processor architectures. Memory 615 can include volatile memory and / or non-volatile memory. Non-volatile memory can include, for example, ROM, PROM, etc. Volatile memory can include, for example, RAM, SRAM, DRAM, etc.

[0152] The storage device or disk 635 can be operably connected to the computer 605 via an input / output (I / O) interface (e.g., card, device) 645 and an input / output port 620, for example, controlled by at least one input / output (I / O) controller 647. The storage device or disk 635 can be, for example, a magnetic disk drive, a solid state drive, a floppy disk drive, a tape drive, a Zip drive, a flash memory card, a memory stick, etc. In addition, the storage device or disk 635 can be a CD-ROM drive, a CD-R drive, a CD-RW drive, a DVD ROM, etc. The memory 615 can store, for example, a process 650 (such as method 200) and / or data 640. The storage device or disk 635 and / or the memory 615 can store an operating system that controls and allocates resources of the computer 605.

[0153] The computer 605 can interact with, control, and / or be controlled by input / output (I / O) devices via an input / output (I / O) controller 647, an I / O interface 645, and an input / output port 620. The input / output devices may include, for example, one or more displays 670, a printer 672 (such as an inkjet printer, a laser printer, or a 3D printer), an audio output device 674 (such as a speaker or a headset), a text input device 680 (such as a keyboard), a cursor control device 682 (such as a mouse, a trackball, a touch screen, a joystick, a pointing stick, an electronic stylus, an electronic handwriting tablet) for pointing and selecting input, an audio input device 684 (such as a microphone or an external audio player), a video input device 686 (such as a video camera and a still camera, or an external video player), an image scanner 688, a video card (not shown), a disk 635, a network device 660, and the like. The input / output port 620 may include, for example, a serial port, a parallel port, and a USB port.

[0154] The computer 605 can operate in a network environment and therefore can be connected to a network device 655 via the I / O interface 645 and / or the I / O port 620. Through the network device 655, the computer 605 can interact with (one or more) networks 660. Through the network 660, the computer 605 can be logically connected to a remote computer 665. The networks that the computer 605 can interact with include, but are not limited to, LANs, WANs, and other networks.

[0155] In one embodiment, a computer may be connected to a sensor 690 via an I / O port 620 or a network 660 to receive information about the physical state of a monitored machine, equipment, system, facility, or other device (which may be collectively referred to as an "asset"). In one embodiment, the sensor 690 is configured to monitor physical phenomena occurring in or around an asset. An asset generally includes any type of machine or facility having components that perform a measurable activity. In one embodiment, the sensor 690 may be operably connected to or secured to an asset, or otherwise configured to detect and monitor physical phenomena occurring in or around an asset. The sensor 690 may be a network-connected sensor for monitoring any type of physical phenomenon. The network connection between the sensor 690 and the network 660 may be wired or wireless. Sensor 690 may include, but is not limited to, voltage sensors, current sensors, temperature sensors, pressure sensors, scales or other weight sensors, rotational speed sensors, angle sensors, distance sensors, displacement sensors, thermometers, flow meter sensors, vibration sensors, microphones, light sensors, electromagnetic radiation sensors, proximity sensors, occupancy sensors, motion sensors, gyroscopes, inclinometers, accelerometers, shock sensors, global positioning system (GPS) sensors, torque sensors, flexure sensors, humidity monitors, liquid level sensors, electronic noses, nuclear radiation detectors, or various other sensors or transducers for generating telemetry (electrical signals describing detected or sensed physical behavior).

[0156] In one embodiment, the computer 605 is configured with logic, such as a software module, to collect readings from the sensors 690 and store them as observations in a time series data structure, such as a time series database. In one embodiment, the computer 605 polls the sensors 690 to retrieve the sensor telemetry readings. In one embodiment, the computer 690 passively receives the sensor telemetry readings actively transmitted by the sensors 690. In one embodiment, the computer 605 receives one or more databases of previously collected sensor 690 observations, such as from the storage device 635 or from the remote computer 665.

[0157] —Definitions and Other Embodiments—

[0158] Any action or function described or claimed herein is not performed by the human mind. Any interpretation that an action or function can be performed by the human mind is inconsistent and contrary to the present disclosure.

[0159] In another embodiment, the described methods and / or their equivalents may be implemented with computer executable instructions. Thus, in one embodiment, a non-transitory computer readable / storage medium is configured with computer executable instructions having stored algorithms / executable applications that, when executed by (one or more) machines, cause the (one or more) machines (and / or associated components) to perform the described methods. Example machines include, but are not limited to, processors, computers, servers operating in a cloud computing system, servers configured with a software as a service (SaaS) architecture, smart phones, and the like. In one embodiment, a computing device is implemented with one or more executable algorithms configured to perform any of the disclosed methods.

[0160] In one or more embodiments, the disclosed methods or their equivalents are performed by any of the following: computer hardware configured to perform the methods; or, computer instructions embodied in a module stored in a non-transitory computer-readable medium, wherein the instructions are configured as an executable algorithm, which is configured to perform the methods when executed by at least one processor of a computing device.

[0161] In one embodiment, each step of the computer-implemented method described herein may be performed by a processor of one or more computing devices that (i) accesses a memory and (ii) is configured with logic that causes the system to perform the steps of the method. For example, the processor accesses and reads from or writes to the memory to perform the steps of the computer-implemented method described herein. These steps may include (i) retrieving any necessary information, (ii) calculating, determining, generating, classifying, or otherwise creating any data, and (iii) storing any calculated, determined, generated, classified, or otherwise created data for subsequent use. References to storage or storing indicate storage as a data structure in a memory or storage device / disk of a computing device.

[0162] In one embodiment, each subsequent step of the method is automatically initiated in response to parsing a received signal or retrieved stored data indicating that the previous step has been performed at least to the extent required for the subsequent step to begin. Generally speaking, the received signal or retrieved stored data indicates the completion of the previous step.

[0163] Although for the purpose of simplifying the description, the method illustrated in the figure is shown and described as a series of square blocks of the algorithm, it should be appreciated that these methods are not limited by the order of the square blocks. Some square blocks may appear in a different order than shown and described and / or appear simultaneously with other square blocks. Moreover, the example method can be implemented using square blocks less than all the illustrated square blocks. Square blocks can be combined or divided into multiple actions / components. In addition, additional and / or alternative methods can adopt additional actions not illustrated in the square blocks.

[0164] The following includes definitions of selected terms used herein. The definitions include various examples and / or forms of components that fall within the scope of the terms and can be used to implement. The examples are not intended to be limiting. Both singular and plural forms of the terms may be within the definitions.

[0165] References to "one embodiment," "an embodiment," "an example," "an example," etc. indicate that the embodiment(s) or examples described herein may include particular features, structures, characteristics, properties, elements, or limitations, but not every embodiment or example must include the particular features, structures, characteristics, properties, elements, or limitations. Furthermore, repeated use of the phrase "in one embodiment" does not necessarily refer to the same embodiment, but may.

[0166] As used herein, a "data structure" is an organization of data stored in a memory, storage device, or other computerized system in a computing system. A data structure can be, for example, any of a data field, a data file, a data array, a data record, a database, a data table, a graph, a tree, a linked list, etc. A data structure can be formed by and include many other data structures (e.g., a database includes many data records). Other examples of data structures are also possible according to other embodiments.

[0167] As used herein, "computer-readable medium" or "computer storage medium" refers to a non-transient medium that stores instructions and / or data configured to perform one or more functions of the disclosed functions when executed. In some embodiments, data can be used as instructions. Computer-readable media can take the form of, but not limited to, non-volatile media and volatile media. Non-volatile media can include, for example, optical disks, magnetic disks, etc. Volatile media can include, for example, semiconductor memories, dynamic memories, etc. Common forms of computer-readable media can include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, other magnetic media, application-specific integrated circuits (ASICs), programmable logic devices, compact disks (CDs), other optical media, random access memories (RAMs), read-only memories (ROMs), memory chips or cards, memory sticks, solid-state storage devices (SSDs), flash drives, and other media that computers, processors, or other electronic devices can work with. If each type of media is selected for implementation in one embodiment, it can include storage instructions for algorithms configured to perform one or more functions of the disclosed and / or claimed functions.

[0168] As used herein, "logic" means a component implemented using computer or electrical hardware, a non-transient medium with instructions of executable applications or program modules stored, and / or a combination of these, to perform any function or action as disclosed herein, and / or to cause a function or action from another logic, method, and / or system to be performed as disclosed herein. Equivalent logic may include firmware, a microprocessor programmed with an algorithm, discrete logic (e.g., ASIC), at least one circuit, an analog circuit, a digital circuit, a programmed logic device, a memory device containing instructions of the algorithm, etc., any of which may be configured to perform one or more of the disclosed functions. In one embodiment, logic may include one or more gates, a combination of gates, or other circuit components that are configured to perform one or more of the disclosed functions. In the case of describing multiple logics, it is possible to merge multiple logics into one logic. Similarly, in the case of describing a single logic, it is possible to distribute that single logic between multiple logics. In one embodiment, one or more of these logics are corresponding structures associated with performing the disclosed and / or claimed functions. Choosing which type of logic to implement can be based on desired system conditions or specifications. For example, if higher speed is considered, hardware will be selected to implement the function. If lower cost is a consideration, stored instructions / executable applications will be chosen to implement the functionality.

[0169] An "operable connection" or a connection through which entities are "operably connected" is a connection through which signals, physical communications, and / or logical communications can be sent and / or received. An operable connection may include a physical interface, an electrical interface, and / or a data interface. An operable connection may include different combinations of interfaces and / or connections sufficient to allow operable control. For example, two entities may be operably connected to transmit signals to each other directly or through one or more intermediate entities (e.g., a processor, an operating system, logic, a non-transient computer-readable medium). Logical and / or physical communication channels may be used to create an operable connection.

[0170] As used herein, a "user" includes, but is not limited to, one or more persons, computers or other devices, or a combination of these.

[0171] Although the disclosed embodiments have been illustrated and described in considerable detail, it is not intended that the scope of the appended claims be limited or in any way restricted to such details. Of course, it is not possible to describe every expected combination of components or methods for the purpose of describing various aspects of the subject matter. Therefore, the present disclosure is not limited to the specific details or illustrative examples shown and described. Therefore, the present disclosure is intended to cover changes, modifications and variations that fall within the scope of the appended claims.

[0172] To the extent that the term "comprising" is employed in the detailed description or claims, it is intended to be inclusive in a manner similar to how the term "comprises" is interpreted when employed as a transitional word in a claim.

[0173] To the extent the term "or" is employed in the detailed description or claims (e.g., A or B), it is intended to mean "A or B or both." When the applicants intend to indicate "only A or B but not both," then the phrase "only A or B but not both" will be used. Thus, the use of the term "or" herein is inclusive, rather than exclusive.

Claims

1. A computer-implemented method comprising: Identify the locations and values ​​of extreme values ​​in the training signal; ensemble averaging the training signal to generate an averaged training signal; placing the values ​​of the extreme values ​​into the averaged training signal at corresponding positions of the extreme values ​​to generate an averaged training signal that retains the extreme values; as well as A machine learning model is trained with an averaged training signal that retains extreme values ​​to facilitate detecting one or more anomalies in one or more monitoring signals from one or more sensors so as to detect incipient or actual failures of a device monitored by the one or more sensors.

2. The computer-implemented method of claim 1 , wherein placing the values ​​of the extrema at the corresponding positions of the extrema into the averaged training signal further comprises: Determine the ensemble averaging window in which one of the extreme values ​​occurs; as well as The ensemble-averaged value corresponding to the ensemble-averaging window is replaced with the value of the one of the extrema.

3. The computer-implemented method of any one of claims 1-2, wherein ensemble averaging the training signal further comprises: Choose the length of the ensemble averaging window; determining the number of ensemble averaging windows to cover the length of the training signal; as well as For the number of ensemble averaging windows, Averages the signal values ​​within the ensemble averaging window to create an averaged signal value, append the averaged signal value to the averaged training signal, and Shifts the ensemble averaging window by the length of the ensemble averaging window.

4. The computer-implemented method of any one of claims 1 to 3, further comprising: ensemble averaging the monitoring signals belonging to the one or more monitoring signals to produce an averaged monitoring signal; as well as The one or more anomalies in the averaged monitoring signal are monitored using the trained machine learning model.

5. The computer-implemented method of claim 4, further comprising receiving a monitoring data stream from a sensor as one or more monitoring signals in a real-time stream, wherein the monitoring signals are overall averaged as they arrive.

6. The computer-implemented method of any of claims 1-3, further comprising detecting an incipient or actual failure of a device based on detection of the one or more anomalies in the one or more monitoring signals from the one or more sensors.

7. The computer-implemented method of claim 6, wherein detecting one or more anomalies in the one or more monitoring signals comprises: ensemble averaging the monitoring signals of the one or more monitoring signals to generate an averaged monitoring signal; executing the trained machine learning model to estimate a value of an average monitoring signal; calculating the residual between the observed value of the averaged supervisory signal and the value estimated by the machine learning model of the averaged supervisory signal; as well as The residuals are analyzed to detect at least one anomaly.

8. The computer-implemented method of any one of claims 1 to 7, further comprising: In a first pass the values ​​of the training signal are parsed to identify the locations and values ​​of the extrema; as well as The values ​​of the training signal are resolved in a second pass to perform an overall average of the training signal.

9. A computer-implemented method as described in any of claims 1-8, wherein the machine learning model is a multivariable state estimation technology model.

10. The computer-implemented method of any of claims 1-9, further comprising taking an action in response to detecting an incipient or actual failure of a monitored device, wherein the action is at least one of: issuing an alarm or taking corrective action.

11. A computer-implemented method as described in any of claims 1-10, wherein the monitored device is one or a portion thereof: a computer, a server installation, a disk drive, a tape drive, a data center, a cooling system, an aircraft, a ship or vessel, a land vehicle, an industrial installation, a utility installation, a refinery, a motor, an engine, a turbine, a pump, a machine tool, a production line, or a robot.

12. One or more non-transitory computer-readable media having computer-executable instructions stored thereon, wherein the computer-executable instructions are configured to cause one or more computers to perform operations including the operations described in any one of claims 1-10 when executing the computer-executable instructions.

13. A computing system comprising one or more computers, wherein the computing system is configured by computer-executable instructions to perform operations including those recited in any one of claims 1-10.

14. A computer program product, the computer program product comprising a computer program which, when executed by at least one processor of a computer, causes the processor to perform operations including the operations of any one of claims 1-10.

15. A computing system comprising: at least one processor; at least one memory operably connected to the processor; as well as A non-transitory computer-readable medium comprising instructions stored thereon that, when executed by at least a processor, cause a computing system to: Identify the location and value of the minimum value and the location and value of the maximum value of the training signal, and retain the minimum value and the maximum value; Performing overall averaging on the training signals in the training set to generate an averaged training signal, wherein the overall averaging eliminates the minimum and maximum values ​​in the averaged training signal; placing the minimum value in the averaged training signal at the position of the minimum value and placing the maximum value in the averaged signal at the position of the maximum value to generate an averaged training signal that retains the extreme values; as well as The machine learning model is trained using the averaged training signal with the extreme values ​​retained to detect anomalies in the signal.

Images