Method and system for unsupervised anomaly detection and causal explanation by majority voting on high-dimensional sensor data

Abstract

To provide a system for detecting anomaly in high-dimensional sensor data associated with one or more machines.SOLUTION: During operation, a system is configured to: obtain sensor data from a set of sensors associated with one or more machines; apply data exploration techniques to the sensor data to automatically process the sensor data in order to identify a subset of feature sensors from the available set of feature sensors; apply an unsupervised machine-learning technique to the identified subset of feature sensors and a target sensor to learn a set of pair-wise univariate models; and determine whether an anomaly occurs in operation of the one or more machines on the basis of the set of pair-wise univariate models, and determine how the anomaly occurs.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates generally to systems and methods for anomaly detection in the operation of industrial machines or systems, and more particularly to detecting anomalies in high-dimensional sensor data. [Background technology]

[0002] The increasing use of Internet of Things (IoT)-enabled devices in many industrial applications, such as digital manufacturing, has resulted in a large amount of readily available sensor data. Such sensor data contains actionable information that can be used to optimize the operational efficiency of factory machines. In particular, detecting operational anomalies is one of the most sought-after objectives for its immediate application in reducing factory downtime and improving productivity and efficiency in manufacturing processes. However, developing anomaly detection techniques that can reliably scale to various factory settings remains challenging. This is because sensor data may be poorly calibrated due to poor data quality and a lack of label information. For example, sensor data may contain numerous missing, corrupted, noisy, and highly correlated values.

[0003] Many real-world industrial IoT applications perform suboptimally due to the untuned nature of the available sensor data. Without proper data preprocessing and model selection, parameter estimation is prone to large biases and distortions that result in significant errors in anomaly detection. To avoid such errors, a data preprocessing stage is often required to select, filter, and resample the sensor data.

[0004] In conventional anomaly detection approaches for industrial IoT applications, such data preprocessing is performed using traditional domain knowledge of sensor data and often without significant automation. Due to the difficulty of acquiring labeled sensor data and the lack of fully automated anomaly detection methods, conventional best-of-breed approaches provide unreliable anomaly detection for different datasets. Furthermore, labeled datasets are rarely available or expensive to acquire. In other words, there is no a priori knowledge of which points are normal or abnormal. Therefore, no clear definition of anomaly is given. Unsupervised learning must be performed from available sensor data without normal or abnormal examples. Such unsupervised learning can impose significant constraints on algorithm development because an inherent performance trade-off exists between precision and recall (e.g., sensitivity), potentially reducing the reliability of anomaly detection for different datasets. To adequately scale anomaly detection methods for practical applications, the anomaly analysis workflow needs to be automated for data preprocessing, model selection, and anomaly detection. Summary of the Invention

[0005] According to one embodiment of the present invention, there is provided a system for detecting anomalies in high-dimensional sensor data associated with one or more machines. In operation, the system acquires sensor data from a set of sensors associated with the machines, the set of sensors including a target sensor and a set of feature sensors, applies data mining techniques to the sensor data to identify a subset of feature sensors from the set of feature sensors, and applies unsupervised machine learning techniques to the identified subset of feature sensors and the target sensor to generate a set of pairwise univariate models. (pairwise univariate model) can be learned to determine whether and how anomalies occur in the operation of one or more machines based on a set of pairwise univariate models.

[0006] In variations on this embodiment, applying data exploration techniques may include one or more of applying data preprocessing techniques, applying data cleansing techniques, and applying feature engineering techniques.

[0007] In a variation of this embodiment, applying the data cleansing technique includes replacing missing values ​​in the sensor data by linear interpolation if the amount of missing values ​​during the recording period of the target sensor data is less than a threshold, and discarding missing values ​​in the sensor data if the amount of missing values ​​during the recording period of the target sensor data is greater than a threshold.

[0008] In a variation of this embodiment, applying the feature engineering technique includes adjusting a set of time delays between the set of feature sensors and the target sensor, calculating correlations between the delay-adjusted set of feature sensors and the target sensor, and ranking the set of feature sensors based on the correlation values.

[0009] In a further variation, each pairwise univariate anomaly model from a set of pairwise univariate anomaly models may be associated with a feature sensor and a target sensor from a subset of feature sensors.

[0010] In a further variation of this embodiment, the unsupervised machine learning technique includes determining a set of regression coefficients to measure the fit of the set of pairwise univariate anomaly models, and calculating a set of anomaly distances between the subset of feature sensors and the target sensor using the set of regression coefficients.

[0011] In a further variation, the unsupervised machine learning technique also includes calculating a set of voting scores associated with the set of pairwise univariate anomaly models by applying a set of clustering functions to the set of anomaly distances, calculating a set of weights for the set of voting scores, combining the set of weights and the set of voting scores to detect anomalies, and quantifying the correspondence of the subset of feature sensors to the detected anomalous events. [Brief explanation of the drawings]

[0012] The patent or application file contains at least one drawing executed in color. Copies of any color drawing(s) of this patent or patent application publication will be provided by the Office upon request and payment of the necessary fee.

[0013] [Figure 1A] 1 shows a plot of an exemplary set of recorded sensor data for multiple feature metrics over a period of time, according to one embodiment of the present invention. [Figure 1B] 4 illustrates a plot of an exemplary set of target sensor data and feature sensor data recorded during a first period of time, according to one embodiment of the present invention. [Figure 1C] 10 shows a plot of an exemplary set of target sensor data and feature sensor data recorded during a second time period, according to one embodiment of the present invention. [Figure 2] 1 shows a flowchart illustrating an exemplary process for performing data exploration on sensor data, according to one embodiment of the present invention. [Figure 3] 1 shows a flowchart illustrating an exemplary process for performing data pre-processing, according to an embodiment of the present invention. [Figure 4] 1 shows a flowchart illustrating an exemplary process for performing data cleansing, according to one embodiment of the present invention. [Figure 5A] 1 illustrates an example of handling missing values ​​in sensor data according to an embodiment of the present invention. [Figure 5B] 10 illustrates the results of performing a moving average on sensor data, according to one embodiment of the present invention. [Figure 6] 1 shows a flowchart illustrating an exemplary process for performing feature engineering, according to one embodiment of the present invention. [Figure 7A] 10 illustrates a visualization of delay-adjusted features according to one embodiment of the present invention. [Figure 7B] 1 illustrates an exemplary data correlation map, according to one embodiment of the present invention. [Figure 8] 1 illustrates an exemplary unsupervised anomaly detection system architecture, according to one embodiment of the present invention. [Figure 9] 10 illustrates pseudocode for detecting anomalies according to one embodiment of the present invention. [Figure 10] 1 illustrates exemplary unsupervised anomaly detection results according to one embodiment of the present invention. [Figure 11] 1 shows a flowchart illustrating an exemplary process for performing unsupervised anomaly detection with majority voting, according to one embodiment of the present invention. [Figure 12] 1 illustrates an exemplary computer system that facilitates an unsupervised anomaly detection system, according to one embodiment of the present invention. [Figure 13] 1 illustrates an exemplary apparatus that facilitates an unsupervised anomaly detection system, in accordance with an embodiment of the present invention.

[0014] In the drawings, like reference numbers refer to like drawing elements. DETAILED DESCRIPTION OF THE INVENTION

[0015] The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosure. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0016] overview The embodiments described herein solve a technical problem of detecting and explaining anomalies in the operation of industrial machines. More specifically, an anomaly monitoring and detection system can acquire sensor data from target sensors and feature sensors built into the machine and preprocess the sensor data to address missing values, outliers, and noise. The system then selects only relevant feature sensors for further processing. Specifically, the system uses the relevant feature sensors to build a set of pairwise univariate anomaly models, each model corresponding to a pair of feature sensors and a target sensor. A set of anomaly distances is calculated from the set of pairwise univariate anomaly models. By applying unsupervised machine learning techniques to the set of anomaly distances and then performing a weighted majority voting process, the system can automatically detect anomalies in the operation of the machine and the responsibility of their associated feature sensors.

[0017] Data Exploration In many real-world industrial IoT applications, the sensor data recorded during the operation of one or more machines is numerous and often poorly calibrated. In other words, the collected sensor data may contain missing, corrupted, noisy, and highly correlated values. FIG. 1A shows a plot of an example set of recorded sensor data for multiple feature metrics over a period of time. Dark areas indicate the presence of sensor data, and light areas indicate the absence of sensor data. As can be seen from FIG. 1A, the recorded sensor data contains periodic missing values. When anomaly detection techniques are applied to such recorded sensor data containing a large number of missing values, the detected anomalies may be false.

[0018] In addition to missing values, other defects may also exist in the sensor data. The sensor data may be obtained from multiple sensors built into one or more machines. From the multiple sensors, an anomaly associated with one of the sensors may be of interest, and this sensor may be identified as the target sensor, while the remaining sensors may be identified as feature sensors. FIG. 1B shows a plot of an exemplary set of target sensor data and feature sensor data recorded during a first time period. FIG. 1C shows a plot of target sensor data and feature sensor data recorded during a second time period. From FIGS. 1B and 1C, it is clear that there is significant data variation between the two plots. During the first time period, the target sensor data and feature sensor data have a high correlation (e.g., ∼0.87), while during the second time period, the correlation is low (e.g., ∼0.3). Therefore, such dynamic changes and the presence of missing values ​​make analyzing the sensor data very challenging.

[0019] Therefore, to avoid errors in anomaly detection due to the state of the sensor data, it is desirable to pre-process the sensor data before applying anomaly detection techniques. Figure 2 shows a flowchart illustrating an exemplary process for performing data exploration on sensor data, according to an embodiment of the present invention. In some embodiments, the data exploration system can include three modules: a data pre-processing module, a data cleansing module, and a feature engineering module.

[0020] During operation, the system may first acquire sensor data (operation 202). Then, the system preprocesses the sensor data to select a set of feature sensors based on a predetermined target sensor (operation 204). In one embodiment, the system may cleanse the preprocessed sensor data to address missing values, outliers, and noise in the selected sensor data (operation 206). The data cleansing operation may improve the quality of the sensor data and reduce the amount of sensor data, thereby reducing computational costs. Following the data cleansing operation, the system performs feature engineering (operation 208). In one embodiment, the feature engineering adjusts the delay associated with data from the feature sensors relative to the target sensor. The delay-shifted feature sensor data is then ranked based on their correlation with the target sensor. Feature sensors with a high correlation with the target sensor may be used for further processing in the anomaly detection system, while feature sensors with a low correlation may be omitted.

[0021] FIG. 3 shows a flowchart illustrating an exemplary process for performing data preprocessing, according to one embodiment of the present invention. During operation, the system acquires sensor data (operation 302). The acquired sensor data may include information collected from multiple sensors embedded in one or more machines in a factory layout. Data associated with one of the sensors may be identified as a target variable for anomaly analysis (operation 304). Based on the identified target variable, the system automatically screens the sensor data to identify candidate feature sensor data associated with the target sensor (operation 306). For example, based on available data attributes in the sensor data, a sensor physically connected to the target sensor in the factory layout may be selected as the feature sensor. In one embodiment, a user may also select a “user-defined” feature for anomaly detection.

[0022] Following data selection according to the process illustrated in the flowchart of Figure 2, a data cleansing module can perform data cleansing to address missing values, outliers, and noise in the loaded sensor data. Figure 4 shows a flowchart illustrating an exemplary process for performing data cleansing according to one embodiment of the present invention. If a selected candidate feature sensor contains a large proportion of missing values ​​during the target sensor's recording period, e.g., 20% of the recording period, data associated with this candidate feature sensor can optionally be discarded. If the remaining feature sensors contain a small proportion of missing values ​​during the target sensor's recording time, these missing values ​​are replaced by linear interpolation (operation 402).

[0023] After the missing values ​​in the sensor data are replaced, the system can perform univariate analysis to determine the feature type and find outliers in each feature sensor data. Typically, there are two types of feature types: monotonic feature type and instant feature type. Based on these feature types, outliers can be identified and replaced by linear interpolation. Specifically, for instant feature type, outliers can be found using the interquartile range (IQR) method. For example, xij If x represents the i-th sample of the j-th feature sensor, then ij is an outlier.

[0024]

number

[0025] Q1 and Q3 are the characteristic sensors x j For monotonic features, the i-th sample of the j-th feature sensor is an outlier if

[0026]

number

[0027] After the outliers are identified, the system can replace them by interpolation and can reduce other variations in the sensor data by performing a moving average on the sensor data with a predefined window size.

[0028] FIG. 5A illustrates an example of handling missing values ​​in sensor data, according to one embodiment of the present invention. Specifically, FIG. 5A illustrates a data map of sensor data recorded over a period of time. Rows in the data map represent timestamps, and columns represent sensors for which data is recorded. The sensors for which data is recorded include a target sensor and a set of feature sensors represented by {F1, F2, F3, F4, F5, F6, F7, F8, F9}. Darker areas 502 in the data map indicate recorded sensor data values, while empty or light areas 504 in the data map indicate missing values. For example, data recorded for feature sensors such as {F5, F6, F7, F8, F9} during the target sensor's recording time is selected 506, while the remaining feature sensors that do not show data in the data map are discarded.

[0029] 5B shows the results of performing a moving average on sensor data according to one embodiment of the present invention. The sensor data shown in plot (a) was previously subjected to missing value handling, univariate analysis, and interpolation. Plots (b)-(c) show the sensor data after moving averages of different window sizes have been applied to the sensor data in plot (a).

[0030] Following performing the data cleansing shown in Figure 4, the system can then apply feature engineering to the sensor data. Figure 6 shows a flowchart illustrating an exemplary process for performing feature engineering, according to one embodiment of the present invention. During operation, the system performs feature transformation (operation 602) on monotonic features and calculates their rate of change as new features.

[0031] In real-world industrial IoT applications, delays typically exist in data collected from different parts of a production line in a factory layout. Thus, data associated with some of the feature sensors may exhibit a time delay relative to the target sensor. The delay between the feature sensor data and the target sensor data can be calculated by using normalized cross-correlation (operation 604), which is expressed as follows:

[0032]

number

[0033] In the formula, μ xj and μ y represent the average values ​​of the jth feature sensor and the target sensor y, respectively, and σ xj and σ y are the standard deviations of the jth feature sensor and the target sensor y, respectively, (*) is the convolution operator, and ∥x j ∥ is x j Then, the length of the feature sensor x j is shifted by the delay calculated in equation (6).

[0034] The delay-shifted feature sensor is fed to the feature correlation module. j This modular correlation between the target sensor y and the target sensor y is calculated (operation 606) by:

[0035]

number

[0036] where cov(x j ,y) is x j represents the covariance between and y.

[0037] Using the correlation calculated in operation 606 according to equation (8), the system can rank the feature sensors accordingly (operation 608). In one embodiment, the system ranks the feature sensors according to a predefined threshold ρ 閾 Select features with correlation values ​​higher than the value, since features that show low correlation with the target sensor may not be suitable for building a pairwise univariate anomaly model.

[0038] 7A shows a visualization of a feature 704 that is delayed relative to a target variable 702 due to a line layout in a factory, according to one embodiment of the present invention. To time-align the feature 704 with the target 702, the delay of the feature 704 is calculated according to equation (6). The feature 704 is then shifted by the delay value calculated from equation (6) to obtain a shifted feature 706.

[0039] 7B illustrates an exemplary data correlation map between data associated with a target sensor T and data associated with a set of feature sensors {F1, F2, F3, F4, F5, F6, F7, F8, F9}, according to one embodiment of the present invention. Feature sensors that exhibit high correlation with the target sensor data are selected, and feature sensors with low correlation with the target sensor data are omitted. For example, a predefined threshold ρ 閾値When it is 0.9, only the feature sensors {F1, F2, F3} are selected. Then, the selected feature sensors are ranked according to their correlation values.

[0040] Unsupervised Anomaly Detection System Architecture FIG. 8 shows an exemplary unsupervised anomaly detection system architecture 800 according to an embodiment of the present invention. The unsupervised anomaly detection system 800 may include a sensor database 802, a data exploration module 804, a model builder module 806, an anomaly detection module 808, and a weighted voting module 810.

[0041] The sensor database 802 stores sensor data {a1, a2,..., a m} collected from a set of M sensors built into one or more machines. The data exploration module 804 can automatically process the sensor data {a1, a2,..., a m} to repair missing values, outliers, and noise in response to the process. After the missing values, outliers, and noise are repaired according to the flowcharts shown in FIGS. 2 to 4, the sensor data has a high correlation with the target sensor data, and a finite set of feature sensors {x1, x2,... x j ,..., x p} (where p << m) is selected. Optimally, the system can omit the remaining feature sensors that show low correlation. After the data exploration stage, p feature sensors and a given target sensor y are available, and each sensor has n time samples. The system can normalize the sensor data so that it has zero mean and unit variance of linear regression. In some embodiments, the system transmits the normalized data associated with the selected feature sensors {x1, x2,... x j ,..., x p} and the target sensor y to another computer system implementing the model builder module 806 to calculate the anomaly distance.

[0042] Model Builder Module806 can construct a set of pairwise univariate anomaly models, each pairwise univariate anomaly model representing a pair of feature sensors x j and perform a pairwise linear regression on the target sensor y.

[0043] y is y=[y i ] n represents the normalized target matrix defined by, where y i represents the i-th sample of the target sensor, where 1≦i≦n. Similarly, X is expressed as X=[x ij ] np represents the normalized feature matrix defined by ij represents the i-th sample of the j-th feature sensor for 1≦i≦n and 1≦j≦p. .j =[x 1j ,x 2j ,...,x nj ] represents the data from the jth feature sensor. Since all sensor data are normalized, their variance is Var[y]=Var[X .j ]=1, and the mean is E[y]=E[X .j ]=0.

[0044] The pairwise univariate anomaly model for the jth feature sensor and target sensor y is calculated as follows: First, the functional relationship between the target sensor and the feature sensor is estimated as follows:

[0045]

number

[0046] where:

[0047]

number

[0048] represents the predicted i-th target value of the j-th feature sensor,

[0049]

number

[0050] are the estimated regression coefficient parameters. The estimated regression coefficient parameters may be:

[0051]

number

[0052] The estimated regression coefficient parameters can be used to measure the fit of the pairwise univariate anomaly model. The measure of model fit is R 2 The model fitness of the jth feature sensor is expressed as

[0053]

number

[0054] can be expressed as

[0055]

number

[0056] is.

[0057]

number

[0058] A higher value of σ implies a better linear model of the j-th feature sensor. Furthermore, the model fit measure of the j-th feature sensor

[0059]

number

[0060] is the estimated regression coefficient as follows:

[0061]

number

[0062] Related to.

[0063]

number

[0064] After the pairwise univariate anomaly model is predicted for the jth feature sensor, the observed target sensor value y i The direct intersection between and the predicted model of the jth feature sensor is determined by:

[0065]

number

[0066] where d ij can be called the anomaly distance of the i-th sample and the j-th feature sensor. ij The mean and variance of are given by:

[0067]

number

[0068] All feature sensors X=[x ij ] np The anomaly distance associated with D=[d ij ] np These anomaly distances are sent to the anomaly detection module 808 to determine the voting score for each model in a set of p pairwise univariate anomaly models.

[0069] The anomaly detection module 808 applies a set of clustering functions to the anomaly distance matrix to determine the voting score. The set of clustering functions is G=[g j ] pEach clustering function is independently calculated for j=1,2,...,p. j Each clustering function classifies the observed anomaly distances into a binary output. Specifically, the output value is set to 1 if the anomaly is present, and 0 if it is not. The clustering function g j can be expressed as follows:

[0070]

number

[0071] where V is [v ij ] np and v ij ∈{0,1}. For a set of clustering functions, the voting matrix V can be expressed as V = G(D).

[0072] In one embodiment, a Gaussian Mixture Model (GMM) clustering function G with two centroids (i.e., k=2) is used. s represents a random variable, and the Gaussian distribution of the random variable has mean μ s and standard deviation σ s N(s | μ s ,σ s ) for GMM, d ij The probability density function of is expressed in terms of a Gaussian distribution as follows:

[0073]

number

[0074] where π k is 0≦π k ≦1 and Σ k π k = 1. The GMM model uses the training data {d ij │i=1,2,...,n}. Assuming μ0<μ1, the abnormal state can be trained by the GMMj (d ij )=1, then v ij = 1 otherwise it can be set to abnormal by voting as 0.

[0075] Anomalies in the feature sensors are represented by the voting score v ij However, according to one embodiment, the accuracy of anomaly detection can be improved by using majority voting in the weighted voting module 810. In the weighted voting module 810, the anomaly weighted score for the i-th target sample is calculated by:

[0076]

number

[0077] where S=[s ij ] np is called the anomaly point matrix, and R 2 By value,

[0078]

number

[0079] We denote the rescaled anomaly distance matrix as<a,b> represents the linear product of a and b. The anomaly score matrix S can be used to visualize the anomaly patterns of the target sensor and contributing feature sensors over time. Such visualization can help users quickly identify targets of interest that have significant anomaly patterns.

[0080] In one embodiment, the anomaly weight matrix W can be used to quantify the contribution of each feature sensor to the anomaly vote score. This quantification of each feature sensor's contribution can be used to rank the importance of the feature sensors to the anomaly.

[0081] For all feature sensors, the majority vote u = [u i ]n can be written as follows:

[0082]

number

[0083] where ° is the Hadamard product and 1 is the p×1 identity matrix. The final determination 812 of the anomaly of the i-th target sample is defined by:

[0084]

number

[0085] Here, 0≦U tr <=1 is a user-defined majority voting threshold. Figure 9 shows pseudocode for detecting anomalies using an unsupervised anomaly detection method with majority voting according to one embodiment.

[0086] FIG. 10 shows example unsupervised anomaly detection results according to one embodiment of the present invention. The top diagram (a) shows a plot of target sensor values ​​recorded over a period of time. Red areas on the curve represent the presence of anomalies. The second diagram (b) in the middle shows a plot of anomaly distances associated with seven selected feature sensors. Peaks in the anomaly distances correspond to falsely detected anomalies. However, after combining the anomaly distances with majority voting techniques, the anomaly detection system can accurately detect anomalies in the target sensor data. The last diagram (c) shows the results of the majority voting technique with seven selected feature sensors according to one embodiment of the present invention. The red line represents the majority voting threshold. Values ​​resulting from the weighted anomaly votes that cross the majority voting threshold (red line) correspond to anomalies. Anomalies marked in the target sensor values ​​(in plot (a)) are detected with high accuracy by the unsupervised anomaly detection system with majority voting.

[0087] 11 shows a flowchart illustrating an exemplary process for performing unsupervised anomaly detection with majority voting, according to one embodiment of the present invention. During operation, the system may obtain sensor data stored in a sensor database (operation 1102). The sensor data is associated with multiple sensors embedded in one or more machines in a factory layout.

[0088] In industrial IoT applications, the amount of collected sensor data is significantly large due to the increasing number of sensors. Furthermore, the sensor data usually includes a large number of poorly adjusted data, including missing, corrupted, noisy, and highly correlated values. When such poor-quality sensor data is used for anomaly detection, the results are inaccurate and therefore unreliable. Furthermore, the computational complexity of anomaly detection increases significantly due to the availability of a large number of sensor data. To improve the quality of the sensor data and reduce the number of feature sensors, data exploration is performed on the sensor data (operation 1104).

[0089] Following the execution of the data exploration, the system may construct a set of linear models. In other words, the system uses data associated with each feature sensor and the target sensor to construct pairwise univariate anomaly models and infer relationships between them. The system calculates a set of anomaly distances for the set of feature sensors based on the inferred relationships (operation 1106).

[0090] During operation 1108, a set of anomalous "candidate" events are identified. They are then confirmed as anomalous events based on majority voting in operations 1110 and 1112, with responsibility attributed to the feature sensor with the highest score. An anomalous event is identified based on an estimated relationship between the target sensor and the set of feature sensors. Specifically, a set of anomalous voting scores is calculated based on a set of anomalous distances and a set of clustering functions (operation 1108).

[0091] Finally, a collective determination of anomalies is made by a majority voting procedure (operation 1110), which may be similar to the operation of weighted voting module 810 shown in FIG.

[0092] Exemplary Computer Systems and Devices 12 illustrates an exemplary computer system facilitating an unsupervised anomaly detection system according to one embodiment of the present invention. Computer system 1200 includes a processor 1202, a memory 1204, and a storage device 1206. Computer system 1200 may be coupled to a display device 1210, a keyboard 1212, and a pointing device 1214, and may also be coupled to a network 1208 via one or more network interfaces. Storage device 1206 may store an operating system 1218 and an unsupervised anomaly detection system 1220.

[0093] The unsupervised anomaly detection system 1220 may include instructions that, when executed by the computer system 1200, cause the computer system 1200 to perform the methods and / or processes described in this disclosure. The unsupervised anomaly detection system 1220 may also include instructions for receiving sensor data associated with one or more sensors (sensor data receiving module 1222), instructions for performing data exploration on the sensor data (sensor data exploration module 1224), and instructions for processing the sensor data after the data exploration phase (linear model builder module 1226). Additionally, the unsupervised anomaly detection system 1220 may include instructions for detecting anomalous events (anomaly detection module 1228) and instructions for performing majority voting on the detected anomalous events (majority voting module 1230).

[0094] FIG. 13 illustrates an exemplary apparatus for facilitating an unsupervised anomaly detection system according to an embodiment of the present invention. The apparatus 1300 may include multiple units or devices that may communicate with each other via wired, wireless, quantum optical, or electrical communication channels. The apparatus 1300 may be implemented using one or more integrated circuits and may include fewer or more units or devices than those illustrated in FIG. 13 . Furthermore, the apparatus 1300 may be integrated into a computer system or implemented as a separate device capable of communicating with other computer systems and / or devices. Specifically, the apparatus 1300 may include units 1302-1312 that perform functions or operations similar to the modules 1220-1230 of the computer system 1200 in FIG. 12 , including a sensor data receiving unit 1302, a sensor data searching unit 1304, a linear model builder unit 1306, an anomaly detection unit 1308, and a majority voting unit 1310. The apparatus 1300 may further include a communication unit 1312.

[0095] In general, embodiments of the present invention provide a method and system for detecting and explaining anomalies in high-dimensional and unlabeled sensor data using unsupervised learning with majority voting. Detecting anomalies in sensor data associated with factory machines is used as an example. In fact, this solution is not limited to detecting anomalies associated with factory machines. It can also be used to detect anomalies in other types of equipment or machines.

[0096] The methods and processes described in the "Detailed Description of the Invention" section may be embodied as code and / or data that may be stored in a computer-readable storage medium as discussed above. When a computer system reads and executes the code and / or data stored on the computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored in the computer-readable storage medium.

[0097] Additionally, the methods and processes described above may be included in hardware modules or devices, which may include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), dedicated or shared processors that execute specific software modules or code at specific times, and other now-known or later-developed programmable logic devices. When the hardware modules or devices are powered up, the methods and processes contained therein are executed.

[0098] The foregoing embodiments described herein have been presented for purposes of illustration and description only. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Accordingly, many modifications and variations will be apparent to those skilled in the art. Additionally, the above disclosure is not intended to limit the invention. The scope of the invention is defined by the appended claims.

Claims

1. 1. A computer-implemented method for detecting anomalies in the operation of a machine, the method comprising: recording via a set of sensors associated with the machine while the machine is operating to obtain sensor data associated with the machine, the set of sensors including a target sensor and a set of feature sensors; storing the sensor data in a computing device; performing a data lookup on the stored sensor data, the data lookup comprising: Identifying a subset of feature sensors from the set of feature sensors to be used in a subsequent unsupervised machine learning process by calculating correlations between the set of feature sensors and the target sensor, ranking the set of feature sensors based on the correlation values, and selecting a subset of feature sensors from the ranked set of feature sensors; and applying data cleansing techniques to improve the quality of the sensor data and reduce the volume of the sensor data, thereby reducing the computational costs associated with the subsequent processing; performing a data search, performing the unsupervised machine learning to construct a set of pairwise univariate models based on the identified subset of feature sensors and the target sensor; Identifying abnormal events in each output of the set of pairwise univariate models using a Gaussian mixture model (GMM), and using the results of the GMM to perform weighted voting to determine whether an abnormality occurs in the operation of the machine by weighted majority vote; A method comprising:

2. applying the data cleansing technique, replacing the missing values ​​in the target sensor data by linear interpolation if the amount of missing values ​​during the recording period of the sensor data is less than a threshold; if the amount of missing values ​​during the recording period of the target sensor data is greater than the threshold, leaving the missing values ​​in the sensor data missing; The method of claim 1 , comprising:

3. The method of claim 1 , wherein each pairwise univariate anomaly model from the set of pairwise univariate anomaly models is associated with a feature sensor from the subset of feature sensors and a target sensor.

4. 1. An apparatus for detecting anomalies in the operation of a machine, comprising: one or more processors; a set of sensors embedded in the machine; a memory that, when executed by the one or more processors, causes the device to: recording via a set of sensors associated with the machine while the machine is operating to obtain sensor data associated with the machine, the set of sensors including a target sensor and a set of feature sensors; storing the sensor data in the device; performing a data lookup on the stored sensor data, the data lookup comprising: Identifying a subset of feature sensors from the set of feature sensors to be used in a subsequent unsupervised machine learning process by calculating correlations between the set of feature sensors and the target sensor, ranking the set of feature sensors based on the correlation values, and selecting a subset of feature sensors from the ranked set of feature sensors; and applying data cleansing techniques to improve the quality of the sensor data and reduce the volume of the sensor data, thereby reducing the computational costs associated with the subsequent processing; performing a data exploration, the data exploration including performing the unsupervised machine learning to construct a set of pairwise univariate models based on the identified subset of feature sensors and the target sensor; a memory storing instructions for determining whether an anomaly occurs in the operation of the machine by a weighted majority vote by using a Gaussian mixture model (GMM) to determine whether each output of the set of pairwise univariate models indicates the presence or absence of an anomaly based on the set of pairwise univariate models, and performing weighted voting using the results of the determination; and An apparatus comprising:

5. applying the data cleansing technique, replacing the missing values ​​in the target sensor data by linear interpolation if the amount of missing values ​​during the recording period of the sensor data is less than a threshold; if the amount of missing values ​​during the recording period of the target sensor data is greater than the threshold, leaving the missing values ​​in the sensor data missing; The apparatus of claim 4 , comprising:

6. The apparatus of claim 4 , wherein each pairwise univariate anomaly model from the set of pairwise univariate anomaly models is associated with a feature sensor from the subset of feature sensors and a target sensor.

7. 1. A non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for detecting anomalies in machine operation, the method comprising: recording via a set of sensors associated with the machine while the machine is operating to obtain sensor data associated with the machine, the set of sensors including a target sensor and a set of feature sensors; storing the sensor data in a computing device; performing a data lookup on the stored sensor data, the data lookup comprising: Identifying a subset of feature sensors from the set of feature sensors relevant to a subsequent unsupervised machine learning process by calculating correlations between the set of feature sensors and the target sensor, ranking the set of feature sensors based on the correlation values, and selecting a subset of feature sensors from the ranked set of feature sensors; and applying data cleansing techniques to improve the quality of the sensor data and reduce the volume of the sensor data, thereby reducing the computational costs associated with the subsequent unsupervised machine learning process; performing a data search, performing unsupervised machine learning to construct a set of pairwise univariate models based on the identified subset of feature sensors and the target sensor; determining whether each output of the set of pairwise univariate models indicates the presence or absence of an abnormality using a Gaussian mixture model (GMM), and performing weighted voting using the results to determine whether an abnormality occurs in the operation of the machine by weighted majority vote; 1. A non-transitory computer-readable storage medium comprising:

8. 8. The non-transitory computer-readable storage medium of claim 7, wherein each pairwise univariate anomaly model from the set of pairwise univariate anomaly models is associated with a feature sensor from the subset of feature sensors and a target sensor.

Images