Industrial data label-free anomaly detection algorithm and system

Abstract

The invention provides an industrial data label-free anomaly detection algorithm, which comprises the following steps: carrying out sliding window processing on obtained continuous production time sequence data, carrying out unsupervised clustering learning, and combining a clustering result as an additional feature into the continuous production time sequence data; performing redundant information processing on the obtained time series data of the K clustering labels by adopting a low-rank graph space-time network method, and finally outputting the data combining time and space characteristics by using full connection; using a recurrent neural network to further enhance the learning of time sequence features of the obtained high-dimensional spatial-temporal feature data, and outputting data distribution in the high-dimensional spatial-temporal feature data; and carrying out probability distribution statistics on the obtained data distribution, and judging whether the time sequence data in the sliding window time period is abnormal or not by combining a clustering result. According to the technical scheme, the abnormal data of the label-free industrial scene assembly line is rapidly detected by using the label-free abnormal data training learning and the low-rank map deep learning network, and the speed and accuracy of risk early warning are effectively improved.

Description

Technical Field

[0001] The present application belongs to the field of time series data processing, and specifically relates to an algorithm and system for detecting anomalies in industrial data without labels. Technical Background

[0002] In the industrial field, time series data usually comes from continuous processes monitored by sensors, such as recording parameters such as temperature, pressure, and vibration. This data is crucial for monitoring equipment health, risk perception, predictive maintenance, and ensuring production quality. However, there is a lot of redundancy in industrial time series data due to the following reasons: (1) Periodic repetition: Many industrial processes are sampled regularly, resulting in periodicity in the time dimension of the data, and adjacent data points may be very similar; (2) Stable operating conditions: Equipment operating in a stable state may produce almost unchanged data for a long period of time; (3) Limited operating modes: Industrial equipment usually has several fixed operating modes, resulting in data repetition in these modes; (4) Sensor noise and error: Sensor readings may contain noise or systematic errors, which can cause random fluctuations or systematic deviations in the data; (5) Spatial redundancy: At the same time, there may be spatial correlation between different sensors, especially when they are close to each other or monitor similar physical phenomena.

[0003] Therefore, on the one hand, the existence of a large number of redundant phenomena leads to low efficiency in data calculation and storage; on the other hand, in real industrial big data scenarios, although the amount of data is large, it is often very difficult to extract effective and trainable abnormal data from it, resulting in that in industrial scenarios for abnormal detection scenarios (such as equipment risk perception), the data set that can be provided to the model for training and learning is often not very large, and the proportion of data marked as abnormal is not large. Therefore, facing industrial sites with a large amount of real-time data generated, the accuracy of fast models is not good, and the accurate models run too slowly. What's more, most algorithms require more events to process and judge, and in some scenarios they often cannot meet the needs of real-time processing.

[0004] Technical content

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an unlabeled anomaly detection algorithm and system for industrial data. By using unlabeled anomaly data training and learning and low-rank graph deep learning networks, it is possible to quickly detect abnormal data of unlabeled industrial scene pipelines, effectively improving the speed and accuracy of risk warning.

[0006] In order to achieve the above objectives, in a first aspect, an embodiment of the present application provides an unlabeled anomaly detection algorithm for industrial data, including:

[0007] After sliding window processing is performed on the acquired continuous production time series data, unsupervised clustering learning is performed, and the clustering result is incorporated into the continuous production time series data as one of the additional features;

[0008] The time series data of the K cluster labels obtained are processed with the low-rank graph spatiotemporal network method to process redundant information, and finally the data combining time and space features is output using full connection;

[0009] The obtained high-dimensional spatiotemporal feature data is further enhanced by using a recursive neural network to learn the time series features, and the data distribution in the high-dimensional spatiotemporal feature data is output;

[0010] The obtained data distribution is subjected to probability distribution statistics, and combined with the clustering result, it is determined whether the time series data within the sliding window time period is abnormal.

[0011] Optionally, the acquiring of continuous production time series data and sliding window processing includes: averaging and aggregating the time series data every 10 seconds, segmenting the data using a sliding window method, and combining and outputting every 64 pieces of segmented data.

[0012] Optionally, the low-rank graph spatiotemporal network includes M low-rank spatiotemporal modules, each of which is composed of several low-rank spatial convolutional networks and low-rank temporal mixing networks; wherein the low-rank spatial convolutional network is used to process abstract association relationships and spatial relationships in the data, and the low-rank temporal mixing network is used to process temporal relationships in the data.

[0013] Optionally, the low-rank spatial convolutional network is used to process abstract association relationships and spatial relationships in data, including: using two self-learning low-rank matrices LR1 and LR2 low-rank graphs in the graph network to represent the connection relationship between nodes, and then using the graph convolution method to learn the spatial relationship in the data:

[0014] Z=f(x,LR1,LR2)=softmax(((LR1*LR2+I)Relu((LR1*LR2+I)X)W (0) )W (1) )

[0015] Among them, I is the n-order unit matrix, W (0) and W (1) The weights that the model needs to learn

[0016] Optionally, the low-rank time mixing network is used to process the time relationship in the data, including: inputting the time series data, sampling at different frequencies, using only the MLP network to perform multi-dimensional learning on the multi-frequency data, and using the MixUp method to finally fully connect the different sampling results and output them.

[0017] Optionally, the obtained high-dimensional spatiotemporal feature data is further enhanced by using a recursive neural network to learn the time series features, including: inputting the high-dimensional spatiotemporal feature data into a stream network model for learning, converting the data into known simple distributions through a series of reversible transformations, outputting the data distribution of the high-dimensional spatiotemporal feature data, and using a fitting function of

[0018]

[0019] Among them, f θ is the spatiotemporal mixing model function.

[0020] Optionally, the high-dimensional spatiotemporal feature data obtained is further enhanced by using a recursive neural network to learn the time series features, and further includes: optimizing the recursive neural network model using a maximum likelihood estimation method, the formula is:

[0021]

[0022] Where N is the total number of windows, K is the number of clusters, θ is the model weight parameter, P is the above fitting function, and f θ is the spatiotemporal mixing model function.

[0023] Optionally, performing probability distribution statistics on the obtained data distribution and judging whether the time series data within the sliding window time period is abnormal in combination with the clustering result includes:

[0024] According to the difference of each clustering result, the time series of different data are mapped to a unique target distribution to obtain different anomaly scores S C , and is scored by the outlier value S C Set a threshold. In actual application, if the abnormal value score of the continuous production time series data output by this algorithm within the sliding window time period exceeds the threshold, it is judged as an abnormal time series;

[0025]

[0026] Among them, P xk is the simple score obtained from the previous data, X c k is the cth sample data in the Kth class in the clustering result.

[0027] Optionally, the abnormal value score S C Set the threshold, including: Save S of the entire training set c , and use the interquartile range (IQR) to set the threshold:

[0028] Threshold=λ(Q3+1.5*(Q3-Q1))

[0029] where Q1 and Q3 are the 25th and 75th percentiles of Sc, and λ is a setting parameter.

[0030] In a second aspect, an embodiment of the present application provides an industrial data unlabeled anomaly detection system, including:

[0031] A preprocessing module, used for performing unsupervised clustering learning on the acquired continuous production time series data after sliding window processing, and combining the clustering result as one of the additional features into the continuous production time series data;

[0032] The low-rank graph processing module is used to process the redundant information of the obtained K cluster label time series data using the low-rank graph spatiotemporal network method, and finally output the data combining time and space features using full connection;

[0033] An algorithm processing module, used to further strengthen the learning of time series features using a recursive neural network on the obtained high-dimensional spatiotemporal feature data, and output the data distribution in the high-dimensional spatiotemporal feature data;

[0034] The statistical module is used to perform probability distribution statistics on the obtained data distribution, and determine whether the time series data within the sliding window time period is abnormal in combination with the clustering result.

[0035] The technical solution of the present application can enable the model to learn the data distribution in the scenario, increase the detection rate of abnormal data, and increase the generalization ability in different scenarios while minimizing the manual addition or subtraction of data features and modification of distribution; at the same time, while ensuring the learning effect of the model, the training and prediction processing speed of the model is improved.

[0036] Instruction Manual

[0037] Figure 1 A schematic diagram of the overall process of an industrial data unlabeled anomaly detection algorithm provided by an embodiment of the present invention;

[0038] Figure 2 An embodiment of the present invention provides an industrial data unlabeled anomaly detection algorithm based on a low-rank spatiotemporal graph enhanced flow model. DETAILED DESCRIPTION

[0039] Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0040] See also Figure 1 , an industrial data unlabeled anomaly detection algorithm provided by an embodiment of the present application includes:

[0041] S100, after obtaining the continuous production time series data and performing sliding window processing, unsupervised clustering learning is performed, and the clustering result is incorporated into the continuous production time series data as one of the additional features.

[0042] Specifically, in the production data reading training phase, the production data of the production line is read. The production line data is mainly time series data, which mainly comes from two sources: the real-time data of production raw material input and the equipment operation status data. Because the continuous production reaction is relatively slow, it is preferred to average and aggregate the time series data every 10 seconds, add the sliding window method to split the data, and combine and output every 64 data.

[0043] Furthermore, the obtained time series data is clustered unsupervisedly, and the similarity in the input data samples is calculated using the unlabeled K-means clustering method based on Euclidean distance and cross-correlation. The data with similar data distribution are aggregated into one category, and the aggregation result is combined into the original time series data as one of the additional features, finally obtaining time series data with K clustering labels.

[0044] S200, the time series data of the obtained K cluster labels are processed with the method of low-rank graph spatiotemporal network for redundant information, and finally the data combining time and space features are output using full connection.

[0045] Since there is a large amount of redundant information in these data and there are no anomaly detection labels in the data set, we expect the model to be able to adaptively learn the data distribution in the production data and distinguish between normal production data and abnormal production data. Among them, in view of the large amount of redundant information in the data, this embodiment uses a low-rank graph space-time network method to process the redundant information, including M low-rank space-time modules, each of which is composed of several spatial low-rank spatial graph convolutional networks and low-rank time hybrid networks. Taking into account the abstract association relationship between key parameters in the production process, the spatial low-rank spatial graph convolutional network is used to process the abstract association relationship and spatial relationship in the data, which is manifested as: using two self-learning low-rank (LowRank) matrices LR1 and LR2, a low-rank graph is constructed to represent the connection relationship between each node. Furthermore, the graph convolution method is used to learn the spatial relationship in the data:

[0046] Z=f(X,L)=softmax((L+I)*Relu((L+I)XW (0) )W (1) )

[0047] Where L is a low-rank graph matrix, obtained by multiplying two self-learning low-rank matrices: L = LR1*LR2. From this, we can get the low-rank graph convolution algorithm:

[0048] Z=f(x,LR1,LR2)=softmax(((LR1*LR2+I)Relu((LR1*LR2+I)X)W (0) )W (1) )

[0049] Among them, I is the n-order unit matrix, W (0) and W (1) are the weights that the model needs to learn.

[0050] At the same time, the temporal relationship of the data is learned through a low-rank time mixture network. The time series data is input and sampled at different frequencies. Only the MLP network is used to perform multi-dimensional learning on the multi-frequency data, and the MixUp method is used to fully connect the different sampling results and output them.

[0051] Finally, the data combining time and space features are output using full connection to obtain high-dimensional spatiotemporal feature data.

[0052] S300, using a recursive neural network to further strengthen the learning of time series features with the obtained high-dimensional spatiotemporal feature data, and outputting the data distribution in the high-dimensional spatiotemporal feature data.

[0053] In order to further enhance the learning ability of the model on time series features, the recurrent neural network RNN ​​is further used for time series feature learning. The high-dimensional spatiotemporal feature data is input into the stream network model for learning, and the data is converted into known simple points through a series of reversible transformations, and the data distribution in the spatiotemporal data is output. The fitting function used is:

[0054]

[0055] Among them, f θ is the spatiotemporal mixing model function.

[0056] In order to prevent each module of the model from being stuck at a local optimal value and effectively reduce the number of parameters, all modules are optimized together to achieve overall performance improvement. * It is obtained through maximum likelihood estimation.

[0057]

[0058] Where N is the total number of windows, K is the number of clusters, θ is the model weight parameter, P is the above fitting function, and f θ is the spatiotemporal mixing model function.

[0059] S400 , performing probability distribution statistics on the obtained data distribution, and combining the clustering result in S100 to determine whether there is an abnormality in the time series data within the sliding window time period.

[0060] Specifically, for each different clustering result, the time series of different data are mapped to a unique target distribution to obtain different anomaly scores S c :

[0061]

[0062] Among them, P xk is the simple score obtained from the previous data, X c k is the cth sample data in the Kth class in the clustering result.

[0063] In order to reduce abnormal disturbances, this technical solution stores S of the entire training set. c , and use the interquartile range (IQR) to set the threshold:

[0064] Threshold=λ(Q3+1.5*(Q3-Q1))

[0065] Where Q1 and Q3 are the 25th and 75th percentiles of Sc, and λ is a setting parameter. In practical applications, the algorithm finally outputs anomaly score S c If it is greater than the threshold, the corresponding continuous production time series data within the sliding window period is an abnormal time series.

[0066] The technical solution in this embodiment is mainly to use the method of low-rank graph network and flow model to construct an industrial production anomaly detection algorithm model, and unlabeled industrial anomaly data sets are used as training and verification data sets in the process. The low-rank graph hybrid spatiotemporal module in the present invention can further solve the problem of data redundancy in anomaly detection by dimensionality reduction and sampling methods, and at the same time, a method combining time and space is adopted to further learn and capture the spatiotemporal features in industrial data. At the same time, through the structure of the flow model, the maximum likelihood estimate of the distribution is calculated, and the data distribution in the time series data is learned, and further learning is learned to distinguish normal data from abnormal data without labels. At the same time, in the face of large-scale industrial field data, the clustering method is adopted to quickly judge the categories of similar points, which can accelerate the learning and judgment of the model. The low-rank graph learning scheme can effectively improve the calculation speed of the model without reducing the model effect. The low-rank learning method of clustering can significantly improve the training and judgment speed of the model.

[0067] The algorithm does not require manual further feature engineering to screen out highly repetitive data and perform labeling. It also does not require manual data augmentation to deliberately add abnormal data to the data set.

[0068] This embodiment also provides a system based on the industrial data unlabeled anomaly detection algorithm, including:

[0069] The preprocessing module is used to perform unsupervised clustering learning on the acquired continuous production time series data after sliding window processing, and incorporate the clustering result into the continuous production time series data as one of the additional features;

[0070] The low-rank graph processing module is used to process the redundant information of the obtained K cluster label time series data using the low-rank graph spatiotemporal network method, and finally output the data combining time and space features using full connection;

[0071] The algorithm processing module is used to further strengthen the learning of time series features using a recursive neural network to obtain the high-dimensional spatiotemporal feature data, and output the data distribution in the high-dimensional spatiotemporal feature data;

[0072] The statistical module is used to perform probability distribution statistics on the obtained data distribution, and combine the clustering results to determine whether there are any abnormalities in the time series data within the sliding window time period.

[0073] The effect that can be achieved by the system in this embodiment is consistent with the effect that can be achieved by the aforementioned industrial data unlabeled anomaly detection algorithm, so it will not be repeated here.

[0074] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0075] The "first" and "second" in the names such as "first" and "second" (if any) mentioned in the embodiments of the present application are only used as name identifiers and do not represent the first or second in order.

[0076] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0077] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0078] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. An unlabeled anomaly detection algorithm for industrial data, characterized by include: After sliding window processing is performed on the acquired continuous production time series data, unsupervised clustering learning is performed, and the clustering result is incorporated into the continuous production time series data as one of the additional features; The time series data of the K cluster labels obtained are processed with the low-rank graph spatiotemporal network method to process redundant information, and finally the data combining time and space features is output using full connection; The obtained high-dimensional spatiotemporal feature data is further enhanced by using a recursive neural network to learn the time series features, and the data distribution in the high-dimensional spatiotemporal feature data is output; The obtained data distribution is subjected to probability distribution statistics, and combined with the clustering result, it is determined whether the time series data within the sliding window time period is abnormal.

2. The unlabeled anomaly detection algorithm for industrial data according to claim 1, characterized in that The obtaining of continuous production time series data and sliding window processing includes: averaging and aggregating the time series data every 10 seconds, segmenting the data using a sliding window method, and combining and outputting the segmented data every 64 pieces.

3. The unlabeled anomaly detection algorithm for industrial data according to claim 1, characterized in that The low-rank graph spatiotemporal network includes M low-rank spatiotemporal modules, each of which is composed of several low-rank spatial convolutional networks and low-rank temporal mixing networks; wherein the low-rank spatial convolutional network is used to process abstract association relationships and spatial relationships in the data, and the low-rank temporal mixing network is used to process temporal relationships in the data.

4. The unlabeled anomaly detection algorithm for industrial data as claimed in claim 3 is characterized in that The low-rank spatial convolutional network is used to process abstract association relationships and spatial relationships in data, including: using two self-learning low-rank matrices LR1 and LR2 low-rank graphs in the graph network to represent the connection relationship between nodes, and then using the graph convolution method to learn the spatial relationship in the data: Z=f(x,LR1,LR2)=softmax(((LR1*LR2+I)Relu((LR1*LR2+I)X)W (0) )W (1) ) where I is the n-order identity matrix, W (0) and W (1) are the weights that the model needs to learn.

5. The unlabeled anomaly detection algorithm for industrial data as claimed in claim 3 is characterized in that The low-rank time mixing network is used to process the time relationship in the data, including: inputting the time series data, adopting different frequencies, using only the MLP network to perform multi-dimensional learning on the multi-frequency data, and using the MixUp method to finally fully connect the different adoption results and output them.

6. The unlabeled anomaly detection algorithm for industrial data according to claim 1, characterized in that The obtained high-dimensional spatiotemporal feature data is further enhanced by using a recursive neural network to learn the time series features, including: inputting the high-dimensional spatiotemporal feature data into a stream network model for learning, converting the data into known simple distributions through a series of reversible transformations, and outputting the data distribution of the high-dimensional spatiotemporal feature data. The fitting function used is Among them, f θ is the spatiotemporal mixing model function.

7. An unlabeled anomaly detection algorithm for industrial data as described in any one of claims 1 to 6, characterized in that The method of using a recursive neural network to further enhance the learning of time series features using the obtained high-dimensional spatiotemporal feature data also includes: optimizing the recursive neural network model using a maximum likelihood estimation method, the formula is: Where N is the total number of windows, K is the number of clusters, θ is the model weight parameter, P is the above fitting function, and f θ is the spatiotemporal mixing model function.

8. An unlabeled anomaly detection algorithm for industrial data as described in any one of claims 1 to 6, characterized in that The performing probability distribution statistics on the obtained data distribution and judging whether the time series data within the sliding window time period is abnormal in combination with the clustering result includes: According to the difference of each clustering result, the time series of different data are mapped to a unique target distribution to obtain different anomaly scores S C : Among them, P xk is the simple score obtained from the previous data, X c k is the cth sample data in the Kth class in the clustering result; The abnormal value score S C A threshold is set. In actual application, if the abnormal value score of the continuous production time series data within the sliding window time period output by the algorithm exceeds the threshold, it is judged as an abnormal time series.

9. The unlabeled anomaly detection algorithm for industrial data as claimed in claim 8, characterized in that The abnormal value score S C Set the threshold, including: Save S of the entire training set c , and use the interquartile range (IQR) to set the threshold: Threshold=λ(Q3+1.5*(Q3-Q1)) where Q1 and Q3 are the 25th and 75th percentiles of Sc, and λ is a setting parameter.

10. An industrial data unlabeled anomaly detection system, characterized in that include: A preprocessing module, used for performing unsupervised clustering learning on the acquired continuous production time series data after sliding window processing, and combining the clustering result as one of the additional features into the continuous production time series data; The low-rank graph processing module is used to process the redundant information of the obtained K cluster label time series data using the low-rank graph spatiotemporal network method, and finally output the data combining time and space features using full connection; An algorithm processing module, used to further strengthen the learning of time series features using a recursive neural network on the obtained high-dimensional spatiotemporal feature data, and output the data distribution in the high-dimensional spatiotemporal feature data; The statistical module is used to perform probability distribution statistics on the obtained data distribution, and determine whether the time series data within the sliding window time period is abnormal in combination with the clustering result.

Images