Anomaly detection and root cause analysis method and system for industrial time series data

Abstract

The present invention belongs to the technical field of anomaly detection for multi-dimensional time series data of industrial equipment, and discloses a method, system and terminal for anomaly detection and root cause analysis for industrial equipment. The method uses a multi-channel variational long-term and short-term inference network to pre-process the historical data of the industrial equipment; constructs a neural network model and transmits the pre-processed data into the neural network model to train the neural network model; uses the trained neural network model to perform online detection of the multi-dimensional time series data of the industrial equipment, detects the operating status of the equipment in real time, performs root cause analysis and alarm notification on anomalies, and locates indicators of anomalies from multi-dimensional data. The present invention integrates anomaly detection and root cause analysis modules into the Internet of Things platform, empowers the Internet of Things platform, can quickly access the status of large-scale equipment in real time, and can also perform intelligent monitoring of the equipment, detect whether the equipment operation is abnormal in real time and perform root cause analysis on the anomaly, reduce human analysis costs and improve enterprise benefits.

Description

Technical Field

[0001] The present invention belongs to the technical field of anomaly detection in industrial multi-dimensional time series data, and in particular relates to a method, system, and terminal for anomaly detection and root cause analysis of industrial time series data. Background Art

[0002] In recent years, with the rapid development of the Industrial Internet, modern industrial manufacturing systems have achieved the ability to perceive and record production operation status, environment, and processes through monitoring equipment such as sensors, controllers, and intelligent instruments, accumulating vast amounts of industrial data. The Industrial Internet aims to achieve more sensitive and efficient automated control and resource allocation for industrial manufacturing systems, while also improving the energy efficiency of smart factories. However, due to the existence of anomalies in industrial equipment, such as product quality defects, equipment failures, performance degradation, and external environmental changes, if equipment anomalies in industrial production are not promptly and effectively identified, they can lead to hidden safety hazards in the production environment, potentially causing collateral damage to the entire production line. Therefore, industrial equipment anomaly detection has become a critical task in intelligent manufacturing and a key research issue in industrial big data analysis.

[0003] With the development and maturity of machine learning technology, data-driven models have become the mainstream approach for anomaly detection. Industrial production is a continuous, real-time process. Furthermore, industrial monitoring equipment is highly diverse, and the accumulated industrial data is a typical form of multidimensional time series data. Therefore, anomaly detection based on multidimensional time series data has attracted significant attention within the Industrial Internet. However, anomaly detection in industrial multidimensional time series data is a challenging task. First, potential correlations and interactions between different dimensions of the data make anomalous patterns more difficult to detect and identify. Second, industrial big data is characterized by large volume, multi-source heterogeneity, high dynamism, continuous sampling, low value density, and high timeliness. These characteristics all pose significant challenges to anomaly detection in industrial equipment.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows:

[0005] (1) Due to the characteristics of industrial big data, such as large volume, multi-source heterogeneity, strong dynamics, continuous sampling, low value density and strong timeliness, and the large amount of noise in industrial production, the anomaly detection problem in industrial multi-dimensional time series data brings difficulties and challenges.

[0006] (2) The indicators of industrial multidimensional time series data are complex and diverse, and the indicator values ​​change dynamically over time. The same indicator may have periodic changes, sudden rises and falls, linear increases and decreases, or nonlinear increases and decreases in the time series. There are also time-dependent characteristics between different indicators. Changes in one indicator may cause corresponding changes in other indicators. Due to the complex time and space dependence characteristics of multidimensional time series data, abnormal patterns in industrial multidimensional time series data are more difficult to detect and identify. Summary of the Invention

[0007] In response to the problems existing in the prior art, the present invention provides a method, system and terminal for anomaly detection and root cause analysis of industrial multi-dimensional time series data, and in particular relates to a method, system and terminal for anomaly detection and root cause analysis of a hierarchical variational autoencoder based on an attention mechanism.

[0008] The present invention is implemented as follows: a method for anomaly detection and root cause analysis of industrial multi-dimensional time series data, which includes: standardizing the data and performing sliding window processing through a data preprocessing module; constructing a neural network model and transferring the preprocessed data into the neural network model to train the neural network model; using the trained neural network model to perform online detection of the multi-dimensional time series data of industrial equipment, detecting the equipment operating status in real time, and performing root cause analysis and alarm notification on anomalies, thereby locating indicators of anomalies from the multi-dimensional data.

[0009] Furthermore, the anomaly detection and root cause analysis methods for industrial equipment also include:

[0010] (1) After receiving real-time data, the anomaly detection platform uses pytorch to train the model. Before model training, it obtains historical data of industrial equipment operation. The platform provides a historical data query interface according to the agreed specifications. The anomaly detection platform obtains historical data through the interface and performs model training. If there is no interface, the anomaly detection platform accumulates training data sets based on the data in the message queue.

[0011] (2) The trained model is uploaded to the HDFS large file system, and the neural network model is used to detect anomalies in the multi-dimensional real-time data stream uploaded by the smart sensor and perform root cause analysis on the indicators within the detection window.

[0012] (3) According to different abnormality levels and root cause analysis results, they are pushed to different equipment management personnel and maintenance personnel to check the operating status of abnormal equipment and perform repairs.

[0013] (4) Integrate the anomaly detection and root cause analysis model into the anomaly detection module of the Internet of Things platform, obtain the operating data of industrial equipment in real time through the Internet of Things platform, and then use the anomaly detection module to perform anomaly detection and root cause analysis on the equipment, and observe the operating status of industrial production in real time.

[0014] Furthermore, the anomaly detection and root cause analysis method for industrial multi-dimensional time series data includes the following steps:

[0015] In step 1, to uniformly express the values ​​of each sensor, the data is processed using maximum and minimum normalization, and the multi-dimensional time series data is enhanced using sliding windows and masks to combat network fluctuations and industrial noise.

[0016] Step 2: Use the hierarchical variational autoencoder neural network based on the attention mechanism to reconstruct the multi-dimensional time series data.

[0017] Step 3: Calculate the anomaly score using the multi-dimensional reconstructed residual matrix.

[0018] Step 4: Use extreme value theory to automatically set the abnormal threshold.

[0019] Step 5: Calculate the anomaly score of the real-time online data, compare it with the threshold, and obtain the anomaly results for root cause analysis.

[0020] Furthermore, the hierarchical variational autoencoder based on the attention mechanism in step 2 includes two parts: encoder and decoder.

[0021] The encoder network is used to construct the posterior distribution q φ (s2, s1|x), where two latent variables are used to represent the potential feature space of multidimensional time series data. s1 represents a spatial random latent variable, and its spatial features are captured using the attention mechanism. s2 represents a temporal random latent variable, and its temporal features are captured using LSTM.

[0022] The decoder network needs to reconstruct the data using the spatiotemporal features of multi-dimensional time series data. To minimize the error between the reconstructed data and the original data, the generated data must also be random. Therefore, the VAE probabilistic model is used to construct the decoder network.

[0023] Online detection is used to input industrial multi-dimensional time series data into a hierarchical variational autoencoder based on the attention mechanism, output the reconstructed time series matrix, calculate the matrix difference, and finally perform anomaly detection.

[0024] Furthermore, the multi-dimensional time series data anomaly detection in step 5 includes:

[0025] (1) sensor→rowdata, the sensor uploads data to different protocol parsing layers of the IoT platform through different IoT protocols such as MQTT, ModbusTcp or nb-iot, and obtains unprocessed data rowdata.

[0026] (2) rowdata→msg{"hum":10,"at":16710046,"sn":"1028"}: The protocol adaptation layer converts rowdata into an internally recognizable triple structure; where the scene number sn represents the device, node represents the acquisition card, attribute represents the sensor data, and at represents the timestamp uploaded to the message queue.

[0027] (3) msg ​​→ influxdb. The data center data processing module of the IoT platform cleans the msg data, verifies it, and stores it in various databases, including time series databases or document databases.

[0028] (4) influxdb→tensorX: Preprocess the data in the time series database into a two-dimensional tensor that can be processed by the model.

[0029] (5) tensor X→X', tensorX obtains the reconstruction matrix through the hierarchical variational autoencoder based on the attention mechanism, and the reconstruction matrix is ​​used to detect whether it is running normally.

[0030] (6)X res ←||X-X'||2: Using X res The residual matrix obtains the offset value, which is then measured using various methods to determine whether the indicator is abnormal. The derivative matrix is ​​then fused to obtain the indicator anomaly score. The indicator with the largest anomaly score is used to perform root cause analysis on the anomaly, resulting in the root cause analysis result.

[0031] (7) SendInfo(s, d, title, subject, content): s represents the source address of the message, d represents the destination address, title represents the title, subject represents the subject, and content represents the specific content. When the IoT platform detects an abnormality, it sends a notification to the device manager to handle the abnormality.

[0032] (8) Update(name, phone, subject, content, deviceid): name represents the name of the maintenance personnel, phone represents the phone number, subject represents the subject, content represents the maintenance content, and deviceid represents the device ID. After the maintenance personnel repairs the device, they report the cause of the device failure and whether it is a false alarm, so that the maintenance personnel can review the fault information after the same abnormality occurs.

[0033] Furthermore, the root cause analysis of abnormal data using a multi-dimensional reconstruction matrix includes: using Bayesian inference to construct a multi-channel variational network and obtain potential time features and spatial dimension features; using a generative network to construct a reconstructed timing matrix within a time window, and using the reconstruction matrix to perform real-time anomaly detection on the device status; after the anomaly detection is completed, the reconstruction matrix is ​​used to perform root cause analysis on the abnormal indicators.

[0034] (1)R M×N' ←Conv(R M×N ): Use 1×1 convolution and Relu activation function to process data in the time dimension. While encoding the data, the activation function introduces nonlinear features to the model.

[0035] (2)R M×N” ←MultiHead(R M×N' ): In order to infer spatial random latent variables, a temporal spatial self-attention mechanism is proposed to learn the correlation and dependency features between its indicators. The attention mechanism is often used to learn the complex correlation and dependency relationships and important features of data, and to improve the model's attention and performance on the input data. (3) R M‘×N ←LSTM(Conv1d(M×N”)): In order to restore the time dimension of feature encoding, 1×1 convolution is used for processing, and deep LSTM is used to construct the time features of multi-dimensional time series data

[0036] (4)P θ (x|s1, s2): The reconstruction matrix is ​​obtained by generating the network, and the ELBO is maximized to obtain the final training goal;

[0037] (5)X res ←||X-X'||2: When acquiring online multi-dimensional time series data for industrial production, the severity of the anomaly is calculated by solving the residual matrix. After obtaining the matrix of the anomaly window, the degree of deviation of the single indicator is compared and the degree of data mutation is measured using the derivative matrix. The root cause of the anomaly is analyzed and an anomaly notification is issued.

[0038] (6) Integrate the anomaly detection and root cause analysis modules into the IoT platform, access the status of large-scale equipment in real time and conduct intelligent monitoring of the equipment, detect whether the equipment is operating abnormally in real time and conduct root cause analysis of the anomaly.

[0039] Another object of the present invention is to provide an anomaly detection and root cause analysis system for industrial multidimensional time series data. The anomaly detection and root cause analysis system for industrial multidimensional time series data includes:

[0040] The data acquisition module first defines the industrial scenario to which the data belongs, the protocol required for data transmission, and the sensor type. Protocols include MQTT, Modbus TCP, and private custom protocols. Sensor types include temperature sensors, humidity sensors, voltage and current sensors, and so on.

[0041] The data acquisition module first defines the industrial scenario to which the data belongs, the protocol required for data transmission, and the sensor type. Protocols include MQTT, Modbus TCP, and private custom protocols. Sensor types include temperature sensors, humidity sensors, voltage and current sensors, and so on.

[0042] The model building module is used to build a hierarchical variational autoencoder neural network model based on the attention mechanism, and pass the preprocessed data into the neural network model to train the neural network model;

[0043] The anomaly detection module is used to detect anomalies in industrial multi-dimensional time series data using a trained autoencoder neural network model and detect the operating status of equipment in real time.

[0044] The root cause analysis module is used to perform root cause analysis and alarm notification on abnormal data using a multi-dimensional reconstruction matrix, and then locate the abnormal indicators from the multi-dimensional data.

[0045] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned method for abnormality detection and root cause analysis of industrial equipment.

[0046] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method for abnormality detection and root cause analysis of industrial equipment.

[0047] Another object of the present invention is to provide an information data processing terminal, which is used to implement the above-mentioned abnormality detection and root cause analysis system for industrial equipment.

[0048] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0049] First, the sources of abnormalities in industrial equipment are mainly divided into two aspects. One is the abnormality of the equipment itself, such as the failure of the equipment generator. The other is environmental abnormalities, such as the temperature sensor detecting a sudden rise in temperature due to air temperature or the abnormality of the carbon dioxide sensor caused by gas leakage, as well as abnormalities caused by human operation. In order to maximize the accuracy of anomaly detection and reduce the false alarm rate of anomaly detection, the present invention performs anomaly detection on multi-dimensional time series data based on deep learning. However, multi-dimensional time series data has a series of characteristics such as large volume, continuous sampling, low value density, strong timeliness, and incoherent collected information, so the data needs to be preprocessed before anomaly detection. The data preprocessing of the present invention is divided into data completion and data dimensionality reduction. Finally, on the basis of retaining the general characteristics of the data, the data is completed and reduced in dimension; the hierarchical variational autoencoder neural network based on the attention mechanism is used to detect anomalies in multi-dimensional time series data, and finally a solution is designed that can not only ensure the accuracy of anomaly detection but also locate the root cause of the anomaly in real time. The present invention relates to a neural network-based anomaly detection algorithm, which uses a data acquisition module to collect historical data of industrial equipment, and uses a protocol adapter module to connect and preprocess the data. After preprocessing, the data is transmitted to a constructed neural network to train its model. Secondly, the present invention uses the trained model to perform online detection of the multi-dimensional time series data of industrial equipment, real-time detection of the operating status of its industrial production, rapid alarm notification to management personnel for abnormal data, and intelligently locate its abnormal indicators from multiple dimensions, reducing the possibility of its equipment downtime and reducing its equipment failure migration costs.

[0050] The present invention provides an anomaly detection and root cause analysis method for industrial equipment. A neural network model is trained on multidimensional time series data from historical industrial operations. The trained neural network model is used to perform real-time detection on the multidimensional time series data of industrial equipment, identifying anomalies as early as possible and performing root cause analysis on the anomalies, thereby reducing enterprise losses. Based on a hierarchical neural network, the present invention constructs a system that can learn not only the spatial features of a multidimensional time series matrix but also its temporal features, thereby increasing the accuracy of anomaly detection in multidimensional time series data. The method uses a convolutional neural network to encode the multidimensional time series matrix to remove the noise impact of inconsistent sensor data caused by network delays. A multi-head attention mechanism is then used to learn features between different indicators. Finally, a long short-term network (LSTM) is used to learn temporal features. To meet different time accuracy requirements, the multidimensional time series matrix can be sliced ​​at different time scales, which can alleviate data delays caused by network fluctuations. In the spatial dimension, different indicators have different correlations, with some positively correlated and some negatively correlated. The multi-head self-attention mechanism learns the potential correlation features between these indicators. In industrial production processes, sensor indicators are correlated over time. Some indicators exhibit different states at different times and exhibit sequential correlations. Long- and short-term neural networks can be used to learn these underlying temporal characteristics. Furthermore, using a variational autoencoder (VAE) to learn its probability distribution can address data noise issues, enhancing model robustness.

[0051] Second, the present invention uses a hierarchical variational autoencoder neural network based on the attention mechanism to perform real-time detection on the multi-dimensional time series data of industrial equipment, and designs an anomaly detection model with strong real-time processing capabilities and high anomaly detection accuracy. By integrating the anomaly detection and root cause analysis modules into the Internet of Things platform, the Internet of Things platform is empowered, which can not only quickly access the status of large-scale equipment in real time but also perform intelligent monitoring of the equipment, detect in real time whether the equipment operation is abnormal and perform root cause analysis of its anomalies, reduce human analysis costs and improve corporate benefits.

[0052] Third, as auxiliary evidence of the invention's creativity, it is also reflected in the following important aspects:

[0053] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0054] In traditional manufacturing, companies also use various smart sensors to acquire large amounts of multidimensional industrial time-series data. However, due to the complex and diverse industrial production environments, the variety of IoT protocols, and the instability of networks during data transmission, there is no universal and effective method for data mining and utilization. Therefore, this paper uses deep learning neural networks to perform anomaly mining on multidimensional time-series data, promptly identifying anomalies in industrial production and reducing enterprise losses.

[0055] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0056] Compared with domestic and foreign industrial multi-dimensional time series data anomaly detection methods, the technical solution of the present invention is superior to other methods after comparison through objective indicator values. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0058] Figure 1 This is a flow chart of a method for abnormality detection and root cause analysis of industrial equipment provided by an embodiment of the present invention;

[0059] Figure 2 is a neural network model diagram provided by an embodiment of the present invention;

[0060] Figure 3 It is a time series matrix diagram for mathematically representing multi-dimensional time series data provided by an embodiment of the present invention;

[0061] Figure 4 This is a flowchart of the offline data training process and online data prediction process provided by an embodiment of the present invention;

[0062] Figure 5 is a probability density distribution diagram of indicators provided by an embodiment of the present invention;

[0063] Figure 6 The original time series matrix heat map and the reconstructed matrix heat map provided by the embodiment of the present invention are:

[0064] Figure 7 This is an architecture diagram of the Internet of Things platform provided by an embodiment of the present invention.

[0065] Figure 8 This is a verification effect diagram provided by an embodiment of the present invention. (a) The indicator of dimension 15 of the multidimensional time series data has a point anomaly, with its value far outside the normal range. (b) Dimensions 1 and 3 do not conform to historical trends, and the contextual time-dependent anomaly formed within the interval is successfully detected.

[0066] (c) The two indicators of dimensions 7 and 15 are normally positively correlated, but in other areas they are negatively correlated, detecting an anomaly between the indicators; (d) The data shows a cyclical oscillation trend, but in other abnormal areas, it violates the historical cyclical oscillation trend, thus detecting an anomaly. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0068] In response to the problems existing in the prior art, the present invention provides a method, system and terminal for abnormality detection and root cause analysis of industrial equipment. The present invention is described in detail below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, the method for abnormality detection and root cause analysis for industrial equipment provided by an embodiment of the present invention includes the following steps:

[0070] S101 uses maximum and minimum normalization to process data, and uses sliding windows and masks to enhance multidimensional time series data to combat network fluctuations and industrial noise.

[0071] S102, using a hierarchical variational autoencoder neural network based on an attention mechanism to reconstruct multidimensional time series data.

[0072] S103, calculating anomaly scores using the multi-dimensional reconstructed residual matrix.

[0073] S104: Automatically set an abnormality threshold using extreme value theory.

[0074] S105: Calculate the anomaly score of the real-time online data, compare it with the threshold, and obtain anomaly results for anomaly detection and root cause analysis.

[0075] As a preferred embodiment, the method for abnormality detection and root cause analysis for industrial equipment provided by the embodiment of the present invention includes the following steps:

[0076] Step 1: After receiving real-time data, the anomaly detection platform uses PyTorch to train a model. Model training requires historical data from industrial equipment operations. The platform can provide a historical data query interface according to agreed-upon specifications. The anomaly detection platform uses this interface to obtain historical data and perform model training. If this interface is not available, the anomaly detection platform accumulates training datasets based on data in the message queue.

[0077] Step 2: Upload the trained model to the HDFS large file system, then use this neural network model to detect anomalies in the multi-dimensional real-time data stream uploaded by industrial equipment and perform root cause analysis on the indicators within its detection window.

[0078] Step 3: Push the results of the abnormality levels and root cause analysis to different equipment management personnel and maintenance personnel, and then check the operating status of the abnormal equipment and perform repairs to reduce the company's losses.

[0079] Step 4: Integrate this anomaly detection and root cause analysis model into the anomaly detection module of the IoT platform to improve its intelligence. Through the IoT platform, quickly and in real time obtain the operating data of industrial equipment. Then use the anomaly detection module to quickly detect anomalies and analyze the root causes of the equipment, and observe the health status of the equipment in real time.

[0080] 1. Hierarchical Variational Autoencoder Neural Network Based on Attention Mechanism

[0081] The embodiment of the present invention provides a hierarchical variational autoencoder neural network based on the attention mechanism, whose structure is divided into two parts, one is the encoder network and the other is the decoder network. Figure 2 shown.

[0082] Encoder network: By modeling multi-dimensional time series data, a time series matrix is ​​constructed such as Figure 3 As shown, the time sliding window is divided and input into the encoder-decoder network, which goes through three stages, namely the convolution layer and the ReLu activation function to process the data in the time dimension. While encoding the data, the activation function introduces nonlinear features, correlation features between indicators, and correlation features in the time dimension into the model.

[0083] Decoder Network: The decoder network must reconstruct the data using the spatiotemporal characteristics of multidimensional time series data. To minimize the error between the reconstructed data and the original data, the generated data must also be random. Therefore, a VAE probabilistic model is used to construct the decoder network.

[0084] Online detection: The multidimensional time series matrix generated by industrial equipment is input into a hierarchical variational autoencoder neural network based on the attention mechanism to generate a reconstructed time series matrix. The matrix difference is then calculated. The larger the difference, the greater the degree of abnormality, ultimately performing accurate anomaly detection.

[0085] Definition of anomaly detection and root cause analysis of industrial multi-dimensional time series data provided by the embodiment of the present invention:

[0086] like Figure 2As shown, the neural network model consists of two parts. The first part uses Bayesian inference to construct an encoder network to obtain its latent temporal and spatial dimensional features. The decoder network then constructs a reconstructed time series matrix within the time window. This reconstructed matrix enables real-time monitoring of device status and accurate anomaly detection. Once anomaly detection is complete, the reconstructed matrix can be used to perform root cause analysis of the anomaly, even when many industrial multi-dimensional time series indicators are difficult to accurately analyze.

[0087] 1) R M×N′ ←Conv(R M×N ): Use 1×1 convolution and Relu activation function to process data in the time dimension. While encoding the data, the activation function introduces nonlinear features to the model.

[0088] 2) R M×N″ ←MultiHead(R M×N′ To infer spatial random latent variables, a temporal spatial self-attention mechanism is proposed to learn the inter-indicator dependencies. Attention mechanisms are often used to learn complex dependencies and important features of data, improving the model's focus on input data and its performance.

[0089] 3) R M‘×N ←LSTM(Conv1d(M×N”)): In order to restore the time dimension of feature encoding, 1×1 convolution is used for processing, and deep LSTM is used to construct the time features of multi-dimensional time series data.

[0090] 4)P θ (x|s1, s2): Obtain the reconstruction matrix through the decoder network and maximize ELBO to obtain the final training goal;

[0091] 5)X res ←||X-X'||2: When obtaining online multidimensional time series data for industrial production, the severity of the anomaly is calculated by solving the residual matrix. After obtaining the matrix of the anomaly window, the degree of deviation of single indicators is compared and the degree of data mutation is measured using the derivative matrix. The root cause of the anomaly is analyzed and an anomaly notification is issued.

[0092] 6) By integrating anomaly detection and root cause analysis modules into the IoT platform, the IoT platform can be empowered to quickly access the status of large-scale equipment in real time and perform intelligent monitoring of the equipment. It can also detect in real time whether the equipment is operating abnormally and perform root cause analysis of the anomaly, reducing human analysis costs and improving corporate benefits.

[0093] The neural network-based industrial equipment anomaly detection algorithm process provided by the embodiment of the present invention is as follows:

[0094] 1) sensor → rowdata, the sensor uploads data to different protocol parsing layers of the IoT platform through different IoT protocols, such as MQTT, ModbusTcp or nb-iot, and obtains unprocessed data rowdata.

[0095] 2) rowdata → msg{"hum":10,"at":16710046,"sn":"1028"}. The protocol adaptation layer converts rowdata into an internally recognizable triple structure, where sn represents the device, node represents the acquisition card, attribute represents the sensor data, and at represents the timestamp, and uploads it to the message queue.

[0096] 3) msg ​​→ InfluxDB. The IoT platform's data center data processing module cleans and verifies the msg data and then stores it in various databases, such as time series databases or document databases.

[0097] 4) InfluxDB → TensorX: Preprocesses the data in the time series database into a two-dimensional tensor that can be processed by the model.

[0098] 5) tensorX→X', tensorX is reconstructed through a multi-channel variational long-term short-term network and a generative network to obtain a reconstructed matrix, which can be used to detect whether the device is operating normally. Figure 6 shown.

[0099] 6)X res ←||X-X'||2: Using X res The residual matrix can be used to obtain the size of the offset value. By measuring the offset value using different methods, it can be determined whether the indicator has an anomaly. By fusing the derivative matrix, the indicator anomaly score can be obtained. The indicators with the largest anomaly scores can be used to perform root cause analysis on the equipment anomaly, and ultimately obtain the root cause analysis results.

[0100] 7) SendInfo(s, d, title, subject, content): s represents the source address of the message, d represents the destination address, title represents the title, subject represents the subject, and content represents the specific content. When the IoT platform detects an anomaly, it uses this method to send a notification to the device administrator, allowing them to address the anomaly.

[0101] 8) Update(name, phone, subject, content, deviceid): name represents the maintenance technician's name, phone represents the phone number, subject represents the subject, content represents the repair details, and deviceid represents the device ID. After the maintenance technician repairs the device, they can report the cause of the fault and whether it is a false alarm, so that others who encounter the same abnormality can review it.

[0102] The anomaly detection and root cause analysis system for industrial equipment provided by an embodiment of the present invention includes:

[0103] The data acquisition module first defines the industrial scenario to which the data belongs, the protocol required for data transmission, and the sensor type. Protocols include MQTT, Modbus TCP, and private custom protocols. Sensor types include temperature sensors, humidity sensors, voltage and current sensors, and so on.

[0104] The model building module is used to build a hierarchical variational autoencoder neural network model based on the attention mechanism, and pass the preprocessed data into the neural network model to train the neural network model;

[0105] Anomaly detection module, which uses trained long-term and short-term neural network models to detect anomalies in industrial multi-dimensional time series data and monitor equipment operating status in real time;

[0106] The root cause analysis module is used to perform root cause analysis and alarm notification on abnormal data using a multi-dimensional reconstruction matrix, and then locate the abnormal indicators from the multi-dimensional data.

[0107] The architecture diagram of the Internet of Things platform integrating the anomaly detection module provided by the embodiment of the present invention is as follows: Figure 7 As shown in the figure, industrial equipment collects operational data through sensors, processes the raw data through the data receiving and parsing layers, and transmits it to the Kafka message middleware. The data processing center receives the data and transfers it to different databases. The anomaly detection module can obtain historical data from the database and online data from Kafka for model training and online detection. Figure 4As shown. If an anomaly is detected, a message can be sent to the equipment maintenance personnel. If the equipment does have an anomaly, it will be marked, and the cause of the anomaly and the steps to solve the anomaly will be written down so that the next time the same anomaly occurs, a comparison can be made to speed up the equipment maintenance. If a false alarm occurs, the data will be taken back into the network framework for learning in order to learn the normal mode of the equipment indicators and avoid the same false alarm next time. Through continuous learning, the accuracy of the model is increased. The Internet of Things platform of the present invention can quickly detect equipment anomalies and perform root cause analysis by intelligently integrating its anomaly detection module and equipment anomaly root cause analysis module, thereby reducing enterprise losses and speeding up equipment maintenance.

[0108] The solution provided by this embodiment not only enables rapid access to IoT devices through the protocol access layer but also converts the protocol into an internal protocol through the protocol adaptation layer. This allows access to large amounts of IoT data through the Kafka message queue, and allows for customized data display using a time series database and Grafana. Data collected by Kafka can also be sent to the anomaly detection module and root cause analysis module for real-time monitoring of device status.

[0109] 1. Install sensors on large industrial equipment. The sensors collect data in real time and access the relevant protocol access layer of the IoT platform through various network protocols. The protocol access layer can push data to the protocol adaptation layer to convert the data into an internal data format and transmit it to Kafka. The data is then cleaned, verified, and stored.

[0110] 2. The attention-based hierarchical variational autoencoder neural network is trained using data obtained from the IoT platform. The model can accurately detect the degree of device anomalies by identifying network fluctuation characteristics, temporal and spatial characteristics.

[0111] 3. Different abnormality levels are classified and sent to different equipment managers or maintenance personnel, including email, SMS, phone and station message notification.

[0112] 4. When equipment maintenance personnel receive equipment anomaly information, they can further view the root cause analysis results in the IoT root cause analysis module. The root cause analysis module can provide a more accurate analysis report by fusing indicator derivative values ​​and residual matrices and using attenuation factors to assist equipment management personnel in repairing the equipment.

[0113] 5. Equipment maintenance personnel can focus on checking the modules that are most likely to cause equipment abnormalities based on the root cause analysis report, thereby speeding up equipment maintenance.

[0114] 6. After the equipment maintenance personnel have checked the real cause of the equipment abnormality, they will upload the maintenance report and fill in whether there is a false alarm in the equipment maintenance module of the IoT platform. The next time the same abnormality occurs, the historical maintenance steps can be used to repair the equipment.

[0115] To demonstrate that the method proposed in this embodiment can effectively capture the temporal and spatial dependencies of multidimensional time series data and thus perform effective anomaly detection, a visualization of anomaly detection cases is presented. These include point anomalies, contextual time-dependent anomalies, inter-indicator anomalies, and periodic anomalies in industrial multidimensional time series datasets.

[0116] from Figure 8 Figure (a) shows that the indicator for dimension 15 of the multidimensional time series data has a small anomaly, with its value far outside the normal range. Figure (b) shows that dimensions 1 and 3 do not conform to historical trends, and the contextual time-dependent anomaly formed within the interval is successfully detected.

[0117] from Figure 8 (c) It can be seen that the two indicators of dimension 7 and 15 are normally positively correlated, but in other areas the two indicators are negatively correlated, and anomalies between the indicators are detected. Figure 8 (d) It can be seen that the data shows a trend of cyclical oscillation, but in other abnormal areas, violations of the historical cyclical oscillation trend occur, thus detecting anomalies.

[0118] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0119] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for anomaly detection and root cause analysis of industrial multidimensional time series data, characterized in that: include: The data is standardized and processed by sliding windows through the data preprocessing module; Build a neural network model and pass preprocessed data into the neural network model to train the neural network model. Use the trained neural network model to perform online detection of multi-dimensional time series data of industrial equipment, detect the equipment operating status in real time, and perform root cause analysis and alarm notification for anomalies, thereby locating abnormal indicators from multi-dimensional data. The method specifically comprises the following steps: Step 1: To uniformly represent the values ​​of each sensor, we use maximum and minimum normalization to process the data, and use sliding windows and masks to enhance the multi-dimensional time series data to combat network fluctuations and industrial noise. Step 2: Use the hierarchical variational autoencoder neural network based on the attention mechanism to reconstruct the multi-dimensional time series data; Step 3: Calculate the anomaly score using the multi-dimensional reconstructed residual matrix; Step 4: Automatically set the abnormal threshold using extreme value theory; Step 5: Calculate the anomaly score of the real-time online data, compare it with the threshold, and obtain the anomaly result for root cause analysis. Multi-dimensional time series data anomaly detection includes: (1) sensor → rowdata, the sensor uploads data to different protocol parsing layers of the IoT platform through different IoT protocols such as MQTT, ModbusTcp or nb-iot, and obtains unprocessed data rowdata; (2) rowdata→msg{"hum":10,"at":16710046,"sn":"1028"}: The protocol adaptation layer converts rowdata into an internally recognizable triple structure; where the scene number sn represents the device, node represents the acquisition card, attribute represents the sensor data, and at represents the timestamp uploaded to the message queue; (3) msg ​​→ influxdb, the data center data processing module of the IoT platform cleans the msg data, verifies it, and stores it in various databases, including time series databases or document databases; (4) influxdb→tensorX: preprocesses the data in the time series database into a two-dimensional tensor that can be processed by the model; (5)tensorX→X ‘ , tensorX obtains the reconstruction matrix through the multi-channel variational long-term and short-term network and the generative network, and uses the reconstruction matrix to detect whether the device is operating normally; (6)X res ←||X-X'||2: Using X res The residual matrix obtains the offset value, and the offset value is measured using different methods to determine whether the indicator has an abnormality. The derivative matrix is ​​fused to obtain the indicator anomaly score. The indicator with the largest anomaly score is used to perform root cause analysis on the device anomaly and obtain the root cause analysis results. (7) SendInfo(s, d, title, subject, content): s represents the source address of the message, d represents the destination address, title represents the title, subject represents the subject, and content represents the specific content. When the IoT platform detects an abnormality, it sends a notification to the device manager to handle the abnormality. (8)Update(name,phone,subject,content,deviceid): name represents the name of the equipment maintenance personnel, phone represents the phone number, subject represents the subject, content represents the maintenance content, and deviceid represents the device ID; after the equipment maintenance personnel repairs the equipment, they report the cause of the equipment failure and whether it is a false alarm, so that the same abnormal information can be checked later.

2. The anomaly detection and root cause analysis method for industrial multi-dimensional time series data according to claim 1, characterized in that: Anomaly detection and root cause analysis methods for industrial multi-dimensional time series data also include: (1) After receiving real-time data, the anomaly detection platform uses pytorch to train the model. Before model training, it obtains historical data of industrial equipment operation. The platform provides a historical data query interface according to the agreed specifications. The anomaly detection platform obtains historical data through the interface and performs model training. If there is no interface, the anomaly detection platform accumulates training data sets based on the data in the message queue; (2) Upload the trained model to the HDFS large file system, use the neural network model to detect anomalies in the multi-dimensional real-time data stream uploaded by the smart sensor, and perform root cause analysis on the indicators within the detection window; (3) According to different abnormality levels and root cause analysis results, they are pushed to different equipment management personnel and maintenance personnel to check the operating status of abnormal equipment and perform repairs; (4) Integrate the anomaly detection and root cause analysis model into the anomaly detection module of the Internet of Things platform, obtain the operating data of industrial equipment in real time through the Internet of Things platform, and then use the anomaly detection module to perform anomaly detection and root cause analysis on the equipment, and observe the operating status of industrial production in real time.

3. The anomaly detection and root cause analysis method for industrial multi-dimensional time series data according to claim 1, characterized in that: The hierarchical variational autoencoder neural network based on the attention mechanism in step 2; The encoder network is used to construct the posterior distribution q φ (s2, s1|x), where two latent variables are used to represent the potential feature space of multidimensional time series data. s1 represents a spatial random latent variable, and its spatial features are captured using the attention mechanism. s2 represents a temporal random latent variable, and LSTM is used to capture its temporal features; In the decoder network, the temporal and spatial characteristics of multi-dimensional time series data need to be used to reconstruct the data. In order to minimize the reconstruction error between the reconstructed data and the original data, the generated data must also be random. Therefore, the VAE probability model is used to construct its decoder network. Online detection is used to input industrial multi-dimensional time series data into a hierarchical variational autoencoder based on the attention mechanism, output the reconstructed time series matrix, calculate the matrix difference, and finally perform anomaly detection.

4. The anomaly detection and root cause analysis method for industrial multi-dimensional time series data according to claim 1, characterized in that: The root cause analysis of abnormal data using a multi-dimensional reconstruction matrix in step 5 includes: using Bayesian inference to construct a multi-channel variational network and obtain potential temporal and spatial dimensional features; using a generative network to construct a reconstructed time series matrix within the time window, and using the reconstruction matrix to perform real-time anomaly detection on the device status; after anomaly detection is completed, the reconstruction matrix is ​​used to perform root cause analysis on the abnormal indicators, specifically including: (1)R M×N' ←Conv(R M×N ): Use 1×1 convolution and ReLU activation function to process data in the time dimension. While encoding the data, the activation function introduces nonlinear features into the model. (2)R M×N” ←MultiHead(R M×N' ): In order to infer spatial random latent variables, a temporal spatial self-attention mechanism is proposed to learn the correlation and dependency features between its indicators; the attention mechanism is often used to learn the complex correlation and dependency relationships and important features of data to improve the model's attention and performance on the input data; (3) R M‘×N ←LSTM(Conv1d(M×N”)): In order to restore the time dimension of feature encoding, 1×1 convolution is used for processing, and deep LSTM is used to construct the time features of multi-dimensional time series data (4)P θ (x|s1, s2): The reconstruction matrix is ​​obtained by generating the network, and the ELBO is maximized to obtain the final training goal; (5)X res ←||X-X'||2: When acquiring online multi-dimensional time series data for industrial production, the severity of the anomaly is calculated by solving the residual matrix. After obtaining the matrix of the anomaly window, the degree of deviation of the single indicator is compared and the degree of data mutation is measured using the derivative matrix. The root cause of the anomaly is analyzed and an anomaly notification is issued. (6) Integrate the anomaly detection and root cause analysis modules into the IoT platform, access the status of large-scale equipment in real time and conduct intelligent monitoring of the equipment, detect whether the equipment is operating abnormally in real time and conduct root cause analysis of the anomaly.

5. An anomaly detection and root cause analysis system for industrial multi-dimensional time series data based on an autoencoder neural network according to any one of claims 1 to 4, characterized in that: The anomaly detection and root cause analysis system for industrial multi-dimensional time series data includes: The data acquisition module first defines the industrial scenario to which the data belongs, the protocol required for data transmission, and the sensor type; the protocols include MQTT protocol, ModbusTcp protocol, and private custom protocols, and the sensor types include temperature sensor, humidity sensor, and voltage and current sensor; The model building module is used to build a hierarchical variational autoencoder neural network model based on the attention mechanism, and pass the preprocessed data into the neural network model to train the neural network model; The anomaly detection module is used to detect anomalies in industrial multi-dimensional time series data using a trained autoencoder neural network model and monitor the operating status of equipment in real time; The root cause analysis module is used to perform root cause analysis and alarm notification on abnormal data using a multi-dimensional reconstruction matrix, and then locate the abnormal indicators from the multi-dimensional data.

6. A computer device, characterized in that: The computer device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor performs the steps of the anomaly detection and root cause analysis method for industrial multidimensional time series data according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the anomaly detection and root cause analysis method for industrial multi-dimensional time series data according to any one of claims 1 to 4.

8. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the industrial anomaly detection and root cause analysis system as claimed in claim 5.

Images