Data anomaly detection method based on intelligent industry, storage medium and terminal

Abstract

The application discloses a data anomaly detection method based on intelligent industry, a storage medium and a terminal, belongs to the technical field of anomaly detection, and various industrial data are processed by using an improved KF algorithm and then input into a first neural network model for anomaly prediction. The improved KF algorithm specifically learns the change trend of the covariance matrix by using a second neural network model, and then updates the covariance matrix. The improved KF algorithm can dynamically adjust the process excitation noise covariance matrix Q, thereby reducing the influence of noise on the KF algorithm, and then ensuring the anomaly prediction accuracy and reliability of the subsequent neural network model. Meanwhile, the improved KF algorithm can process the industrial data, eliminate redundant data, uniformly process each information source, and thus ensure the measurement accuracy. Meanwhile, the improved KF algorithm can also determine the time sequence characteristics of the industrial data stream, and thus the method can accurately detect the data stream without time sequence characteristics.

Description

Technical Field

[0001] This invention relates to the field of anomaly detection technology, and in particular to a data anomaly detection method, storage medium, and terminal based on smart industry. Background Technology

[0002] With the application of technologies such as industrial big data and the Industrial Internet of Things, the industrial sector has begun its digital transformation, and countries around the world have introduced corresponding development strategies. In industrial activities, sensors generate massive amounts of data in a short period of time, making it difficult for humans to analyze potential anomalies in equipment based on this vast amount of sensor data.

[0003] In recent years, numerous researchers have explored machine learning methods to extract features from data streams for anomaly detection. Based on the modeling type used for anomaly detection, data stream anomaly detection methods can be categorized into three types: statistical modeling methods, temporal feature modeling methods, and spatial feature modeling methods. According to whether the anomaly detection algorithm requires manually labeled datasets, it can be divided into supervised learning anomaly detection algorithms and unsupervised learning anomaly detection algorithms.

[0004] Existing technologies utilize the K-means clustering algorithm to detect intrusion anomalies in wireless sensor networks (WSNs). Specifically, by analyzing the statistical characteristics of the data stream, cluster centers for normal and anomalous data are calculated, and anomaly detection is performed by calculating the Euclidean distance between the WSN data and the cluster centers. This method achieves high detection accuracy for specific types of intrusion anomalies. Existing technologies also propose combining the SOM algorithm with K-means clustering to achieve online anomaly detection. This algorithm reconstructs anomalous clusters or splits normal clusters by real-time updating the network structure and utilizing the fitness of new data, thus enabling the detection of novel anomalies. Both of these algorithms have high detection accuracy when detecting statistical anomalies and do not require pre-labeled datasets, making them unsupervised anomaly detection algorithms. However, these algorithms also have some drawbacks:

[0005] 1. It is easily affected by noise points and isolated points. When calculating the center point of each cluster, the calculated mean will have a large error compared with the actual mean.

[0006] 2. The K value in the algorithm is not easy to obtain. It is usually selected by personal experience or obtained by using the sum of squared errors and silhouette coefficient method. However, these methods often cannot obtain the true number of clusters, which reduces the clustering effect.

[0007] 3. It is only applicable to datasets with convex clusters and numerical data. For other datasets, the performance still needs to be improved.

[0008] 4. The selection of initial cluster centers has a significant impact on the final classification result. If the initial cluster centers are not selected properly, clustering may fail.

[0009] Unlike the clustering algorithms mentioned above, existing technologies also propose applying Long Short-Term Memory (LSTM) algorithms to anomaly detection in industrial IoT sensor data streams. Specifically, LSTM models are used to predict future data, and anomalies are detected by analyzing the error between the predicted and actual values. Furthermore, existing technologies also propose using LSTM models to extract temporal features from industrial control system data and using Softmax to classify anomalies. These methods utilize RNNs to extract temporal features from data streams, predict future data using prediction algorithms, and detect anomalies by analyzing the error between the predicted and actual values; or they use classification algorithms to detect anomalies. These algorithms achieve good detection accuracy for data streams with definite temporal features, but fail to accurately detect data streams without definite temporal features, and their detection performance still needs further improvement.

[0010] In addition, in industrial settings, the lack of consistent information sources due to issues with various data acquisition devices, such as sensors, can lead to different results for measurements of the same unit, reducing the accuracy of subsequent anomaly detection. Summary of the Invention

[0011] The purpose of this invention is to overcome the problems of the prior art and provide a data anomaly detection method, storage medium and terminal based on smart industry.

[0012] The objective of this invention is achieved through the following technical solution: a data anomaly detection method based on smart industry, the method comprising the following steps:

[0013] Various industrial data are processed by Kalman filtering and then input into the first neural network model for anomaly prediction. The covariance matrix update of the Kalman filtering process includes:

[0014] The second neural network model learns the changing trend of the covariance matrix of the Kalman filter algorithm, and then updates the covariance matrix.

[0015] In one example, the covariance matrix The update formula is:

[0016]

[0017] Among them, the function f(·) learns the changing trend of the fitted covariance matrix through the second neural network model; P k-1 Let represent the covariance matrix at the previous time step.

[0018] In one example, the changing trend of the covariance matrix of the learned Kalman filter algorithm includes:

[0019] Based on the state at time k-1 Predict the state at time k ;

[0020] The error covariance matrix at time k-1 Introducing intermediate variables ;

[0021] Through intermediate variables Update Kalman gain coefficient ;

[0022] According to the Kalman gain coefficient Obtain the optimal state estimate at time k ;

[0023] Update the covariance matrix at time k .

[0024] In one example, the step of learning the changing trend of the covariance matrix of the Kalman filter algorithm through the second neural network model further includes:

[0025] The state transition matrix, process excitation noise covariance matrix, and initial covariance matrix are used as inputs to train the second neural network model, thereby obtaining a second neural network model that can predict and update the covariance matrix.

[0026] In one example, the first neural network model is a gated recurrent neural network that applies an attention mechanism, or a long short-term memory network that applies an attention mechanism.

[0027] In one example, when the gated recurrent neural network applying the attention mechanism makes predictions, it includes:

[0028] An attention mechanism is used to perform dot product processing on the output of each hidden unit in the gated recurrent neural network and the weight matrix.

[0029] In one example, the second neural network model is a gated recurrent neural network or a long short-term memory network.

[0030] In one example, the process of performing Kalman filtering on the collected industrial data further includes:

[0031] Perform outlier handling, missing value handling, and normalization on industrial data.

[0032] It should be further noted that the technical features corresponding to the above examples can be combined or substituted to form new technical solutions.

[0033] The present invention also includes a storage medium storing computer instructions thereon, which, when executed, perform the steps of the data anomaly detection method based on smart industry formed by any or more of the above examples.

[0034] The present invention also includes a terminal comprising a memory and a processor, wherein the memory stores computer instructions executable on the processor, and the processor executes the steps of the data anomaly detection method based on smart industry formed by any or more of the above examples when executing the computer instructions.

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] This invention captures the changing trend of the covariance matrix of the Kalman filter algorithm through a first neural network model, which can dynamically adjust the process excitation noise covariance matrix Q, making the updated covariance matrix P closer to the true value. This reduces the impact of noise on the Kalman filter algorithm, improves the accuracy of Kalman filtering, and thus ensures the accuracy and reliability of anomaly prediction in the subsequent neural network model. Simultaneously, the improved Kalman filter algorithm is used to process industrial data, eliminating redundant data and uniformly processing various information sources to ensure measurement accuracy. Furthermore, the improved Kalman filter algorithm can determine the temporal characteristics of industrial data streams, enabling this method to accurately detect anomalies in data streams that lack temporal characteristics. Attached Figure Description

[0037] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, which are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to denote the same or similar parts. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application.

[0038] Figure 1 This is a flowchart of an anomaly detection method in an example of the present invention;

[0039] Figure 2 This is a structural diagram of the improved KF algorithm in one example of the present invention;

[0040] Figure 3 This is a structural diagram of a preferred example of anomaly detection method of the present invention;

[0041] Figure 4 The flowchart is a preferred example of the anomaly detection method of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] In the description of this invention, it should be noted that the directions or positional relationships indicated by terms such as "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" are based on the directions or positional relationships shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. Furthermore, the use of ordinal numbers (e.g., "first and second," "first to fourth," etc.) is for distinguishing objects and is not limited to this order, and should not be construed as indicating or implying relative importance.

[0044] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0045] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0046] In one example, such as Figure 1 As shown, a data anomaly detection method based on smart industry is proposed, which specifically includes the following steps:

[0047] S1': The covariance matrix of the Kalman filter algorithm is learned through the second neural network model, and then the covariance matrix P is updated to improve the KF algorithm (Kalman filter algorithm). The improved KF algorithm is then used to process various industrial data to complete the local feature extraction.

[0048] S2': Input the industrial data after Kalman filtering into the first neural network model to complete the data fusion of multiple data, further explore the relationship between the data, and thus achieve anomaly prediction.

[0049] Specifically, in step S1, the second neural network model can be a gated recurrent neural network (GRU network) or a long short-term memory network (LSTM network), and in this example, a GRU network is preferred. Further, data collected by different data acquisition devices, such as sensors, are treated as industrial data, including temperature data, air pressure data, flow rate data, etc. In this step, various industrial data can be of the same type, such as all being temperature data, the difference being that they are collected by different data acquisition devices, and can be temperature data from different devices or different parts of the same device, etc.

[0050] Furthermore, the Kalman filter algorithm is an existing algorithm that includes a prediction part and an update part. The specific data processing formula for the prediction part includes:

[0051]

[0052]

[0053] in, This represents the estimated state at time k; A represents the estimated state at time k-1; B represents the state transition matrix; and C represents the control matrix. P represents the input at time k; P represents the noise covariance matrix; P k-1 Represents the covariance matrix at the previous time step; A represents the intermediate variable of the covariance matrix; Q represents the process excitation noise covariance matrix. As can be seen from formula (2), the update equation of the covariance matrix P is determined by the state transition matrix A and the process noise matrix Q, and the selection of the noise matrix Q has a great influence on the state update of Kalman filtering.

[0054] The specific data processing formulas for the state update section include:

[0055]

[0056]

[0057]

[0058] in, Indicates the Kalman coefficient; C represents the covariance matrix at time k; T represents the observation matrix; R represents the matrix transpose; and R represents the observation noise covariance matrix. I represents the observable; I represents the identity matrix.

[0059] The Kalman filter (KF) algorithm considers the influence of noise during inference, which is passed to the next state value through the covariance matrix P. Accurately updating the covariance matrix P improves filtering accuracy and prevents filter divergence. In this invention, the improved KF algorithm utilizes the powerful representational ability of a neural network model to accurately capture state changes in the covariance matrix. Specifically, it uses the neural network model to capture the changing trend of the covariance matrix in the KF algorithm, dynamically adjusting the process excitation noise covariance matrix Q (in existing KF algorithms, Q is an empirically fixed value). This makes the updated covariance matrix P closer to the true value, thereby reducing the impact of noise on the KF algorithm, improving the accuracy of Kalman filtering, and ensuring the accuracy and reliability of subsequent anomaly predictions by the neural network model. Furthermore, this invention's method combines the improved KF algorithm with a neural network model for anomaly detection. Since it does not employ a clustering algorithm, it avoids the drawbacks of existing unsupervised anomaly detection algorithms based on clustering. Moreover, Kalman filtering is friendly to initial values; typically, initial values ​​can be randomly selected or measured values ​​at the initial time point can be used. Even if there are deviations, they will be rapidly reduced through gradual iteration. During the iterative process, the Kalman filter considers both observation noise and system noise, and combines the observations with the estimates to obtain the optimal estimate, making it less susceptible to noise and isolated points.

[0060] Furthermore, this invention employs an improved KF algorithm to process industrial data, which can eliminate redundant data and perform unified processing on various information sources, thereby ensuring measurement accuracy. At the same time, the improved KF algorithm can also determine the temporal characteristics of industrial data streams, thus enabling this method to accurately detect anomalies in data streams that do not have temporal characteristics.

[0061] In one example, the covariance matrix The update formula is:

[0062]

[0063] The function f(·) learns the changing trend of the fitted covariance matrix through the second neural network model.

[0064] In one example, such as Figure 2 As shown, the changing trends of the covariance matrix of the Kalman filter algorithm learned using the second neural network model include:

[0065] S11': Based on the state at time k-1 Predict the state at time k Specifically, the prediction is based on the above formula (1);

[0066] S12': The error covariance matrix at time k-1 Introducing intermediate variables The derivation is based on the above formula (6);

[0067] S13': Through intermediate variables Update Kalman gain coefficient Specifically, the update is based on the above formula (3);

[0068] S14': Based on the Kalman gain coefficient Obtain the optimal state estimate at time k Specifically, the processing is based on the above formula (4);

[0069] S15': Update the covariance matrix at time k Specifically, the update is based on the above formula (5). It should be further explained that... Figure 2 In Chinese, "predict" means to predict and "update" means to update.

[0070] In one example, before learning the trend of the covariance matrix of the Kalman filter algorithm through the second neural network model, the following steps are also included:

[0071] The state transition matrix A, the process excitation noise covariance matrix Q, and the initial covariance matrix P are used as inputs to train the second neural network model, thereby obtaining a second neural network model that can predict and update the covariance matrix. Based on the nonlinear processing capability of the second neural network model, the process excitation noise covariance matrix Q is dynamically adjusted.

[0072] In one example, the first neural network model is a gated recurrent neural network (GRU_Attention) that applies an attention mechanism. Alternatively, a long short-term memory (LSTM_Attention) network that applies an attention mechanism can also be used.

[0073] In one example, the process of inputting the industrial data after Kalman filtering into the first neural network model also includes:

[0074] S02': Constructing the GRU Attention Network. Specifically, industrial data has strong temporal characteristics, and the accurate extraction of temporal features is crucial for accurate prediction of future data. Therefore, this invention employs a GRU network capable of processing temporal data and applies an Attention mechanism based on GRU. The output of each hidden unit is multiplied by a correlation weight matrix to obtain more complete temporal features, further ensuring the accuracy of anomaly detection in data streams without temporal features and improving the network's anomaly detection performance. The number of hidden layer units in the GRU is set to g based on the length of the sample data sequence. This invention uses a fully connected layer with a Softmax activation function to implement the Attention mechanism. Its input is the output of each unit in the GRU hidden layer, and its output is the correlation weight between the output of each hidden layer and the final predicted target.

[0075] The specific implementation of extracting temporal features from data using the GRU_Attention network is as follows:

[0076] Each repeating module in a GRU network contains an update gate and a reset gate. The input to the reset gate is the hidden state h from the previous time step. t-1 and the input X at the current moment t After merging, we get [h] t-1 X t After multiplying by the weights, the result is re-inputted into the sigmoid function to obtain a vector r between 0 and 1. t The formula is as follows:

[0077]

[0078] r t This determines which information from the hidden state received in the previous time step needs to be discarded and which needs to be retained. A value of 1 indicates that all hidden state information is retained. Calculate the candidate hidden states. The formula is as follows:

[0079]

[0080] Here, tanh() is the hyperbolic activation function, and W is the weight parameter. The update gate controls the state variable h of the previous GRU unit. t-1 and the vectors newly entering the GRU unit For the new state vector h t The degree of influence is mainly determined by the following two formulas:

[0081]

[0082]

[0083] Among them W z To update the gate parameters, the hidden state h at different time steps is obtained using GRU. n This is used as the input to the Attention mechanism layer. The weight coefficients of the Attention mechanism layer are calculated using the following formulas:

[0084]

[0085]

[0086]

[0087] Among them, e t e j This represents the value of the attention probability distribution determined by the GRU output vector at times t and j, where u and w are weight coefficients, b is the bias coefficient, and s is the weight coefficient. t This represents the output of the attention layer at time t. The formula for the output layer is as follows:

[0088]

[0089] Where w represents the weight matrix of the fully connected layer, b1 represents the bias of the fully connected layer, and y t The predicted label is output, and the current anomaly detection result is obtained.

[0090] In one example, data preprocessing is included before the collected industrial data is subjected to Kalman filtering:

[0091] S0': Outlier handling, missing value handling, and normalization are performed on industrial data. Specifically, measured values ​​deviating from the mean by more than three standard deviations are considered highly outliers. Outlier handling involves removing outliers to ensure the reliability of the data stream. Missing value handling involves filling in missing data, which can be done using historical values ​​under the same conditions or the average of adjacent data. Normalization converts the data stream into a scalar for easier subsequent data processing.

[0092] Combining the above examples yields preferred examples of the present invention, such as... Figure 3-4 As shown, the anomaly detection method at this time includes the following steps:

[0093] S1: Collect industrial sample data;

[0094] S2: Perform outlier handling, missing value handling, and normalization on industrial data;

[0095] S3: Input the various preprocessed data into the improved Kalman filter for further processing;

[0096] S4: Construct the GRU_Attention network;

[0097] S5: The different data that have undergone Kalman filtering are concatenated and input into the GRU_Attention network for anomaly prediction;

[0098] S6: Perform inverse normalization transformation on the predicted data to obtain the final anomaly detection result.

[0099] Of course, as an option, the construction of the GRU_Attention network in S4 can also be performed before step S3.

[0100] This application also includes a storage medium having the same inventive concept as the data anomaly detection method based on smart industry composed of any or more of the above examples, wherein computer instructions are stored thereon, and the computer instructions execute the steps of the data anomaly detection method based on smart industry described above when they are run.

[0101] Based on this understanding, the technical solution of this embodiment, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] This application also includes a terminal having the same inventive concept as the data anomaly detection method based on smart industry comprised of any or more of the above examples, including a memory and a processor. The memory stores computer instructions executable on the processor, and the processor executes the steps of the data anomaly detection method based on smart industry when executing the computer instructions. The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0103] In the embodiments provided by the present invention, each functional unit can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0104] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. A data anomaly detection method based on smart industry, characterized in that: Includes the following steps: Various industrial data are processed by Kalman filtering and then input into the first neural network model for anomaly prediction. The covariance matrix update of the Kalman filtering process includes: By learning the changing trend of the covariance matrix of the Kalman filter algorithm through the second neural network model, the process excitation noise covariance matrix can be dynamically adjusted, and then the covariance matrix can be updated. The covariance matrix The update formula is: ； Among them, the function f(·) learns the changing trend of the fitted covariance matrix through the second neural network model; P k-1 Represents the covariance matrix at the previous time step; Before learning the trend of the covariance matrix of the Kalman filter algorithm through the second neural network model, the following steps are also included: The state transition matrix, process excitation noise covariance matrix, and initial covariance matrix are used as inputs to train the second neural network model, thereby obtaining a second neural network model that can predict and update the covariance matrix. The changing trends of the covariance matrix of the learned Kalman filter algorithm include: Based on the state at time k-1 Predict the state at time k ; The error covariance matrix at time k-1 Introducing intermediate variables ; Through intermediate variables Update Kalman gain coefficient ; According to the Kalman gain coefficient Obtain the optimal state estimate at time k ; Update the covariance matrix at time k ; The first neural network model is a gated recurrent neural network that applies an attention mechanism, or a long short-term memory network that applies an attention mechanism. When using gated recurrent neural networks with attention mechanisms for prediction, the following are included: An attention mechanism is used to perform dot product between the output of each hidden unit in the gated recurrent neural network and the weight matrix. In a gated recurrent neural network, each repetitive module contains an update gate and a reset gate; the input to the reset gate is the hidden state from the previous time step. and the input at the current moment After the merger, we obtained After multiplying by the weights, the result is fed back into the sigmoid function, yielding a vector between 0 and 1. The formula is as follows: ； This determines which information from the hidden state transmitted in the previous moment needs to be discarded and which needs to be retained; if A value of 1 indicates that all hidden state information is retained; calculate alternative hidden states. The formula is as follows: ； in, The function is a hyperbolic activation function, where W is the weight parameter; the update gate controls the state variables of the previous gated recurrent neural unit. and vectors of newly entered gated recurrent neural units For the new state vector The degree of influence is determined by the following two formulas: ； ； in To update the gate parameters, the hidden state h at different time points is obtained through a gated recurrent neural network. n This is used as the input to the Attention mechanism layer; the weight coefficients of the Attention mechanism layer are calculated using the following formulas: ； ； ； in, , This represents the value of the attention probability distribution determined by the GRU output vector at times t and j. and These are weighting coefficients. It is the bias coefficient. This represents the output of the attention layer at time t; the formula for the output layer is as follows: ； in, This represents the weight matrix of the fully connected layer. This indicates the offset of the fully connected layer. The predicted label is output, and the current anomaly detection result is obtained.

2. The data anomaly detection method based on smart industry according to claim 1, characterized in that: The second neural network model is a gated recurrent neural network or a long short-term memory network.

3. The data anomaly detection method based on smart industry according to claim 1, characterized in that: Before performing Kalman filtering on industrial data, the following steps are also included: Perform outlier handling, missing value handling, and normalization on industrial data.

4. A storage medium storing computer instructions thereon, characterized in that: When the computer instructions are executed, they perform the steps of the data anomaly detection method based on smart industry as described in any one of claims 1-3.

5. A terminal, comprising a memory and a processor, wherein the memory stores computer instructions executable on the processor, characterized in that: When the processor executes the computer instructions, it performs the steps of the data anomaly detection method based on smart industry as described in any one of claims 1-3.

Images