Anomaly detection model construction method, anomaly detection method, and anomaly detection system

Abstract

This invention relates to the construction method of an anomaly detection model, an anomaly detection method, and an anomaly detection system, within the field of machine learning, particularly concerning fault or anomaly detection in industrial systems. It addresses the shortcomings of existing anomaly detection models, such as weak ability to extract and analyze nonlinear, strongly temporally sequential, and high-dimensional data, as well as reliance on initial data distribution, personal experience, and the inability to dynamically adjust thresholds. The anomaly detection model is built based on the LSTM-AE framework and an adaptive threshold with automatically adjustable parameters; the anomaly detection method and system employ the anomaly detection model for anomaly detection. This invention is applied to anomaly detection in industrial systems.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to fault or anomaly detection in industrial systems. Background Technology

[0002] Modern industrial systems are becoming increasingly complex and integrated, facing challenges from the dynamic diversity of tasks and operating environments. This leads to increasingly prominent reliability and security issues. Meanwhile, with the development of low-cost sensing and communication technologies, modern industrial systems are typically equipped with various fault and anomaly monitoring and detection systems to monitor and record system status, performance, operating environment, and load parameters online.

[0003] Existing fault and anomaly monitoring and detection systems can generally be divided into the following two types:

[0004] 1. Data from both normal and abnormal operating states is collected using sensors. Mathematical models for both normal and abnormal states are then established using analytical modeling tools. The classification boundaries between these states are then identified to detect anomalies. Commonly used analytical modeling tools include SVM (Support Vector Machine), RF (Random Forest), and deep learning algorithms.

[0005] 2. Data is collected by sensors during normal system operation, and a mathematical model of the normal state is established using analytical modeling tools. This model identifies the boundaries of the data during normal operation or captures features that differentiate the normal operation from abnormal operation, thus enabling anomaly detection. Commonly used analytical modeling tools include PCA (Principal Component Analysis), One-Class SVM (Single-Class Support Vector Machine), and deep learning methods.

[0006] Compared to ordinary systems that require anomaly detection, industrial systems have the following characteristics: First, industrial systems are typically large and complex, and the data collected by sensors is often a large amount of nonlinear data; second, advancements in sensor technology have made data collection increasingly easier, leading to larger and more complex databases, and requiring analysis of increasingly higher-dimensional data; third, data collected in the industrial field also exhibits strong temporal correlation, which refers to the degree to which data occurs and changes over time; the higher the correlation, the stronger the temporal correlation.

[0007] When existing fault and anomaly monitoring and detection systems are applied to industrial systems with the above characteristics, the following problems arise:

[0008] 1. Existing anomaly detection models have weak feature extraction capabilities for nonlinear data, which reduces the accuracy of anomaly detection for nonlinear data in industrial systems;

[0009] 2. Existing anomaly detection models are insufficient in capturing features of time-series data, which reduces the accuracy of anomaly detection for time-series data in industrial systems.

[0010] 3. Existing anomaly detection models have weak analytical capabilities for high-dimensional data, which reduces the efficiency of anomaly detection in industrial systems;

[0011] 4. Existing anomaly detection models require the initial distribution of data as a parameter to determine the anomaly detection threshold. However, the initial distribution of data cannot be clearly determined. A common technique is to approximate the initial distribution of data as a Gaussian distribution. This results in a model with a certain degree of error, which reduces the accuracy of anomaly detection in industrial systems.

[0012] 5. Existing anomaly detection models require manual setting of some parameters when determining anomaly detection thresholds. The setting of these parameters depends on personal experience. Different initial parameters will lead to different data models, resulting in different anomaly detection results. Consequently, the accuracy of anomaly detection in industrial systems depends on manual inspection, and the results of different people's detections are inconsistent.

[0013] 6. Existing anomaly detection models commonly use the 3σ method or extreme value theory to determine anomaly detection thresholds. Both of these methods determine fixed thresholds, which cannot be dynamically adjusted according to changes in environmental parameters. This results in poor environmental adaptability for anomaly detection in industrial systems, and consequently, low accuracy in anomaly detection under different working environments.

[0014] 7. Existing anomaly detection methods or systems employ existing anomaly detection models for detecting operational anomalies in industrial systems. However, their poor data extraction and analysis capabilities lead to low accuracy of detection results. Furthermore, the reliance on human experience in setting anomaly detection threshold parameters results in poor accuracy and consistency of detection results. Additionally, the inability to adaptively adjust anomaly detection thresholds leads to poor adaptability of the detection methods or systems to different working environments. Summary of the Invention

[0015] The purpose of this invention is to propose a method for constructing an anomaly detection model, an anomaly detection method, and an anomaly detection system to solve the aforementioned technical problems existing in the prior art.

[0016] The above-mentioned technical problems of the present invention are mainly solved by the following technical solutions:

[0017] The technical solution of the anomaly detection model construction method described in this invention is as follows:

[0018] The method for constructing an anomaly detection model includes:

[0019] S1. Under the stable operating condition of the industrial system under test without any abnormalities, data of multiple state parameters of the industrial system under test are collected as raw training time series data samples. The set of raw training time series data samples is the raw training time series dataset of the industrial system under the condition of no abnormalities.

[0020] S2. Perform data preprocessing on the original training time series data samples to obtain training data samples. The set of training data samples is the training dataset of the industrial system under no-abnormal conditions.

[0021] S3. Utilize the long-time sequence modeling capability of the LSTM deep model and construct a preliminary LSTM-AE model based on the autoencoder principle.

[0022] S4. Reconstruct and train the initial LSTM-AE model using the training dataset, optimize the network weight values ​​of the initial LSTM-AE model, and obtain the trained LSTM-AE model and the error sequence of the training data samples.

[0023] S5. Using the error sequence of the training dataset, obtain the adaptive threshold formula for the industrial system under test. The formula includes a training data reconstruction error parameter. By assigning a value to the training data reconstruction error parameter, the value of the adaptive threshold can be obtained. The value of the adaptive threshold is used to judge outliers, thus completing the construction of the anomaly detection model.

[0024] Furthermore, a preferred embodiment is provided, wherein in step S1:

[0025] The original training time series data samples are represented as follows:

[0026]

[0027] The original training time series dataset is represented as follows:

[0028] X={x (1) x (2) ,....,x (i) ,....,x (n)}

[0029] in,

[0030] i∈[1, n], j∈[1, m]

[0031] In the formula, n represents the original training time-series data samples collected at n time points, and m represents the original training time-series data samples composed of m dimensions of features. This represents the j-th dimension of the original training time series data sample collected at the i-th time point.

[0032] Furthermore, a preferred embodiment is provided, wherein step S2 is specifically as follows:

[0033] S2.1. Standardize the data in each dimension of the collected raw training time-series data samples, expressed by the formula:

[0034]

[0035] The standardized original training time series data samples are represented as follows:

[0036]

[0037] The standardized original training time series dataset is represented as follows:

[0038] X * ={x (*1) x (*2) ,....,x (*i) , ..., x (*n)}

[0039] in,

[0040] j∈[1, m]

[0041] In the formula, Let u(X) represent the j-th dimension of the original training time series data sample collected at time point i after standardization. j ) represents the mean of n original training time series data samples in the j-th dimension, σ(X) j () represents the variance of n original training time series data samples in the j-th dimension;

[0042] S2.2. Extract training data samples from the standardized original training time-series data samples at equal time intervals. The specific process is as follows:

[0043] Let I be the extraction interval and L be the extraction length. Then the first extracted training data sample is:

[0044] a (1) ={x (*1) x (*2) , ..., x (*L)}

[0045] The extracted k-th training data sample is represented as:

[0046] a (k) ={x (*(1+(k-1)I)) x (*(2+(k-1)I)) ,....,x(*(L+(k-1)I))}

[0047] Let N be the total number of extracted training data samples, then the last item of the Nth extracted training data sample is:

[0048]

[0049] in,

[0050]

[0051] The set of N training data samples is called the training dataset, denoted as:

[0052] A={a (1) a (2) , ..., a (k) , ..., a (N)}

[0053] Furthermore, a preferred embodiment is provided, wherein step S3 is specifically as follows:

[0054] The initial LSTM-AE model consists of three layers, two of which are LSTM layers and the other is a linear fully connected layer.

[0055] The two LSTM layers are the encoding layer and the decoding layer, respectively;

[0056] The dimensions of the input parameters and the output parameters of the encoding layer, decoding layer, and linear fully connected layer are all set to m, that is, they have the same dimensions as the data samples input to the initial LSTM-AE model.

[0057] The coding layer consists of L LSTM units, and each LSTM unit of the coding layer... This represents the encoded hidden state at time t;

[0058] The decoding layer also consists of L LSTM units, and each LSTM unit in the decoding layer... This represents the decoded hidden state at time t;

[0059] In the forward propagation of the neural network, the decoded hidden state of the decoder layer is set to be the same as the encoded hidden state of the encoder layer at t=L, that is... When 1≤t≤L-1, the decoding hidden state of the decoding layer is set to... Among them, F D This represents the operation process of the LSTM unit in the decoding layer. Let x be the hidden decoding state of the decoding layer at time t+1. t This is the input to the initial LSTM-AE model corresponding to time t;

[0060] Set the decoding hidden state of the decoding layer If the input to the linear fully connected layer at time t is , then the output of the linear connected layer at time t is . This is the reconstruction output of the initial LSTM-AE model at time t, where f represents the activation function, and w and b are the network weight values ​​of the initial LSTM-AE model.

[0061] Furthermore, a preferred embodiment is provided, wherein step S4 is specifically as follows:

[0062] S4.1 Constructing the neural network loss function, the specific process is as follows:

[0063] Take the training data sample a from training dataset A (k) As input to the initial LSTM-AE model, the corresponding reconstructed output sequence is:

[0064]

[0065] in, Let be the reconstructed output at time t, which is compared with the training data sample a. (k) x in (*(t+(k-1)I)) Correspondingly, let x be... (*(t+(k-1)I)) For x t The loss function of the constructed neural network is then:

[0066]

[0067] S4.2. Reconstruct and train the initial LSTM-AE model, and update and optimize the network weight values. The specific process is as follows:

[0068] Set a predefined number of training iterations, and sequentially input all training data samples from training dataset A into the initial LSTM-AE model. With the goal of minimizing the neural network loss function, update and optimize the network weight values. When the number of iterations reaches the predefined number of training iterations, the training ends, and the trained LSTM-AE model is obtained.

[0069] S4.3 After training, transfer the training data samples a from training dataset A. (k) As input to the trained LSTM-AE model, the corresponding reconstructed output sequence is:

[0070]

[0071] in, The reconstructed output of the trained LSTM-AE model at time t is compared with the training data sample a. (k) x in (*(t+(k-1)I)) Correspondingly, let x be... (*(t+(k-1)I)) For xt Then x t The corresponding reconstruction error is:

[0072]

[0073] Then the training data sample a (k) The corresponding error sequence is:

[0074] e tr ={e1, e2, ..., e t ,....,e L}

[0075] Furthermore, a preferred embodiment is provided, wherein step S5 is specifically as follows:

[0076] S5.1 The calculation formula for the error threshold includes an abnormal probability value q, which is to be set. The specific process is as follows:

[0077] The distribution of extreme values ​​in the error sequence of the training dataset is calculated using the following formula:

[0078]

[0079] in,

[0080] γ∈R, 1+γx>0

[0081] In the formula, x represents the data in the error sequence; γ is the extreme value coefficient, which depends on the initial distribution of the error sequence data;

[0082] Using the method based on the second extreme value theorem, the distribution of extreme values ​​in the error sequence is fitted. If and only if there exists σ, for x∈R and 1+γx>0, then:

[0083]

[0084] In the formula, F t (X) represents the cumulative distribution function of X, where t is the initial threshold and Xt is the portion exceeding the threshold t. The distribution of extreme values ​​in the portion of Xt follows a generalized Pareto distribution. Based on the second extreme value theorem, the extreme values ​​are fitted to the Xt portion using the generalized Pareto distribution. The extreme value coefficients γ and σ are estimated using the maximum likelihood method and are expressed as follows: and The formula for calculating the error threshold is as follows:

[0085]

[0086] Where, N tLet Xt represent the number of data points, n be the number of data points, and q be the anomaly probability value. The error threshold T can be obtained by setting the anomaly probability value q. tr ;

[0087] S5.2 The optimal anomaly probability value is automatically obtained using a formula method. The anomaly probability value q in the error threshold calculation formula is set as the optimal anomaly probability value, and the error threshold T is calculated. tr The specific process is as follows:

[0088] Determining the anomaly probability value q is equivalent to an optimization process, the optimization objective and calculation process of which are as follows:

[0089]

[0090] Among them, T tr For the error threshold; e tr For the error sequence; Δμ(e tr ) and Δσ 2 (e tr ) represents the change in the mean and variance of the error sequence after removing outliers; |e a | represents the number of anomalies, which can be interpreted as the goal determined by the q-value being to remove data exceeding a threshold so that e tr The values ​​at which the mean and standard deviation decay to the maximum;

[0091] When the first formula When the value of is maximized, the value of the anomaly probability q in the formula is the optimal anomaly probability value;

[0092] In the formula for calculating the error threshold, the anomaly probability value q is set to be equal to the optimal anomaly probability value, thus obtaining the error threshold T. tr ;

[0093] S5.3, Adaptive Threshold The formula is:

[0094]

[0095] In the formula, ε is a coefficient; η is a hyperparameter coefficient; T rc To reconstruct error parameters from training data, different values ​​can be assigned to these parameters to adjust the error threshold T. tr Make adjustments in real time.

[0096] The present invention discloses an anomaly detection method based on LSTM-AE and dynamic threshold adjustment, the technical solution of which is as follows:

[0097] S6. Under the test operating conditions of the industrial system under test, collect real-time data of multiple state parameters of the industrial system under test as raw test timing data samples. The set of raw test timing data samples is the raw test timing dataset of the industrial system under test conditions.

[0098] S7. Perform data preprocessing on the collected raw test timing data samples to obtain test data samples. The set of test data samples is the test dataset of the industrial system under test conditions.

[0099] S8. Using test data samples as input to the anomaly detection model, calculate the reconstruction error and error sequence for each test data sample, and assign the reconstruction error value of the test data sample to the training data reconstruction error parameter to obtain the value of the adaptive threshold; the anomaly detection model is constructed using the construction method described in any of the above embodiments.

[0100] S9. Compare the reconstruction error of the test data sample with the value of the adaptive threshold to determine whether the industrial system under test has any abnormalities.

[0101] Furthermore, a preferred embodiment is provided, wherein in step S6:

[0102] The original test timing data sample is represented as follows:

[0103]

[0104] The original test time series dataset is represented as follows:

[0105] Y = {y (1) y (2) , ..., y (i) , ..., y (n)}

[0106] in,

[0107] i∈[1, n′], j∈[1, m]

[0108] In the formula, n′ represents the original test time series data samples collected at n′ time points, and m represents the original test time series data samples composed of m dimensions of features. This represents the j-th dimension of the original test time series data sample collected at the i-th time point.

[0109] Furthermore, a preferred embodiment is provided, wherein step S7 is specifically as follows:

[0110] S7.1. Standardize the data for each dimension in the collected raw test time series data samples, as expressed by the formula:

[0111]

[0112] The standardized original test timing data sample is represented as follows:

[0113]

[0114] The standardized original training time series dataset can be represented as:

[0115] Y * ={y (*1) y (*2) ,....,y (*i) , ..., y (*n′)}

[0116] in,

[0117] j∈[1, m]

[0118] In the formula, U(Y) represents the j-th dimension of the standardized original test time series data sample collected at time point i. j ) and represent the mean of the n′ original training time series data samples in the j-th dimension, σ(Y) j ) represents the variance of n′ original test time series data samples in the j-th dimension;

[0119] S7.2. Extract test data samples from the standardized original test time series data samples at equal time intervals. The specific process is as follows:

[0120] Let I be the extraction interval and L be the extraction length. Then the first extracted test data sample is:

[0121] b (1) ={y (*1) y (*2) , ..., y (*L)}

[0122] The extracted k-th test data sample is represented as:

[0123] b (k) ={y (*(1+(k-1)I)) y (*(2+(k-1)I)) ,....,y (*(L+(k-1)I))}

[0124] Let N′ be the total number of extracted test data samples, then the last data item of the N′-th extracted test data sample is:

[0125]

[0126] It can be seen that,

[0127]

[0128] Where n′ is the number of sample data in the standardized multidimensional time series data;

[0129] The set of N′ test data samples is the test dataset, denoted as:

[0130] B = {b} (1) b (2) , ..., b (k) , ..., b (N′)}

[0131] The anomaly detection system of the present invention has the following technical solution:

[0132] The system includes a processor and a memory, wherein the memory is used to store executable instructions of the processor, and the processor is configured to execute an anomaly detection method described in any of the above embodiments by executing the executable instructions.

[0133] The present invention has the following beneficial effects:

[0134] This invention is applied to the field of anomaly detection in industrial systems, and can effectively detect abnormal data in the operating data of industrial systems.

[0135] 1. To address the problem that existing anomaly detection models have weak feature extraction capabilities for nonlinear or time-series data, resulting in low accuracy in anomaly detection for industrial systems, the anomaly detection model described in this invention utilizes the nonlinear data processing capabilities and long-term sequence dependency memory capabilities of LSTM-AE, effectively adapting to the nonlinear and time-series characteristics of industrial data and improving the accuracy of anomaly detection for industrial systems.

[0136] 2. To address the problem that existing anomaly detection models have weak analytical capabilities for high-dimensional data, resulting in low efficiency in anomaly detection for industrial systems, the anomaly detection model described in this invention utilizes the high-dimensional data analysis capabilities of LSTM-AE to quickly and efficiently analyze and model high-dimensional data, thereby improving the efficiency of anomaly detection for industrial systems.

[0137] 3. Existing anomaly detection models use an approximate Gaussian distribution of the initial data distribution as a parameter when determining the anomaly detection threshold, which leads to errors in the established initial data model and reduces the accuracy of anomaly detection in industrial systems. The anomaly detection model described in this invention proposes a method that uses extreme value theory to analyze the reconstruction error and obtain the anomaly detection threshold. This method does not depend on the initial data distribution and avoids the error caused by approximating the initial data distribution as a Gaussian distribution, thus improving the accuracy of anomaly detection in industrial systems.

[0138] 4. Existing anomaly detection models require manual parameter setting when determining anomaly detection thresholds, leading to reliance on manual inspection for accuracy in industrial systems and inconsistent results among different individuals. This invention proposes an automatic parameter setting method based on extreme value theory, which automatically obtains the error threshold without requiring manual parameter adjustment. This reduces reliance on human experience and avoids fluctuations in accuracy due to differences in individual experience, enabling long-term stable anomaly detection.

[0139] 5. Existing anomaly detection models mostly employ the 3σ method or extreme value theory when determining thresholds, which cannot dynamically adjust the thresholds according to changes in environmental parameters. This results in poor environmental adaptability for anomaly detection in industrial systems, leading to low accuracy in anomaly detection under different working environments. The anomaly detection model described in this invention proposes a method for dynamically adjusting the threshold using a formula, improving the adaptive capability of the constructed model and thus enhancing its environmental adaptability for anomaly detection in industrial systems. Simultaneously, it avoids the limitations of traditional fixed thresholds in anomaly detection accuracy, thereby improving the accuracy of anomaly detection in industrial systems under different working environments.

[0140] 6. The anomaly detection method and system based on LSTM-AE and dynamic threshold adjustment described in this invention, using the anomaly detection model described in this invention, improves the accuracy of detecting operational anomalies in industrial systems, reduces reliance on human experience, improves the accuracy and consistency of detection results, and enhances adaptability to different working environments.

[0141] The anomaly detection model, the anomaly detection method, and the anomaly detection system described in this invention are applied to anomaly detection in industrial systems. Attached Figure Description

[0142] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0143] Figure 1 In a preferred embodiment of the present invention, the model block diagram of the preliminary LSTM-AE is shown.

[0144] Figure 2 In a preferred embodiment of the present invention, a flowchart of a method for constructing an anomaly detection model is provided.

[0145] Figure 3 The flowchart of an anomaly detection method based on LSTM-AE and dynamic threshold adjustment is shown in a preferred embodiment of the present invention. Detailed Implementation

[0146] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solutions of the present application, rather than limitations on the technical solutions of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0147] Implementation Method 1: Combination Figure 1 , Figure 2 This embodiment describes a method for constructing an anomaly detection model, and the specific implementation details are as follows:

[0148] S1. Under the stable operating condition of the industrial system under test without any abnormalities, data of multiple state parameters of the industrial system under test are collected as raw training time series data samples. The set of raw training time series data samples is the raw training time series dataset of the industrial system under the condition of no abnormalities.

[0149] S2. Perform data preprocessing on the original training time series data samples to obtain training data samples. The set of training data samples is the training dataset of the industrial system under no-abnormal conditions.

[0150] S3. Utilize the long-time sequence modeling capability of the LSTM deep model and construct a preliminary LSTM-AE model based on the autoencoder principle.

[0151] S4. Reconstruct and train the initial LSTM-AE model using the training dataset, optimize the network weight values ​​of the initial LSTM-AE model, and obtain the trained LSTM-AE model and the error sequence of the training data samples.

[0152] S5. Using the error sequence of the training dataset, obtain the adaptive threshold formula for the industrial system under test. The formula includes a training data reconstruction error parameter. By assigning a value to the training data reconstruction error parameter, the value of the adaptive threshold can be obtained. The value of the adaptive threshold is used to judge outliers, thus completing the construction of the anomaly detection model.

[0153] In this embodiment, the anomaly detection model of the present invention is constructed based on the LSTM-AE framework and an adaptive threshold formula. The LSTM-AE framework's nonlinear data extraction capabilities, long-term sequence-dependent memory capabilities, and high-dimensional data analysis capabilities are well-suited to the nonlinear, temporal, and high-dimensional data characteristics of industrial applications. Using the anomaly detection model of the present invention, constructed based on the LSTM-AE framework, to detect anomalies in industrial systems can improve the accuracy of the detection results. Furthermore, the anomaly detection model of the present invention employs an adaptive threshold that is independent of the initial data distribution, can automatically set parameters, and can be adjusted in real time. When applied to anomaly detection, this reduces model-specific errors, decreases reliance on human experience, and adapts to environmental changes, effectively reducing environmental errors and improving the accuracy, stability, and consistency of anomaly detection.

[0154] Implementation Method Two: Combination Figure 1 and Figure 2 This embodiment further defines step S1 in the method for constructing an anomaly detection model described in Embodiment 1. In step S1:

[0155] The original training time series data samples are represented as follows:

[0156]

[0157] The original training time series dataset is represented as follows:

[0158] X={x (1) x (2) ,....,x (i) ,....,x (n)}

[0159] in,

[0160] i∈[1, n], j∈[1, m]

[0161] In the formula, n represents the original training time-series data samples collected at n time points, and m represents the original training time-series data samples composed of m dimensions of features. This represents the j-th dimension of the original training time series data sample collected at the i-th time point.

[0162] In this embodiment, for example, the monitoring sensors of the industrial system collect 10,000 original training time series data samples at each time point. Each original training time series data sample contains parameters in 5 dimensions, which can be temperature, humidity, gas pressure, light intensity, and concentration of harmful gases, respectively. Thus, the collected original training time series dataset contains a total of 10,000 × 5 data points.

[0163] Implementation Method 3: Combination Figure 1 and Figure 2 This embodiment further defines step S2 in the method for constructing an anomaly detection model described in Embodiment 2. The specific implementation details are as follows:

[0164] Step S2 is as follows:

[0165] S2.1. Standardize the data in each dimension of the collected raw training time-series data samples, expressed by the formula:

[0166]

[0167] The standardized original training time series data samples are represented as follows:

[0168]

[0169] The standardized original training time series dataset is represented as follows:

[0170] X * ={x (*1) x ( * 2) , ..., x (*i) , ..., x (*n)}

[0171] in,

[0172] j∈[1, m]

[0173] In the formula, Let u(X) represent the j-th dimension of the original training time series data sample collected at time point i after standardization. j ) represents the mean of n original training time series data samples in the j-th dimension, σ(X) j () represents the variance of n original training time series data samples in the j-th dimension;

[0174] S2.2. Extract training data samples from the standardized original training time-series data samples at equal time intervals. The specific process is as follows:

[0175] Let I be the extraction interval and L be the extraction length. Then the first extracted training data sample is:

[0176] a( 1 )={x (*1) x (*2) , ..., x (*L)}

[0177] The extracted k-th training data sample is represented as:

[0178] a (k) ={x (*(1+(k-1)I)) x (*(2+(k-1)I)) , ..., x (*(L+(k-1)I))}

[0179] Let N be the total number of extracted training data samples, then the last item of the Nth extracted training data sample is:

[0180]

[0181] in,

[0182]

[0183] The set of N training data samples is called the training dataset, denoted as:

[0184] A={a (1) a (2) , ..., a (k) , ..., a (N)}

[0185] In this embodiment, the significance of standardizing the data is that in practical problems, the sample data we obtain are multi-dimensional, that is, a sample is characterized by multiple features. The dimensions and magnitudes of these features are different. When detecting anomalies, if the original data values ​​are used directly, their influence on the anomaly judgment will be different. However, through standardization, different features can have the same scale.

[0186] Implementation Method 4: Combination Figure 1 and Figure 2 This embodiment further defines step S3 in the method for constructing an anomaly detection model described in Embodiment 3. The specific implementation details are as follows:

[0187] Step S3 is as follows:

[0188] The initial LSTM-AE model consists of three layers, two of which are LSTM layers and the other is a linear fully connected layer.

[0189] The two LSTM layers are the encoding layer and the decoding layer, respectively;

[0190] The dimensions of the input parameters and the output parameters of the encoding layer, decoding layer, and linear fully connected layer are all set to m, that is, they have the same dimensions as the data samples input to the initial LSTM-AE model.

[0191] The coding layer consists of L LSTM units, and each LSTM unit of the coding layer... This represents the encoded hidden state at time t;

[0192] The decoding layer also consists of L LSTM units, and each LSTM unit in the decoding layer... This represents the decoded hidden state at time t;

[0193] In the forward propagation of the neural network, the decoded hidden state of the decoder layer is set to be the same as the encoded hidden state of the encoder layer at t=L, that is... When 1≤t≤L-1, the decoding hidden state of the decoding layer is set to... Among them, F D This represents the operation process of the LSTM unit in the decoding layer. Let x be the hidden decoding state of the decoding layer at time t+1. t This is the input to the initial LSTM-AE model corresponding to time t;

[0194] Set the decoding hidden state of the decoding layer If the input to the linear fully connected layer at time t is , then the output of the linear connected layer at time t is . This is the reconstruction output of the initial LSTM-AE model at time t, where f represents the activation function, and w and b are the network weight values ​​of the initial LSTM-AE model.

[0195] In this embodiment, the preliminary LSTM-AE model combines the advantages of LSTM model in processing long-term series and AE model in unsupervised feature learning. It has strong feature extraction capabilities for nonlinear or time-series data and strong analytical capabilities for high-dimensional data, making it suitable for analyzing and processing multi-dimensional time-series data collected in industrial systems.

[0196] Implementation Method 5: Combination Figure 1 and Figure 2 This embodiment further defines step S4 in the method for constructing an anomaly detection model described in Embodiment 4. The specific implementation details are as follows:

[0197] Step S4 is as follows:

[0198] S4.1 Constructing the neural network loss function, the specific process is as follows:

[0199] Take the training data sample a from training dataset A (k) As input to the initial LSTM-AE model, the corresponding reconstructed output sequence is:

[0200]

[0201] in, Let be the reconstructed output at time t, which is compared with the training data sample a. (k) x in (*(t+(k-1)I)) Correspondingly, let x be... (*(t+(k-1)I)) For x t The loss function of the constructed neural network is then:

[0202]

[0203] S4.2. Reconstruct and train the initial LSTM-AE model, and update and optimize the network weight values. The specific process is as follows:

[0204] Set a predefined number of training iterations, and sequentially input all training data samples from training dataset A into the initial LSTM-AE model. With the goal of minimizing the neural network loss function, update and optimize the network weight values. When the number of iterations reaches the predefined number of training iterations, the training ends, and the trained LSTM-AE model is obtained.

[0205] S4.3 After training, transfer the training data samples a from training dataset A. (k) As input to the trained LSTM-AE model, the corresponding reconstructed output sequence is:

[0206]

[0207] in, The reconstructed output of the trained LSTM-AE model at time t is compared with the training data sample a. (k) x in (*(t+(k-1)I)) Correspondingly, let x be... (*(t+(k-1)I)) For x t Then x t The corresponding reconstruction error is:

[0208]

[0209] Then the training data sample a (k) The corresponding error sequence is:

[0210] e tr ={e1, e2, ..., e t ,....,e L}

[0211] Implementation Method Six: Combination Figure 1 and Figure 2 This embodiment further defines step S5 in the method for constructing an anomaly detection model described in Embodiment 5. The specific implementation details are as follows:

[0212] Step S5 is as follows:

[0213] S5.1 The calculation formula for the error threshold includes an abnormal probability value q, which is to be set. The specific process is as follows:

[0214] The distribution of extreme values ​​in the error sequence of the training dataset is calculated using the following formula:

[0215]

[0216] in,

[0217] γ∈R, 1+γx>0

[0218] In the formula, x represents the data in the error sequence; γ is the extreme value coefficient, which depends on the initial distribution of the error sequence data;

[0219] Using the method based on the second extreme value theorem, the distribution of extreme values ​​in the error sequence is fitted. If and only if there exists σ, for x∈R and 1+γx>0, then:

[0220]

[0221] In the formula, F t (X) represents the cumulative distribution function of X, where t is the initial threshold and Xt is the portion exceeding the threshold t. The distribution of extreme values ​​in the portion of Xt follows a generalized Pareto distribution. Based on the second extreme value theorem, the extreme values ​​are fitted to the Xt portion using the generalized Pareto distribution. The extreme value coefficients γ and σ are estimated using the maximum likelihood method and are expressed as follows: and The formula for calculating the error threshold is as follows:

[0222]

[0223] Where, N t Let Xt represent the number of data points, n be the number of data points, and q be the anomaly probability value. The error threshold T can be obtained by setting the anomaly probability value q. tr ;

[0224] S5.2 The optimal anomaly probability value is automatically obtained using a formula method. The anomaly probability value q in the error threshold calculation formula is set as the optimal anomaly probability value, and the error threshold T is calculated. tr The specific process is as follows:

[0225] Determining the anomaly probability value q is equivalent to an optimization process, the optimization objective and calculation process of which are as follows:

[0226]

[0227] Among them, T tr For the error threshold; e tr For the error sequence; Δμ(e tr ) and Δσ 2 (e tr ) represents the change in the mean and variance of the error sequence after removing outliers; |e a | represents the number of anomalies, which can be interpreted as the goal determined by the q-value being to remove data exceeding a threshold so that e tr The values ​​at which the mean and standard deviation decay to the maximum;

[0228] When the first formula When the mean is maximized, the value of the anomaly probability q in the formula is the optimal anomaly probability value;

[0229] In the formula for calculating the error threshold, the anomaly probability value q is set to be equal to the optimal anomaly probability value, thus obtaining the error threshold T. tr ;

[0230] S5.3, Adaptive Threshold The formula is:

[0231]

[0232] In the formula, ε is a coefficient; η is a hyperparameter coefficient; T rc To reconstruct error parameters from training data, different values ​​can be assigned to these parameters to adjust the error threshold T. tr Make adjustments in real time.

[0233] In this embodiment, a method based on the second extreme value theorem is used to fit the extreme values ​​using the generalized Pareto distribution. This method determines the threshold formula without relying on the initial data distribution, reducing the inherent error of the anomaly detection model. Furthermore, this embodiment proposes a method that equates the determination of the fixed threshold to an optimization process. This eliminates the need for manual parameter setting when determining the fixed threshold, reducing the reliance on human experience in constructing the anomaly detection model. By using a formula to obtain an adaptive threshold from the fixed threshold, the adaptive threshold value can be automatically adjusted based on the real-time data collected from the industrial system to be anomaly detected, i.e., different test data samples. This avoids the limitations of traditional fixed thresholds in anomaly detection accuracy and can improve the accuracy of anomaly detection in industrial systems under different working environments.

[0234] Implementation Method Seven: Combination Figure 1 , Figure 2 This embodiment describes an anomaly detection method based on LSTM-AE and dynamic threshold adjustment. The specific implementation details are as follows:

[0235] The anomaly detection method includes:

[0236] S6. Under the test operating conditions of the industrial system under test, collect real-time data of multiple state parameters of the industrial system under test as raw test timing data samples. The set of raw test timing data samples is the raw test timing dataset of the industrial system under test conditions.

[0237] S7. Perform data preprocessing on the collected raw test timing data samples to obtain test data samples. The set of test data samples is the test dataset of the industrial system under test conditions.

[0238] S8. Using test data samples as input to the anomaly detection model, calculate the reconstruction error and error sequence for each test data sample, and assign the reconstruction error value of the test data sample to the reconstruction error parameter of the training data to obtain the value of the adaptive threshold; the anomaly detection model is constructed using the construction method described in any one of embodiments one through six.

[0239] S9. Compare the reconstruction error of the test data sample with the value of the adaptive threshold to determine whether the industrial system under test has any abnormalities.

[0240] Implementation Method 8: Combination Figure 1 and Figure 2 This embodiment further defines step S6 in the anomaly detection method based on LSTM-AE and dynamic threshold adjustment described in Embodiment 7. The specific implementation details are as follows:

[0241] In step S6:

[0242] The original test timing data sample is represented as follows:

[0243]

[0244] The original test time series dataset is represented as follows:

[0245] Y = {y (1) y (2) , ..., y (i) , ..., y (n)}

[0246] in,

[0247] i∈[1, n′], j∈[1, m]

[0248] In the formula, n′ represents the original test time series data samples collected at n′ time points, and m represents the original test time series data samples composed of m dimensions of features. This represents the j-th dimension of the original test time series data sample collected at the i-th time point.

[0249] In this embodiment, for example, the monitoring sensors of the industrial system collect 20,000 raw test time series data samples at each time point. Each raw test time series data sample contains parameters in 5 dimensions. These 5 dimensions should be the same as the dimensions of the raw training time series data samples, namely temperature, humidity, gas pressure, light intensity, and concentration of harmful gases. Thus, the collected raw test time series dataset contains a total of 20,000 × 5 data points.

[0250] Implementation Method Nine: Combination Figure 1 and Figure 2 This embodiment further defines step S7 in the anomaly detection method based on LSTM-AE and dynamic threshold adjustment described in Embodiment 7. The specific implementation details are as follows:

[0251] Step S7 is as follows:

[0252] S7.1. Standardize the data for each dimension in the collected raw test time series data samples, as expressed by the formula:

[0253]

[0254] The standardized original test timing data sample is represented as follows:

[0255]

[0256] The standardized original training time series dataset can be represented as:

[0257] Y * ={y (*1) y (*2) , ..., y (*i) , ..., y (*n′)}

[0258] in,

[0259] j∈[1, m]

[0260] In the formula, U(Y) represents the j-th dimension of the standardized original test time series data sample collected at time point i. j ) and represent the mean of the n′ original training time series data samples in the j-th dimension, σ(Y) j ) represents the variance of n′ original test time series data samples in the j-th dimension;

[0261] S7.2. Extract test data samples from the standardized original test time series data samples at equal time intervals. The specific process is as follows:

[0262] Let I be the extraction interval and L be the extraction length. Then the first extracted test data sample is:

[0263] b (1) ={y (*1) y (*2) , ..., y (*L)}

[0264] The extracted k-th test data sample is represented as:

[0265] b (k) ={y (*(1+(k-1)I)) y (*(2+(k-1)I)) ,....,y (*(L+(k-1)I))}

[0266] Let N′ be the total number of extracted test data samples, then the last data item of the N′-th extracted test data sample is:

[0267]

[0268] It can be seen that,

[0269]

[0270] Where n′ is the number of sample data in the standardized multidimensional time series data;

[0271] The set of N′ test data samples is the test dataset, represented as:

[0272] B = {b}(1) b (2) , ..., b (k) , ..., b (N′)}

[0273] Implementation Method 10: Combination Figure 1 and Figure 2 This embodiment further defines step S8 in the anomaly detection method based on LSTM-AE and dynamic threshold adjustment described in Embodiment 7. The specific implementation details are as follows:

[0274] Step S8 is as follows:

[0275] Take the training data sample b from test dataset A (k) As input to the anomaly detection model, the corresponding reconstructed output sequence is:

[0276]

[0277] in, The reconstructed output of the anomaly detection model at time t is compared with the test data sample b. (k) y in (*(t+(k-1)I)) Correspondingly, let y be... (*(t+(k-1)I)) For y t , then y t The corresponding reconstruction error is:

[0278]

[0279] Then training data sample b (k) The corresponding error sequence is:

[0280] e′ tr ={e′1,e′2,...,e′ t , ..., e′ L}

[0281] The reconstruction error e′ t The value of is assigned to the reconstruction error parameter of the training data to obtain the value of the adaptive threshold.

[0282] In this embodiment, the anomaly detection model is the anomaly detection model described in any of the preceding embodiments.

[0283] Implementation Method Eleven: Combination Figure 1 and Figure 2 This embodiment further defines step S9 in the anomaly detection method based on LSTM-AE and dynamic threshold adjustment described in Embodiment 7. The specific implementation details are as follows:

[0284] If the reconstruction error of the test data sample is greater than the adaptive error threshold, then the corresponding test data sample contains abnormal data, proving that the industrial system under test has an anomaly.

[0285] Implementation Method Twelve, Combination Figure 1 and Figure 2 This embodiment describes a comparative experiment between the anomaly detection method based on LSTM-AE and dynamic threshold adjustment described in Embodiment 1 and existing anomaly detection methods. The specific experimental content is as follows:

[0286] Table 1 Comparison of TEP multi-parameter anomaly detection methods

[0287]

[0288] Experiments were conducted on the TEP dataset, as shown in Table 1. The 3σ-based method also achieved good detection results, which is partly due to the high discriminative power between normal and abnormal states. However, the proposed method still achieves further improvement compared to the 3σ method, with the final F1-Score reaching 100%. This demonstrates the high adaptability of the proposed method.

[0289] Implementation Method Thirteen: Combination Figure 1 , Figure 2 This embodiment describes an anomaly detection system, the specific implementation details of which are as follows:

[0290] The system includes a processor and a memory, the memory being used to store executable instructions of the processor, the processor being configured to execute an anomaly detection method based on LSTM-AE and dynamic threshold adjustment as described in any one of embodiments seven through eleven via executing the executable instructions.

[0291] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, reasonable combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing an anomaly detection model, comprising: include: S1. Under the stable operating condition of the industrial system under test without any abnormalities, data of multiple state parameters of the industrial system under test are collected as raw training time series data samples. The set of raw training time series data samples is the raw training time series dataset of the industrial system under the condition of no abnormalities. S2. Perform data preprocessing on the original training time series data samples to obtain training data samples. The set of training data samples is the training dataset of the industrial system under no-abnormal conditions. S3. Utilize the long-time sequence modeling capability of the LSTM deep model and construct a preliminary LSTM-AE model based on the autoencoder principle. S4. Reconstruct and train the initial LSTM-AE model using the training dataset, optimize the network weight values ​​of the initial LSTM-AE model, and obtain the trained LSTM-AE model and the error sequence of the training data samples. S5. Using the error sequence of the training dataset, obtain the adaptive threshold formula of the industrial system under test. The formula includes a training data reconstruction error parameter. By assigning a value to the training data reconstruction error parameter, obtain the value of the adaptive threshold. The value of the adaptive threshold is used to judge outliers, thus completing the construction of the anomaly detection model. Step S3 is as follows: The initial LSTM-AE model consists of three layers, two of which are LSTM layers and the other is a linear fully connected layer. The two LSTM layers are the encoding layer and the decoding layer, respectively; The dimensions of the input parameters and the output parameters of the encoding layer, decoding layer, and linear fully connected layer are all set to m, that is, the same dimension as the data samples input to the initial LSTM-AE model; The coding layer consists of L LSTM units, and each LSTM unit of the coding layer... ( () represents the encoded hidden state at time t; The decoding layer is also composed of L LSTM units, and the decoding hidden state at time t of each LSTM unit of the decoding layer is represented as ​​ In the forward propagation of the neural network, the decoded hidden state of the decoder layer is set to be the same as the encoded hidden state of the encoder layer at t=L, that is... = When 1 ≤ t ≤ L-1, set the decoding hidden state of the decoding layer to . , of which F D This represents the operation process of the LSTM unit in the decoding layer. This represents the hidden decoding state of the decoding layer at time t+1. This is the input to the initial LSTM-AE model corresponding to time t; Set the decoding hidden state of the decoding layer If the input to the linear fully connected layer at time t is , then the output of the linear connected layer at time t is . , which is the reconstruction output of the initial LSTM-AE model at time t, f represents the activation function, and w and b are the network weight values ​​of the initial LSTM-AE model; Step S4 is as follows: S4.1 Constructing the neural network loss function; S4.

2. Reconstruct and train the initial LSTM-AE model, and update and optimize the network weight values. The specific process is as follows: Set a predefined number of training iterations, and sequentially input all training data samples from training dataset A into the initial LSTM-AE model. With the goal of minimizing the neural network loss function, update and optimize the network weight values. When the number of iterations reaches the predefined number of training iterations, the training ends, and the trained LSTM-AE model is obtained. S4.3 After training, transfer the training data samples from training dataset A. As input to the trained LSTM-AE model, the corresponding reconstructed output sequence is: P={ } in, The reconstructed output of the trained LSTM-AE model at time t is compared with the training data samples. In Correspondingly, remember for ,but The corresponding reconstruction error is: Then the training data sample The corresponding error sequence is: 。 2.The method of claim 1, wherein, In step S1: The original training time series data samples are represented as follows: = The original training time series dataset is represented as follows: X1= in, i∈[1,n], j∈[1,m] In the formula, n represents the original training time-series data samples collected at n time points, and m represents the original training time-series data samples composed of m dimensions of features. This represents the j-th dimension of the original training time series data sample collected at the i-th time point. 3.The method of claim 1, wherein, Step S,2 is as follows: S2.

1. Standardize the data in each dimension of the collected raw training time-series data samples, expressed by the formula: The standardized original training time series data samples are represented as follows: = The standardized original training time series dataset is represented as follows: X * = in, j∈[1,m] In the formula, This represents the j-th dimension of the original training time-series data sample collected at time point i after standardization. This represents the mean of n original training time-series data samples along the j-th dimension. Let represent the variance of n original training time series data samples in the j-th dimension; S2.

2. Extract training data samples from the standardized original training time-series data samples at equal time intervals. The specific process is as follows: Let I be the extraction interval and L be the extraction length. Then the first extracted training data sample is: ={ } The extracted k-th training data sample is represented as: = Let N be the total number of extracted training data samples, then the last item of the Nth extracted training data sample is: = = in, N +1 The set of N training data samples is called the training dataset, denoted as: A= 。 4. The method of claim 1, wherein the method further comprises: The specific process of constructing the neural network loss function in S4.1 is as follows: The training data samples in the training data set A are input into the preliminary LSTM-AE model As the input of the preliminary LSTM-AE model, the corresponding reconstruction output sequence is: X’={ } in, The reconstructed output at time t is compared with the training data samples. In Correspondingly, remember for The loss function of the constructed neural network is then: 。 5. The method of claim 1, wherein the method further comprises: Step S5 is as follows: S5.1 The calculation formula for the error threshold includes an abnormal probability value q, which is to be set. The specific process is as follows: The distribution of extreme values ​​in the error sequence of the training dataset is calculated using the following formula: in, ∈R，1+ In the formula, The data is from the error sequence; These are the extreme value coefficients, which depend on the initial distribution of the error sequence data; Using a method based on the second extreme value theorem, the distribution of extreme values ​​in the error sequence is fitted if and only if σ exists. ∈R and 1+ Then we have: In the formula, F t ( x )express x Let be the cumulative distribution function, t be the initial threshold, and Xt be the portion exceeding the threshold t. The distribution of extreme values ​​in the portion of Xt follows a generalized Pareto distribution. Based on the second extreme value theorem, the extreme values ​​are fitted to the Xt portion using the generalized Pareto distribution. The extreme value coefficients γ and σ are estimated using the maximum likelihood method, and are expressed as: and The formula for calculating the error threshold is obtained as follows: Where Nt represents the number of Xt values, n is the number of data points, and q is the anomaly probability value. The error threshold T is obtained by setting the anomaly probability value q. tr ; S5.2, automatically obtaining the optimal abnormal probability value by formula method, setting the abnormal probability value q in the calculation formula of the error threshold as the optimal abnormal probability value, and calculating the error threshold T tr The specific process is as follows: Determining the anomaly probability value q is equivalent to an optimization process, the optimization objective and calculation process of which are as follows: in, This is the error threshold; It is an error sequence; and This represents the changes in the mean and variance of the error sequence after removing outliers; This represents the number of anomalies. The q-value is determined by the goal of removing data exceeding a threshold, resulting in... The values ​​at which the mean and standard deviation decay to the maximum; When the first formula When the value of is maximized, the value of the anomaly probability q in the formula is the optimal anomaly probability value; In the formula for calculating the error threshold, the anomaly probability value q is set equal to the optimal anomaly probability value to obtain the error threshold. ; S5.3, Adaptive threshold φ′, the formula is: In the formula, For coefficients; It is a hyperparameter coefficient; To reconstruct error parameters from training data, different values ​​are assigned to these parameters to adjust the error threshold. Make adjustments in real time.

6. An anomaly detection method based on LSTM-AE and dynamic threshold adjustment, characterized in that, include: S6. Under the test operating conditions of the industrial system under test, collect real-time data of multiple state parameters of the industrial system under test as raw test timing data samples. The set of raw test timing data samples is the raw test timing dataset of the industrial system under test conditions. S7. Perform data preprocessing on the collected raw test timing data samples to obtain test data samples. The set of test data samples is the test dataset of the industrial system under test conditions. S8. Using test data samples as input to the anomaly detection model, calculate the reconstruction error and error sequence for each test data sample, and assign the reconstruction error value of the test data sample to the reconstruction error parameter of the training data to obtain the value of the adaptive threshold; the anomaly detection model is constructed using the construction method described in any one of claims 1-3; S9. Compare the reconstruction error of the test data sample with the value of the adaptive threshold to determine whether the industrial system under test has any abnormalities.

7. The anomaly detection method based on LSTM-AE and dynamic threshold adjustment according to claim 6, characterized in that, In step S6: The original test timing data sample is represented as follows: = The original test time series dataset is represented as follows: Y= in, i∈[1,n′], j∈[1,m′] In the formula, n′ represents the original test time series data samples collected at n′ time points, and m′ is the number of dimensional features of the original test time series data samples. This represents the j-th dimension of the original test time series data sample collected at the i-th time point.

8. The anomaly detection method based on LSTM-AE and dynamic threshold adjustment according to claim 7, characterized in that, Step S7 is as follows: S7.

1. Standardize the data for each dimension in the collected raw test time series data samples, as expressed by the formula: The standardized original test timing data sample is represented as follows: = The standardized original test and training time series dataset is represented as follows: Y * = in, j∈[1,m′] wherein denotes the data of the j-th dimension of the i-th normalized raw test time series data sample collected at the i-th time point, and denotes the mean of the n' raw training time series data samples in the j-th dimension, denotes the variance of the n' raw test time series data samples in the j-th dimension; S7.

2. Extract test data samples from the standardized original test time series data samples at equal time intervals. The specific process is as follows: Let I be the extraction interval and L be the extraction length. Then the first extracted test data sample is: ={ } The extracted k-th test data sample is represented as: = Let N′ be the total number of extracted test data samples, then the last data item of the N′-th extracted test data sample is: = = N′ +1 Where n′ is the number of sample data in the standardized multidimensional time series data; The set of N′ test data samples is the test dataset, represented as: B= 。 9. An anomaly detection system, the system comprising: A processor and a memory, characterized in that the memory is used to store executable instructions of the processor, the processor being configured to execute an anomaly detection method based on LSTM-AE and dynamic threshold adjustment as described in any one of claims 6-8 by executing the executable instructions.

Images