Industrial sensor data anomaly detection method, device and equipment

Abstract

The application discloses an industrial sensor data anomaly detection method, device and equipment, and relates to the technical field of Internet of Things.The method comprises the following steps: obtaining a plurality of test samples in a first data set collected by an industrial sensor; inputting the plurality of test samples into a target anomaly detection model to determine the anomaly scores of the test samples, wherein the target anomaly detection model is obtained by embedding a BiLSTM model into a GAN; and performing anomaly detection on the plurality of test samples based on a clustering algorithm and the anomaly scores.The scheme of the application can avoid a cumbersome feature engineering process, realize automatic feature selection and filtering, guarantee the accuracy of the model, extract the time dependence in the data collected by the industrial sensor, and effectively improve the robustness of the anomaly detection model.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a method, apparatus, and equipment for detecting abnormal data from industrial sensors. Background Technology

[0002] Attacks and anomaly detection in IoT infrastructure sensors are increasingly important issues in the IoT field. Modern industries typically install numerous sensors to monitor system operation and facilitate the identification of potential security vulnerabilities by maintenance personnel. With the increasing use of sensors in industry, the threats and attacks on this infrastructure are also increasing. Denial-of-service attacks, data type probing, malicious control, malicious operation, scanning, and misconfiguration are all anomalies that can lead to IoT system failures. With the widespread deployment of 5G and IoT, the ability to analyze massive amounts of sensor data and accurately identify anomalies has become a pressing need for modern industrial enterprises.

[0003] Unlike problems and tasks in conventional models, anomaly detection targets a small number of uncertain and rare events. Anomaly detection technologies in the industrial field can be categorized by algorithmic model into statistical models, linear models, similarity-based models, ensemble learning models, and neural network models.

[0004] However, statistical and machine learning-based methods rely heavily on feature engineering. Industrially, sensor data is typically time-series data, and current technological solutions often neglect the processing of this time-series data, failing to fully utilize its temporal dependencies. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, and device for detecting anomalies in industrial sensor data, thereby solving the problem that the anomaly detection process in related technologies relies heavily on feature engineering and cannot fully utilize the time dependency relationship of time-series data.

[0006] To achieve the above objectives, embodiments of this application provide a method for detecting anomalies in industrial sensor data, including:

[0007] Multiple test samples are obtained from the first dataset collected by the industrial sensors;

[0008] Multiple test samples are input into the target anomaly detection model to determine the anomaly score of each test sample. The target anomaly detection model is obtained by embedding a BiLSTM network model into a Generative Adversarial Network (GAN).

[0009] Anomaly detection is performed on multiple test samples based on clustering algorithms and the anomaly scores.

[0010] Optionally, test samples are obtained from the data collected by industrial sensors, including:

[0011] Based on the pre-set size and step size of the first sliding window, the first dataset is divided into multiple first sub-time series;

[0012] One of the first sub-time series is identified as one of the test samples.

[0013] Optionally, multiple test samples are input into the target anomaly detection model to determine the anomaly score for each test sample, including:

[0014] The random space sample is input into the target generator of the target anomaly detection model so that the target generator generates the optimal reconstruction test sample, wherein the random space sample is the test sample mapped to the random latent space;

[0015] Determine the optimal reconstruction test sample and the reconstruction loss of the test sample;

[0016] The test sample is input into the target discriminator of the target anomaly detection model to determine the discrimination loss;

[0017] Based on the reconstruction loss and the discrimination loss, the anomaly score of each test sample is determined.

[0018] Optionally, random space samples are input into the target generator of the target anomaly detection model to enable the target generator to generate optimal reconstruction test samples, including:

[0019] Extract the contextual temporal features of the random spatial samples;

[0020] Based on the contextual temporal characteristics of the random spatial samples, reconstructed test samples are generated;

[0021] Based on the similarity between the reconstructed test sample and the test sample corresponding to the reconstructed test sample, the reconstructed test sample is updated using the gradient descent method to generate the optimal reconstructed test sample.

[0022] Optionally, the test sample is input into the target discriminator of the target anomaly detection model to determine the discrimination loss, including:

[0023] Extract the contextual temporal features of the test samples;

[0024] The discrimination loss is determined based on the contextual temporal features of the test samples.

[0025] Optionally, based on the clustering algorithm and the anomaly score, anomaly detection is performed on multiple test samples, including:

[0026] The highest and lowest anomaly scores among the anomaly scores are determined as the initial cluster centers;

[0027] Based on the distance from each of the abnormal scores to the two initial cluster centers, the test samples corresponding to the abnormal scores are classified.

[0028] Each cluster center is recalculated, and the test samples corresponding to each abnormal score are classified according to the distance from each abnormal score to each recalculated cluster center, until each cluster center meets the pre-set termination condition.

[0029] Based on the classification of the test samples, it is determined whether the test samples are abnormal.

[0030] Optionally, the method further includes:

[0031] Training samples are obtained from a second dataset acquired by industrial sensors, wherein the first dataset is different from the second dataset.

[0032] The anomaly detection model is iteratively trained based on the training samples to obtain the target anomaly detection model.

[0033] Optionally, training samples are obtained from the second dataset acquired by industrial sensors, including:

[0034] Based on the pre-set size and step size of the second sliding window, the second dataset is divided into multiple second sub-time series;

[0035] One of the second sub-time series is determined as one of the training samples.

[0036] The anomaly detection model is iteratively trained based on the training samples to obtain the target anomaly detection model, including:

[0037] Random noise is input into the generator of the anomaly detection model to obtain generated samples;

[0038] The training samples and the generated samples are input into the discriminator of the anomaly detection model, the parameters of the discriminator are adjusted, and the discrimination result is output.

[0039] Based on the discrimination result, the generator and the discriminator are iteratively trained until the target anomaly detection model is obtained, wherein the parameters include at least one of forget gate weight, input gate weight, output gate weight and bias term.

[0040] Optionally, the training samples and the generated samples are input into the discriminator of the anomaly detection model, the parameters of the discriminator are adjusted, and the discrimination result is output, including:

[0041] Extract the contextual temporal features of the training samples;

[0042] Based on the contextual temporal features of the training samples, the training samples are discriminated to adjust the parameters of the discriminator and output the discrimination result corresponding to the training samples;

[0043] The generated samples are judged, and the judgment result corresponding to the generated samples is output.

[0044] Optionally, based on the discrimination result, the generator and the discriminator are iteratively trained until the target anomaly detection model is obtained, including:

[0045] The sample whose discrimination result is to be generated is input into the generator, the parameters of the generator are adjusted based on the gradient descent method, and the generation result is output.

[0046] The generated result and the discrimination result are used as training samples, which are then input into the adjusted discriminator. The parameters of the discriminator are adjusted again based on the gradient descent method. The discrimination result is output, and the process returns to the steps of inputting the discrimination result as the generated sample into the generator, adjusting the parameters of the generator, and outputting the generated result, so as to iteratively train the generator and discriminator of the anomaly detection model until the target anomaly detection model is obtained.

[0047] This application embodiment also provides an industrial sensor data anomaly detection device, including:

[0048] The first acquisition module is used to acquire multiple test samples from the first dataset collected by the industrial sensor;

[0049] The determination module is used to input multiple test samples into the target anomaly detection model and determine the anomaly score of each test sample, wherein the target anomaly detection model is obtained by embedding a BiLSTM network model into a generative adversarial network (GAN).

[0050] The detection module is used to perform anomaly detection on multiple test samples based on a clustering algorithm and the anomaly score.

[0051] This application embodiment also provides an industrial sensor data anomaly detection device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the industrial sensor data anomaly detection method as described above.

[0052] This application embodiment also provides a readable storage medium storing a program, which, when executed by a processor, implements the industrial sensor data anomaly detection method as described above.

[0053] The above-mentioned technical solution of this application has at least the following beneficial effects:

[0054] The industrial sensor data anomaly detection method of this application embodiment first obtains multiple test samples from a first dataset collected by industrial sensors; second, inputs the multiple test samples into a target anomaly detection model to determine the anomaly score of each test sample, wherein the target anomaly detection model is obtained by embedding a BiLSTM model into a Generative Adversarial Network (GAN); finally, based on a clustering algorithm and the anomaly scores, anomaly detection is performed on the multiple test samples. Thus, firstly, the target anomaly detection model obtained by embedding a BiLSTM model into a GAN in this application embodiment, and the anomaly detection of test samples using the target anomaly detection model, effectively avoids the cumbersome feature engineering process in existing technologies, achieving automatic feature selection and filtering while ensuring model accuracy; secondly, it can extract the time dependency relationship in the data collected by industrial sensors, effectively improving the robustness of the anomaly detection model; and thirdly, by using a clustering algorithm to adaptively determine anomalies in the test samples, it achieves clustering of one-dimensional anomaly score data of sensor data, avoiding the "curse of dimensionality" that traditional algorithms easily fall into when processing large datasets. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating the industrial sensor data anomaly detection method according to an embodiment of this application;

[0056] Figure 2 This is a schematic diagram of the LSTM network structure;

[0057] Figure 3 This is a schematic diagram of the BiLSTM model.

[0058] Figure 4 This is a framework diagram of a GAN network;

[0059] Figure 5 This is a schematic diagram of the structure of the industrial sensor data anomaly detection device according to an embodiment of this application;

[0060] Figure 6 This is a schematic diagram of the structure of an industrial sensor data anomaly detection device according to an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0063] The industrial sensor data anomaly detection method, apparatus, and equipment provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0064] like Figure 1 The diagram shown is one of the flowcharts of an industrial sensor data anomaly detection method according to an embodiment of this application. The method includes:

[0065] Step 101: Obtain multiple test samples from the first dataset collected by the industrial sensors;

[0066] Step 102: Input multiple test samples into the target anomaly detection model to determine the anomaly score of each test sample. The target anomaly detection model is obtained by embedding a Bi-directional Long Short-Term Memory (BiLSTM) model into a Generative Adversarial Network (GAN).

[0067] It's important to note that the BiLSTM model can extract temporal dependencies between time-domain data. In this step, the BiLSTM model is embedded into a GAN, allowing both the generator and discriminator of the GAN to simultaneously incorporate the BiLSTM model. This enables the extraction of temporal dependencies from test samples when inputting them into the target anomaly detection model to determine their anomaly scores. Furthermore, the characteristics of Bi-LSTM-GAN can be leveraged to automatically remove irrelevant features. This effectively avoids the cumbersome feature engineering process of existing technologies, reduces reliance on feature engineering, and achieves automatic feature selection and filtering while maintaining model accuracy. It can extract temporal dependencies from data collected by industrial sensors, effectively improving the robustness of the anomaly detection model.

[0068] Step 103: Based on the clustering algorithm and the anomaly score, perform anomaly detection on multiple test samples.

[0069] It should be noted that the clustering algorithm mentioned here can be the K-means clustering algorithm, but is not limited to it.

[0070] Compared to most current methods that determine anomalies by setting thresholds, this step uses clustering algorithms to determine anomalies in test samples. This avoids the need for users to constantly debug to find a suitable threshold, which consumes a lot of effort. Furthermore, the continuous parameter tuning process may lead to insufficient model generalization ability, meaning that the final model cannot generalize to new data.

[0071] The industrial sensor data anomaly detection method of this application embodiment first obtains multiple test samples from a first dataset collected by industrial sensors; second, inputs the multiple test samples into a target anomaly detection model to determine the anomaly score of each test sample, wherein the target anomaly detection model is obtained by embedding a BiLSTM model into a Generative Adversarial Network (GAN); finally, based on a clustering algorithm and the anomaly scores, anomaly detection is performed on the multiple test samples. Thus, firstly, by obtaining the target anomaly detection model by embedding a BiLSTM model into a GAN and using the target anomaly detection model to detect anomalies in the test samples, this method effectively avoids the cumbersome feature engineering process in existing technologies, achieving automatic feature selection and filtering while ensuring model accuracy; secondly, it can extract the time dependencies in the data collected by industrial sensors, effectively improving the robustness of the anomaly detection model; and thirdly, by using a clustering algorithm to adaptively detect anomalies in the test samples, it achieves clustering of one-dimensional anomaly score data from sensor data, avoiding the "curse of dimensionality" that traditional algorithms easily fall into when processing large datasets.

[0072] As an optional implementation, step 101 involves obtaining test samples from the data acquired by the industrial sensors, including:

[0073] Based on the pre-set size and step size of the first sliding window, the first dataset is divided into multiple first sub-time series;

[0074] One of the first sub-time series is identified as one of the test samples.

[0075] In this optional implementation, taking the steel smelting scenario as an example, the equipment data that each sensor can collect includes, for example, current, pressure, temperature, location, and sensor network traffic data (TCP connection duration, protocol type, number of times accessing system open and dark files and directories, number of failed login attempts, number of connections with the same target host as the current connection within a certain period of time, etc.). These data are used as input features and belong to multivariate time series. The size of the first sliding window is set to W and the sliding step size is S. The first dataset is divided into multiple multivariate time subsequences of equal length, and each subsequence is used as a test sample.

[0076] For example, if the first dataset includes 7 sets of data (a0, a1, a2, a3, a4, a5, a6), the sliding window size is 5, and the sliding step size is 1, then the test samples obtained by this optional implementation method are as follows: (a0, a1, a2, a3, a4), (a1, a2, a3, a4, a5), (a2, a3, a4, a5, a6).

[0077] As an optional implementation, step 102 involves inputting multiple test samples into the target anomaly detection model to determine the anomaly score for each test sample, including:

[0078] The random space sample is input into the target generator of the target anomaly detection model so that the target generator generates the optimal reconstruction test sample, wherein the random space sample is the test sample mapped to the random latent space;

[0079] Specifically, this step involves mapping the test sample to the latent vector space where the random noise resides to find the random noise (random space sample) that is closest to the test sample. Then, the noise that is closest to the test sample is input into the target generator to generate multiple reconstructed test samples, and the optimal reconstructed test sample is determined from the multiple reconstructed test samples.

[0080] Determine the optimal reconstruction test sample and the reconstruction loss of the test sample;

[0081] Specifically, this step involves determining the reconstruction loss using the following formula:

[0082]

[0083] Where, x test,i For the i-th test sample, G(z) *,i ) represents the optimal reconstructed test sample corresponding to the i-th test sample.

[0084] The test sample is input into the target discriminator of the target anomaly detection model to determine the discrimination loss;

[0085] Specifically, this step involves determining the discriminant loss according to the following formula:

[0086] loss discrimination =D(X) test )

[0087] Based on the reconstruction loss and the discrimination loss, the anomaly score of each test sample is determined.

[0088] Specifically, this step involves determining the anomaly score using the following formula:

[0089] L total =λloss rescontruction +(1-λ)loss discrimination

[0090] Among them, L total Let λ be the anomaly score, and λ be the relative importance of the loss function. rescontruction To reconstruct the loss, loss discrimination To determine the loss.

[0091] It should be noted that when the anomaly score approaches 1, the test sample is more likely to be an anomalous sample; conversely, when the anomaly score approaches 0, the test sample is more likely to be a normal sample.

[0092] In this optional implementation, the target generator and target discriminator are obtained by training the generator and discriminator of the GAN. In this way, the information of the trained generator and discriminator is fully utilized, thereby improving the performance of the anomaly detection model and increasing the accuracy of sensor anomaly detection.

[0093] As an optional implementation, random space samples are input into the target generator of the target anomaly detection model to enable the target generator to generate optimal reconstruction test samples, including:

[0094] (A) Extract the contextual temporal features of the random spatial samples;

[0095] Here, it's important to note that the BiLSTM model embedded in the GAN is used for both fake sample generation and discrimination. Sensor data (such as that from a steel ladle refining furnace) is typically large and contains significant contextual temporal information. To remove unimportant information and extract contextual temporal features, the BiLSTM model is employed. This model not only incorporates the LSTM's "forget gate" (for discarding unimportant information) and "memory gate" (for retaining important information) but also extracts the contextual information of the features. The LSTM network structure is as follows: Figure 2 As shown, where:

[0096]

[0097]

[0098]

[0099]

[0100]

[0101] in, The symbol W represents the Hadamard product, which is the element-wise multiplication of matrices. f For the weight of the forget gate, W i W represents the input gate weights. o x is the output gate weight. t For input, h t-1 and C t-1 Both contain information from the neurons in the previous layer, but C t-1 It will continue throughout the entire lifecycle of LSTM, b i b f b c b o For bias.

[0102] In addition, such as Figure 3 The diagram shows the structure of the BiLSTM model, which is composed of a forward LSTM and a backward LSTM. This allows the acquisition of contextual information from time series data.

[0103] In other words, in this step, by inputting random space samples into a target generator embedded with a BiLSTM model, the contextual information of the random space samples can be extracted. This solves the problem that the processing of time series data is often ignored and the time dependencies of time series data cannot be fully utilized. It achieves a good extraction of the time dependencies between sensor-acquired data.

[0104] (B) Generate reconstructed test samples based on the contextual temporal characteristics of the random spatial samples;

[0105] (C) Based on the similarity between the reconstructed test sample and the test sample corresponding to the reconstructed test sample, update the reconstructed test sample using the gradient descent method to generate the optimal reconstructed test sample.

[0106] The two steps described above can be specifically as follows: First, based on the contextual temporal features, a reconstructed test sample is randomly generated. The similarity between this reconstructed test sample and its corresponding test sample is calculated. Then, based on the calculated similarity results, the reconstructed test sample is updated using gradient descent until the optimal reconstructed test sample is obtained. Specifically, this can be achieved using the formula... Calculate the similarity between the reconstructed test sample and the corresponding test sample, where G(Z) represents the reconstructed test sample, and X represents the similarity between the reconstructed test sample and the corresponding test sample. test This is a test sample.

[0107] As an optional implementation, the test sample is input into the target discriminator of the target anomaly detection model to determine the discrimination loss, including:

[0108] Extract the contextual temporal features of the test samples;

[0109] The discrimination loss is determined based on the contextual temporal features of the test samples.

[0110] Similarly, the target discriminator embeds a BiLSTM model. Through the structure and performance of the BiLSTM model, it is possible to extract the contextual information of the test samples input to the target discriminator. The extraction method is similar to that of the target generator, and will not be elaborated here.

[0111] As an optional implementation, step 103 involves performing anomaly detection on multiple test samples based on a clustering algorithm and the anomaly scores, including:

[0112] The highest and lowest anomaly scores among the anomaly scores are determined as the initial cluster centers;

[0113] Here, it should be noted that, taking K-means clustering as an example, since K-means clustering is unaware of whether the input sensor data is abnormal, selecting the initial cluster centers is difficult. Traditional K-means clustering uses random selection. However, in the BiLSTM-GAN framework (target anomaly detection model) of this application, since the anomaly scores of all test samples have been obtained, it is equivalent to knowing which test samples are likely to be anomalous. Therefore, in this optional implementation, the initial cluster centers are set to the highest anomaly score Max(L). total) and minimum anomaly score Min(L total This ensures that the distance between the two initial cluster centers is maximized, thus solving the problem of instability caused by the selection of initial cluster centers in the K-means clustering algorithm, while also ensuring that the algorithm can converge quickly.

[0114] Based on the distance from each of the abnormal scores to the two initial cluster centers, the test samples corresponding to the abnormal scores are classified.

[0115] Specifically, this step involves classifying the test samples corresponding to abnormal scores into the class defined by the nearest initial cluster center.

[0116] Each cluster center is recalculated, and the test samples corresponding to each abnormal score are classified according to the distance from each abnormal score to each recalculated cluster center, until each cluster center meets the pre-set termination condition.

[0117] In other words, during the process of anomaly detection of test samples based on the K-means clustering algorithm and anomaly scores, cluster centers are repeatedly calculated. After each calculation, the test samples are reclassified based on the distance from the anomaly score to the current cluster center. This process continues until the cluster centers meet a pre-set termination condition, at which point the clustering process is considered complete. This achieves the adaptive division of the test samples into two classes. The pre-set termination condition can be that the position of the cluster centers no longer changes, or that the distance between two adjacent cluster centers is less than a preset value.

[0118] Based on the classification of the test samples, it is determined whether the test samples are abnormal.

[0119] Specifically, test samples located in the cluster containing cluster centroids with higher outlier scores are considered outlier samples, while test samples located in the cluster containing cluster centroids with lower outlier scores are considered normal samples.

[0120] In this alternative implementation, on the one hand, the output of the target anomaly detection model is used as the input to the K-means clustering algorithm, improving the overall performance of the model. On the other hand, the anomaly detection method is improved based on the K-means clustering algorithm, achieving adaptive anomaly detection without the need for additional manual adjustment of the detection threshold. Furthermore, clustering the anomaly scores of data collected by industrial sensors based on the K-means clustering algorithm avoids the "curse of dimensionality" that traditional K-means clustering algorithms easily fall into when dealing with large datasets. Finally, the initial cluster center selection is slowed down, optimizing the selection of the initial cluster centers and solving the problem of unstable algorithm results caused by improper selection of initial cluster centers in the K-means clustering algorithm, while ensuring that the algorithm can converge quickly.

[0121] Furthermore, as an optional implementation, the method also includes:

[0122] Training samples are obtained from a second dataset acquired by industrial sensors, wherein the first dataset is different from the second dataset.

[0123] In this step, different datasets are used to train and test the model, avoiding the problem of inaccurate test results caused by using the same dataset for training and testing.

[0124] The anomaly detection model is iteratively trained based on the training samples to obtain the target anomaly detection model.

[0125] In this embodiment, the trained target anomaly detection model is used to test the test samples. This fully utilizes the information from the trained generator (target generator) and discriminator (target discriminator) to calculate the reconstruction loss and discrimination loss, thereby obtaining the anomaly score of the test samples. This improves model performance and enhances the accuracy of anomaly detection for industrial sensors.

[0126] It should be noted that the acquired test data of industrial sensors can be divided into training set and test set. First, the anomaly detection model is trained using the training set, and then the trained anomaly detection model (target anomaly detection model) is used to test the test set. This can improve the accuracy of the test. Alternatively, the anomaly detection model can be trained only when needed. This application embodiment does not limit the timing of training the anomaly detection model.

[0127] It should also be noted that before training the anomaly detection model, it is necessary to first establish the anomaly detection model. The process of establishing the anomaly detection model specifically includes: establishing a BiLSTM model and embedding the BiLSTM model into the GAN network, that is, both the discriminator and the generator in the GAN network use the BiLSTM model to construct the BiLSTM-GAN anomaly detection model.

[0128] The BiLSTM-GAN anomaly detection model consists of two layers:

[0129] The inner layer is a BiLSTM model that serves as both a generator and a discriminator (e.g., ...). Figure 2 and Figure 3 (As shown), used for generating and identifying fake samples;

[0130] The outer layer is like Figure 4 The GAN network framework shown.

[0131] As an optional implementation, training samples are obtained from the second dataset acquired by industrial sensors, including:

[0132] Based on the pre-set size and step size of the second sliding window, the second dataset is divided into multiple second sub-time series;

[0133] One of the second sub-time series is determined as one of the training samples.

[0134] It should be noted that the process of dividing the second dataset into multiple second sub-time series is similar to the process of dividing the first dataset into multiple first sub-time series. To avoid repetition, it will not be described again here.

[0135] As an optional implementation, the anomaly detection model is iteratively trained based on the training samples to obtain the target anomaly detection model, including:

[0136] Random noise is input into the generator of the anomaly detection model to obtain generated samples;

[0137] In this step, random noise is the sample in the random space. By inputting the sample in the random space into the generator, the generator can generate a "fake sample" M(z), i.e., anomaly, based on the sample in the random space. The form of the M(z) sample is consistent with the network input, and its value is composed of random noise.

[0138] The training samples and the generated samples are input into the discriminator of the anomaly detection model, the parameters of the discriminator are adjusted, and the discrimination result is output.

[0139] Based on the discrimination result, the generator and the discriminator are iteratively trained until the target anomaly detection model is obtained, wherein the parameters include at least one of forget gate weight, input gate weight, output gate weight and bias term.

[0140] In this optional implementation, the discriminator D learns in an unsupervised manner, receiving samples from the generator and determining whether each sample is a real sample. During training, the generator G aims to generate as realistic fake samples as possible to mislead the discriminator D into thinking they are real samples, while the discriminator D aims to identify as many "fake samples" generated by the generator G as possible from "real samples." The two form a dynamic game, and through continuous iteration, both improve their capabilities. Ultimately, the discriminator D can accurately identify which samples are anomalous and can be directly used as an anomaly detection tool, while the generator G can generate samples that are very close to reality, used to reconstruct test samples in the subsequent anomaly detection stage.

[0141] As a specific implementation, random noise is input into the generator to obtain generated samples, including:

[0142] Extract the contextual temporal features of the random noise;

[0143] Specifically, this step involves extracting contextual temporal features from random noise based on the characteristics of the BiLSTM model embedded in the generator.

[0144] The generated sample is obtained based on the contextual temporal characteristics of the random noise.

[0145] In this optional implementation, by extracting the contextual temporal features of random noise and obtaining the generated samples based on these features, the temporal dependencies of the random noise can be extracted, and irrelevant features can be removed, thereby improving the performance and robustness of the model.

[0146] As an optional implementation, the training samples and the generated samples are input into the discriminator of the anomaly detection model, the parameters of the discriminator are adjusted, and the discrimination result is output, including:

[0147] Extract the contextual temporal features of the training samples;

[0148] Specifically, this step involves using the characteristics of the BiLSTM model embedded in the discriminator to remove unimportant information from the training samples and extract the contextual temporal features of the training samples.

[0149] Based on the contextual temporal features of the training samples, the training samples are discriminated to adjust the parameters of the discriminator and output the discrimination result corresponding to the training samples;

[0150] The generated samples are judged to adjust the parameters of the discriminator and output the judgment result corresponding to the generated samples.

[0151] As an optional implementation, based on the discrimination result, the generator and the discriminator are iteratively trained until the target anomaly detection model is obtained, including:

[0152] The sample whose discrimination result is to be generated is input into the generator, the parameters of the generator are adjusted based on the gradient descent method, and the generation result is output.

[0153] The generated result and the discrimination result are used as training samples, which are then input into the adjusted discriminator. The parameters of the discriminator are adjusted again based on the gradient descent method. The discrimination result is output, and the process returns to the steps of inputting the discrimination result as the generated sample into the generator, adjusting the parameters of the generator, and outputting the generated result, so as to iteratively train the generator and discriminator of the anomaly detection model until the target anomaly detection model is obtained.

[0154] In other words, during the training of the anomaly detection model, the parameters of the generator are first fixed, and the discriminator is trained to update the parameters of the discriminator. Then, the parameters of the discriminator are fixed again, and the generator is trained to update the parameters of the generator. In this way, the discriminator and the generator are trained iteratively.

[0155] The training process of the anomaly detection model is briefly explained below:

[0156] Obtain training samples;

[0157] Random noise is input into the generator to obtain generated samples;

[0158] The training samples and generated samples are input into the discriminator to obtain anomaly scores;

[0159] The parameters of the generator and discriminator are updated based on the anomaly score to obtain the target anomaly detection model.

[0160] It should be noted here that the main training parameter of the anomaly detection model is the number of layers in the LSTM network, and the empirical value is generally no more than 3 layers.

[0161] The specific training process is as follows:

[0162] Determine if the number of iterations is less than the preset number; if so, generate a sample composed of random noise.

[0163] First, fix the generator, then maximize V(D, G) using gradient descent (learning rate = 0.1), and update network parameters such as discriminator weights:

[0164]

[0165] Where x follows the distribution of the second dataset (training set); z follows the distribution of random noise; D(x) represents the probability that the discriminator identifies whether the real dataset is real, and D(G(z)) represents whether the discriminator judges whether the data generated by the generator is real.

[0166] With the discriminator fixed, based on gradient descent, minimize V(D, G) and update network parameters such as generator weights:

[0167]

[0168] Finally, save the network parameters of all generators and discriminators after training.

[0169] The following describes the industrial sensor data anomaly detection method of this application embodiment, using sensor data from a ladle refining furnace process as an example:

[0170] First, set the sliding window size and step size to divide the dataset into multiple time subsequences;

[0171] Secondly, a BiLSTM model is trained using sensor data (argon blowing rate, ladle temperature, wire feeding rate, and furnace pressure) and a GAN network to obtain the contextual temporal features of the ladle refining furnace sensor data. It should be noted that the BiLSTM model, in addition to determining the activation function (such as...), also needs to consider the contextual temporal features of the sensor data. Figure 2 As shown, apart from the tanh activation function, loss function (cross-entropy loss function), and dropout rate of each network node (dropout = 0.2), the main training parameter of BiLSTM is the number of layers in the LSTM network, which is generally no more than 3 layers.

[0172] Next, the GAN model is trained and its parameters such as weights are saved. The output of this model is used as the input of the next layer of improved K-means clustering algorithm to optimize the selection of the initial cluster centers and to cluster the abnormal scores of all samples. This achieves the goal of judging whether the temperature of the current batch of steel in the ladle refining furnace is abnormal when it is tapped in the future, based on the context and time-series features under multidimensional data. It should be noted that the main training parameter of the GAN model is the number of iterations.

[0173] It should be noted that since the tapping temperature affects the casting quality of steel products, the goal of this anomaly detection is to detect abnormal temperatures that may occur during the tapping of this batch of steel based on existing sensor data. If an anomaly is detected, it will be indicated so that subsequent adjustments can be made to stabilize the tapping temperature of the molten steel in advance.

[0174] like Figure 5 As shown in the illustration, this application also provides an industrial sensor data anomaly detection device, comprising:

[0175] The first acquisition module 501 is used to acquire multiple test samples from the first dataset collected by the industrial sensor;

[0176] The determination module 502 is used to input multiple test samples into the target anomaly detection model and determine the anomaly score of each test sample, wherein the target anomaly detection model is obtained by embedding a dual-ended long short-term memory network (BiLSTM) model into a generative adversarial network (GAN).

[0177] The detection module 503 is used to perform anomaly detection on multiple test samples based on a clustering algorithm and the anomaly score.

[0178] The industrial sensor data anomaly detection device of this application embodiment first acquires multiple test samples from a first dataset collected by industrial sensors; second, a determining module 502 inputs the multiple test samples into a target anomaly detection model to determine the anomaly score of each test sample, wherein the target anomaly detection model is obtained by embedding a BiLSTM model into a Generative Adversarial Network (GAN); finally, a detection module 503 performs anomaly detection on the multiple test samples based on the K-means clustering algorithm and the anomaly scores. Thus, firstly, the target anomaly detection model obtained by embedding a BiLSTM model into a GAN in this application embodiment, and the anomaly detection of test samples using the target anomaly detection model, effectively avoids the cumbersome feature engineering process in the prior art, achieving automatic feature selection and filtering while ensuring model accuracy; secondly, it can extract the time dependency relationship in the data collected by industrial sensors, effectively improving the robustness of the anomaly detection model; and thirdly, it uses the K-means clustering algorithm to adaptively determine anomalies in the test samples.

[0179] Optionally, the first acquisition module 501 is specifically used for:

[0180] Based on the pre-set size and step size of the first sliding window, the first dataset is divided into multiple first sub-time series;

[0181] One of the first sub-time series is identified as one of the test samples.

[0182] Optionally, the determining module 502 includes:

[0183] The first generation submodule is used to input random space samples into the target generator of the target anomaly detection model so that the target generator generates the optimal reconstruction test sample, wherein the random space sample is a sample mapped from the test sample to the random latent space;

[0184] The first determining submodule is used to determine the optimal reconstruction test sample and the reconstruction loss of the test sample;

[0185] The second determination submodule is used to input the test sample into the target discriminator of the target anomaly detection model to determine the discrimination loss;

[0186] The third determination submodule is used to determine the anomaly score of each of the test samples based on the reconstruction loss and the discrimination loss.

[0187] Optionally, the first generation submodule includes:

[0188] The first extraction unit is used to extract the contextual temporal features of the random spatial samples;

[0189] The first generation unit is used to generate reconstructed test samples based on the contextual temporal features of the random spatial samples;

[0190] The second generation unit is used to update the reconstructed test sample based on the similarity between the reconstructed test sample and the test sample corresponding to the reconstructed test sample, and generate the optimal reconstructed test sample.

[0191] Optionally, the second determining submodule includes:

[0192] The second extraction unit is used to extract the contextual temporal features of the test sample;

[0193] The first determining unit is used to determine the discrimination loss based on the contextual temporal features of the test sample.

[0194] Optionally, the determination module 503 includes:

[0195] The fourth determination submodule is used to determine the highest and lowest abnormal scores among the abnormal scores as the initial cluster centers;

[0196] The processing submodule is used to classify the test samples corresponding to the abnormal scores based on the distances from each abnormal score to the two initial cluster centers.

[0197] The calculation submodule is used to recalculate each cluster center and classify the test samples corresponding to each abnormal score according to the distance from each abnormal score to each recalculated cluster center, until each cluster center meets the preset termination condition.

[0198] The determination submodule is used to detect whether the test sample is abnormal based on the classification of the test sample.

[0199] Optionally, the device further includes:

[0200] The second acquisition module is used to acquire training samples from the second dataset collected by the industrial sensor, wherein the first dataset is different from the second dataset;

[0201] The third acquisition module is used to iteratively train the anomaly detection model based on the training samples to obtain the target anomaly detection model.

[0202] Optionally, the second acquisition module is specifically used for:

[0203] Based on the pre-set size and step size of the second sliding window, the second dataset is divided into multiple second sub-time series;

[0204] One of the second sub-time series is determined as one of the training samples.

[0205] Optionally, the third acquisition module includes:

[0206] The first acquisition submodule is used to input random noise into the generator of the anomaly detection model to obtain generated samples;

[0207] The second processing submodule is used to input the training samples and the generated samples into the discriminator of the anomaly detection model, adjust the parameters of the discriminator, and output the discrimination result;

[0208] The third processing submodule is used to iteratively train the generator and the discriminator based on the discrimination result until the target anomaly detection model is obtained, wherein the parameters include at least one of forget gate weight, input gate weight, output gate weight and bias term.

[0209] Optionally, the first acquisition submodule includes:

[0210] The third extraction unit is used to extract the contextual temporal features of the random noise;

[0211] The first acquisition unit is used to obtain the generated sample based on the contextual temporal features of the random noise.

[0212] Optionally, the second processing submodule includes:

[0213] The fourth extraction unit is used to extract the contextual temporal features of the training samples;

[0214] The first processing unit is configured to discriminate the training samples based on the contextual temporal features of the training samples, so as to adjust the parameters of the discriminator and output the discrimination result corresponding to the training samples;

[0215] The second processing unit is used to discriminate the generated samples, adjust the parameters of the discriminator, and output the discrimination result corresponding to the generated samples.

[0216] Optionally, the third processing submodule includes:

[0217] The third processing unit is used to input the samples whose discrimination result is to be generated into the generator, adjust the parameters of the generator based on the gradient descent method, and output the generation result;

[0218] The fourth processing unit is used to input the generated result and the sample whose discrimination result is the training sample into the adjusted discriminator, readjust the parameters of the discriminator based on the gradient descent method, output the discrimination result, and return to the steps of inputting the sample whose discrimination result is the generated sample into the generator, adjusting the parameters of the generator, and outputting the generated result, so as to iteratively train the generator and discriminator of the anomaly detection model until the target anomaly detection model is obtained.

[0219] like Figure 6 As shown in the figure, this application embodiment also provides an industrial sensor data anomaly detection device, including: a processor 600, a memory 620 and a program stored in the memory 620 and executable on the processor 600. When the program is executed by the processor, it implements the various processes of the industrial sensor data anomaly detection method embodiment described above and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0220] The transceiver 610 is used to receive and send data under the control of the processor 600.

[0221] Among them, Figure 6 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 600 and memory represented by memory 620 together. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 610 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 630 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0222] The processor 600 is responsible for managing the bus architecture and general processing, while the memory 620 can store the data used by the processor 600 when performing operations.

[0223] This application also provides a readable storage medium storing a program. When executed by a processor, this program implements the various processes described in the above-described industrial sensor data anomaly detection method embodiment, achieving the same technical effect. To avoid repetition, it will not be described again here. The readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0224] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0225] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting anomalies in industrial sensor data, characterized in that, include: Multiple test samples are obtained from the first dataset collected by the industrial sensors; Multiple test samples are input into the target anomaly detection model to determine the anomaly score of each test sample. The target anomaly detection model is obtained by embedding a BiLSTM network model into a Generative Adversarial Network (GAN). Based on the clustering algorithm and the anomaly score, anomaly detection is performed on multiple test samples; The process involves inputting multiple test samples into a target anomaly detection model to determine the anomaly score for each test sample, including: The random space sample is input into the target generator of the target anomaly detection model so that the target generator generates the optimal reconstruction test sample, wherein the random space sample is the test sample mapped to the random latent space; Determine the optimal reconstruction test sample and the reconstruction loss of the test sample; The test sample is input into the target discriminator of the target anomaly detection model to determine the discrimination loss; Based on the reconstruction loss and the discrimination loss, the anomaly score of each test sample is determined.

2. The method according to claim 1, characterized in that, From the data collected by industrial sensors, test samples are obtained, including: Based on the pre-set size and step size of the first sliding window, the first dataset is divided into multiple first sub-time series; One of the first sub-time series is identified as one of the test samples.

3. The method according to claim 1, characterized in that, Inputting random space samples into the target generator of the target anomaly detection model to enable the target generator to generate optimal reconstruction test samples includes: Extract the contextual temporal features of the random spatial samples; Based on the contextual temporal characteristics of the random spatial samples, reconstructed test samples are generated; Based on the similarity between the reconstructed test sample and the test sample corresponding to the reconstructed test sample, the reconstructed test sample is updated using the gradient descent method to generate the optimal reconstructed test sample.

4. The method according to claim 1, characterized in that, The test sample is input into the target discriminator of the target anomaly detection model to determine the discrimination loss, including: Extract the contextual temporal features of the test samples; The discrimination loss is determined based on the contextual temporal features of the test samples.

5. The method according to claim 1, characterized in that, Based on the clustering algorithm and the anomaly scores, anomaly detection is performed on multiple test samples, including: The highest and lowest anomaly scores among the anomaly scores are determined as the initial cluster centers; Based on the distance from each of the abnormal scores to the two initial cluster centers, the test samples corresponding to the abnormal scores are classified. Each cluster center is recalculated, and the test samples corresponding to each abnormal score are classified according to the distance from each abnormal score to each recalculated cluster center, until each cluster center meets the pre-set termination condition. Based on the classification of the test samples, it is determined whether the test samples are abnormal.

6. The method according to claim 1, characterized in that, The method further includes: Training samples are obtained from a second dataset acquired by industrial sensors, wherein the first dataset is different from the second dataset. The anomaly detection model is iteratively trained based on the training samples to obtain the target anomaly detection model.

7. The method according to claim 6, characterized in that, From the second dataset collected by industrial sensors, training samples are obtained, including: Based on the pre-set size and step size of the second sliding window, the second dataset is divided into multiple second sub-time series; One of the second sub-time series is determined as one of the training samples.

8. The method according to claim 6, characterized in that, The anomaly detection model is iteratively trained based on the training samples to obtain the target anomaly detection model, including: Random noise is input into the generator of the anomaly detection model to obtain generated samples; The training samples and the generated samples are input into the discriminator of the anomaly detection model, the parameters of the discriminator are adjusted, and the discrimination result is output. Based on the discrimination result, the generator and the discriminator are iteratively trained until the target anomaly detection model is obtained, wherein the parameters include at least one of forget gate weight, input gate weight, output gate weight and bias term.

9. The method according to claim 8, characterized in that, Random noise is input into the generator to obtain generated samples, including: Extract the contextual temporal features of the random noise; The generated sample is obtained based on the contextual temporal characteristics of the random noise.

10. The method according to claim 8, characterized in that, The training samples and the generated samples are input into the discriminator of the anomaly detection model, the parameters of the discriminator are adjusted, and the discrimination result is output, including: Extract the contextual temporal features of the training samples; Based on the contextual temporal features of the training samples, the training samples are discriminated to adjust the parameters of the discriminator and output the discrimination result corresponding to the training samples; The generated samples are judged to adjust the parameters of the discriminator and output the judgment result corresponding to the generated samples.

11. The method according to claim 8, characterized in that, Based on the discrimination result, the generator and the discriminator are iteratively trained until the target anomaly detection model is obtained, including: The sample whose discrimination result is to be generated is input into the generator, the parameters of the generator are adjusted based on the gradient descent method, and the generation result is output. The generated result and the discrimination result are used as training samples, which are then input into the adjusted discriminator. The parameters of the discriminator are adjusted again based on the gradient descent method. The discrimination result is output, and the process returns to the steps of inputting the discrimination result as the generated sample into the generator, adjusting the parameters of the generator, and outputting the generated result, so as to iteratively train the generator and discriminator of the anomaly detection model until the target anomaly detection model is obtained.

12. An industrial sensor data anomaly detection device, characterized in that, include: The first acquisition module is used to acquire multiple test samples from the first dataset collected by the industrial sensor; The determination module is used to input multiple test samples into the target anomaly detection model and determine the anomaly score of each test sample, wherein the target anomaly detection model is obtained by embedding a BiLSTM network model into a generative adversarial network (GAN). The detection module is used to perform anomaly detection on multiple test samples based on a clustering algorithm and the anomaly score; The determining module includes: The first generation submodule is used to input random space samples into the target generator of the target anomaly detection model so that the target generator generates the optimal reconstruction test sample, wherein the random space sample is a sample mapped from the test sample to the random latent space; The first determining submodule is used to determine the optimal reconstruction test sample and the reconstruction loss of the test sample; The second determination submodule is used to input the test sample into the target discriminator of the target anomaly detection model to determine the discrimination loss; The third determination submodule is used to determine the anomaly score of each of the test samples based on the reconstruction loss and the discrimination loss.

13. An industrial sensor data anomaly detection device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the industrial sensor data anomaly detection method as described in any one of claims 1 to 11.

14. A readable storage medium, characterized in that, The readable storage medium stores a program that, when executed by a processor, implements the industrial sensor data anomaly detection method as described in any one of claims 1 to 11.

Images