A method for integrated anomaly detection of mechanical equipment that incorporates temporal relationships

Abstract

An integrated anomaly detection method for mechanical equipment, incorporating temporal relationships, is proposed. This method combines a Long Short-Term Memory (LSTM-FCN) network module for extracting temporal relationships with a Fully Convolutional Network (FCN) module for extracting frequency domain features to construct an integrated LSTM-FCN network model. Multiple base classifiers are iteratively generated based on the Bagging ensemble model to classify unlabeled monitoring data. This invention obtains a subset of unlabeled samples through Bagging ensemble, overcomes the classification difficulties caused by unknown labels on anomaly data by randomly reducing the contamination rate to improve performance, and leverages the powerful time-frequency domain feature extraction capabilities of the LSTM-FCN network to overcome the high demands on operator experience and technical skills required by traditional diagnostic methods. This approach enables the classification of unlabeled monitoring data and points the way for the automation and intelligentization of anomaly detection in large equipment.

Description

Technical Field

[0001] This invention belongs to the field of mechanical equipment anomaly detection technology, specifically relating to an integrated anomaly detection method for mechanical equipment that incorporates temporal relationships. Background Technology

[0002] Rotating machinery is a widely used type of machine in modern industry, found extensively in aviation, power, petroleum, and chemical industries. It often serves as a critical component of production equipment, and accidents involving it can cause significant economic losses. Therefore, the detection of anomalies in rotating machinery is of paramount importance. Among the many methods for detecting anomalies in rotating machinery, signal processing-based anomaly detection technology is currently the most widely used method in industrial settings. It relies primarily on fault characteristic frequencies and characteristic spectra as criteria. This diagnostic method requires technicians to be familiar with the fault mechanisms of the components, understand the mechanical structure of the equipment, and possess a wealth of prior knowledge. However, relying on a comprehensive diagnostic approach similar to the traditional Chinese medicine diagnostic methods of "observation, auscultation, inquiry, and palpation" results in low efficiency and accuracy, making it unsuitable for increasingly complex mechanical equipment systems.

[0003] In the field of anomaly detection, various deep learning methods have been widely used and have achieved good results. However, for data-driven anomaly detection, especially deep learning-based methods, improving model classification accuracy often requires a large amount of labeled sample data. Taking fault classification research as an example, for a network model, the larger the amount of labeled data, the richer the fault features the network can extract, and the better the final classification effect. However, in actual production, labeled data needs to be labeled manually. In the context of big data, the primary characteristic of data in the manufacturing process is its massive volume. Labeling massive amounts of data undoubtedly requires enormous human, material, and time costs, thus making it difficult to obtain a large number of labeled samples. Furthermore, in many practical industrial systems, the amount of samples from abnormal operating conditions is often insufficient.

[0004] Research shows that ensemble learning can accomplish anomaly detection tasks by constructing and combining multiple learners. The Bagging concept in ensemble learning involves perturbing the original dataset using a bootstrapping resampling method to obtain a series of subsets and train a series of classifiers, which are then combined using some aggregation technique. This method is conceptually simple, but many practices have shown its effectiveness. Packaging often improves the performance of individual classifiers when they are unrelated to each other. Therefore, the Bagging method can be used to construct anomaly detection algorithms. Meanwhile, traditional classifiers start from the mechanical structure and fault mechanisms of equipment, manually judging the bearing state based on state information from different signals such as vibration, pressure, temperature, and noise collected by condition monitoring. However, this method requires technicians to be familiar with the fault mechanisms of the components, understand the mechanical structure of the equipment, and possess a large amount of prior knowledge. It demands a high level of technical skill from the operators, making it difficult to adapt to increasingly complex mechanical equipment systems and unsuitable as a base learner for classification. Long Short-Term Memory (LSTM) networks and Fully Convolutional Networks (FCNs) possess excellent characteristics such as large-scale parallel processing, associative memory, self-organizing learning, robustness, and fault tolerance. They can effectively extract time-domain and frequency-domain features of mechanical equipment monitoring signals, achieve complex classification decisions, overcome some limitations in artificial intelligence research, and open up new avenues for intelligent diagnosis.

[0005] There are currently no published papers on using the Bagging method to integrate LSTM-FCN subnetworks to jointly discriminate unlabeled anomaly detection in mechanical equipment. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention proposes a method for detecting similar anomalies in mechanical equipment by integrating temporal relationships. This method utilizes the Bagging method to integrate multiple LSTM-FCN sub-networks for joint discrimination, thereby achieving label-free anomaly detection in mechanical equipment.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A mechanical equipment integrated anomaly detection method that integrates temporal relationships is proposed. It combines a long short-term memory network module for extracting temporal relationships and a fully convolutional network module for extracting frequency domain features to construct an integrated long short-term memory-fully convolutional (LSTM-FCN) network model. Based on the Bagging ensemble model, multiple base classifiers are generated iteratively to complete the classification and judgment of unlabeled monitoring data.

[0009] The aforementioned method for integrated anomaly detection of mechanical equipment based on time-series relationships includes the following steps:

[0010] 1) When using the integrated Long Short-Term Memory-Fully Convolutional (LSTM-FCN) network model for anomaly detection of mechanical equipment, first obtain the anomaly sample data set P and the unlabeled sample data set U;

[0011] 2) Perform random uniform sampling on the unlabeled sample set U to generate an unlabeled sample subset U. t It is considered a normal sample set, and the size of this subset is the same as the size of the abnormal sample data set;

[0012] 3) Combination P and unlabeled sample subset U t Generate a training set and use it as training data for an integrated Long Short-Term Memory-Fully Convolutional (LSTM-FCN) network model to obtain a classifier;

[0013] 4) Data that was not sampled during this sampling is called out-of-package data U. oob =UU t ;

[0014] 5) Transfer the external data U oob Input the trained classifier to obtain the predicted classification probability s(x);

[0015] 6) Repeat steps 2)-5) to perform T samplings, and finally obtain T classifiers. Accumulate the predicted classification probability s(x) of each sample in the T classifiers and average them to obtain the final classification probability.

[0016] 7) If the final classification probability is greater than 0.5, it is determined to be an abnormal sample; otherwise, it is a normal sample.

[0017] The Long Short-Term Memory (LSTM) network module is composed of multiple layers of stacked LSTM basic units. The LSTM basic units preserve long-term memory features through gate structures. The LSTM basic units consist of forget gates, input gates, output gates, and cell states.

[0018] The fully convolutional network module consists of a stack of basic network structures composed of convolutional layers, batch normalization (BN) layers, and a ReLU function. The output of the convolutional layers is symmetrical and non-sparse. The BN layers are used to accelerate the training and convergence of the network, control gradient explosion to prevent gradient vanishing, and prevent overfitting. Finally, the ReLU function is added to increase the non-linear relationship between the layers of the network.

[0019] The beneficial effects of this invention are as follows:

[0020] This invention obtains a subset of unlabeled samples through Bagging ensemble and overcomes the classification difficulties caused by unknown labels of abnormal data by randomly reducing the contamination rate to improve performance. At the same time, it utilizes the powerful time-frequency domain feature extraction capability of the LSTM-FCN network to overcome the problem that traditional diagnostic methods require high levels of operator experience and technical expertise. Thus, it completes the classification and judgment of unlabeled monitoring data, pointing the way for the automation and intelligence of anomaly detection in large equipment. Attached Figure Description

[0021] Figure 1 This is a structural diagram of the basic unit of the Long Short-Term Memory Network in an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram of the long short-term memory network module in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of a fully convolutional network module according to an embodiment of the present invention.

[0024] Figure 4 The diagram below shows the integrated Long Short-Term Memory-Fully Convolutional (LSTM-FCN) network model for an embodiment of the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] A mechanical equipment integrated anomaly detection method that integrates temporal relationships is proposed. It combines a long short-term memory network module for extracting temporal relationships and a fully convolutional network module for extracting frequency domain features to construct an integrated long short-term memory-fully convolutional (LSTM-FCN) network model. Based on the Bagging integrated model, multiple base classifiers are iteratively generated to complete the classification and judgment of unlabeled monitoring data.

[0027] Reference Figure 1 The basic unit of the Long Short-Term Memory (LSTM) network in this embodiment overcomes the risk of gradient vanishing in traditional Recurrent Neural Networks (RNNs) through a special gate structure, effectively preserving long-term memory features. It consists of a forget gate, an input gate, an output gate, and cell states. Each line in the diagram represents the path of a vector; merging two lines indicates concatenation, while forking represents copying its content, with the copy being passed to different positions. At time t, each sequence index position propagates forward along with the hidden state h. t There is also another hidden state, namely the cell state, denoted as C. t In addition, the forget gate is used in LSTM to control whether the hidden cell state of the previous layer is forgotten with a certain probability, and the input gate is used to output the hidden state h of the previous sequence. t and this sequence data x tOutput gate h t The update consists of two parts, the first part is o t That is, the hidden state h of the previous sequence. t-1 and this sequence data x t The second part consists of the hidden state C. t It consists of the Tanh activation function.

[0028] Reference Figure 2 In this embodiment, the Long Short-Term Memory (LSTM) network module is composed of three layers of basic LSTM network units stacked together. The size of each layer is [512, 256, 512]. It can construct and analyze the temporal relationship of time-series signals from multiple scales. Each LSTM module outputs a three-dimensional array, which consists of the output sequence length, the hidden layer size, and the batch size for each training session. Finally, a DropOut layer is connected after each LSTM layer to prevent overfitting.

[0029] Reference Figure 3 The fully convolutional network module in this embodiment consists of a stack of basic network structures composed of convolutional layers, batch normalization (BN) layers, and a ReLU function. The output of the convolutional layers is generally symmetrical and non-sparse, and normalization makes it more stable. The BN layers are used to accelerate the training and convergence speed of the network, control gradient explosion to prevent gradient vanishing, and prevent overfitting. Finally, the ReLU function is added to increase the nonlinear relationship between the layers of the network. Based on the above basic network structure, this invention stacks three basic network structures as the final network structure, where the filter size of each basic network structure is [128, 256, 128].

[0030] Reference Figure 4 The block diagram of the integrated Long Short-Term Memory-Fully Convolutional (LSTM-FCN) network model in this embodiment includes the entire algorithm's operation flow, including collecting data and dividing it into training and test sets, then using a mechanical device that integrates temporal relationships to integrate similar anomaly detection methods to train the model on the data, and finally using the test set to test the model's performance.

[0031] The aforementioned method for integrated anomaly detection of mechanical equipment based on time-series relationships includes the following steps:

[0032] 1) When using an integrated Long Short-Term Memory-Fully Convolutional (LSTM-FCN) network model for anomaly detection in mechanical equipment, its network structure is as follows: Figure 4As shown, it is a three-layer stacked network. First, we need to obtain the abnormal sample data set P and a large number of unlabeled sample data U. The unlabeled sample data U can be used as training data by mixing existing monitoring data with known fault types and normal sample data, or by mixing numerical simulation data of possible fault types with normal sample data. In order to consider the influence of measurement noise, appropriate noise can be added to the simulation data that generates the training data.

[0033] Step 2) Perform random uniform sampling on the unlabeled sample set U to generate an unlabeled sample subset U. t It is considered a normal sample set, and the size of this subset is the same as the size of the abnormal sample data set;

[0034] Step 3) Combine P and the unlabeled sample subset U t Generate a training set and use it as training data for an integrated Long Short-Term Memory-Fully Convolutional (LSTM-FCN) network model to obtain a classifier;

[0035] Step 4) Obtain the data that was not sampled in this sampling, which is called out-of-package data U. oob =UU t ;

[0036] Step 5) Transfer the external data U oob Input the trained classifier to obtain the predicted classification probability s(x);

[0037] Step 6) Repeat steps 2)-5) to perform sampling T times, and finally obtain T classifiers. Accumulate the predicted classification probability s(x) of each sample in the T classifiers and average them to obtain the final classification probability.

[0038] This embodiment is illustrated by... Figure 2 The stacked LSTM modules shown extract timing relationships through... Figure 3 The fully convolutional network module shown extracts frequency domain features, and the prediction probability is given by combining the output results of the two modules through the Softmax function. The final classifier is generated by iterative update.

[0039] Step 7) If the final classification probability is greater than 0.5, it is determined to be an abnormal sample; otherwise, it is a normal sample. Anomaly detection complete.

Claims

1. A method for detecting similar anomalies in mechanical equipment by incorporating temporal relationships, characterized in that: The long short-term memory network module for extracting temporal relationships and the fully convolutional network module for extracting frequency domain features are combined to construct an integrated long short-term memory-fully convolutional LSTM-FCN network model. Multiple base classifiers are then generated iteratively based on the Bagging ensemble model to complete the classification and judgment of unlabeled monitoring data. The aforementioned method for integrated anomaly detection of mechanical equipment based on time-series relationships includes the following steps: 1) When using the integrated long short-term memory-fully convolutional LSTM-FCN network model for anomaly detection of mechanical equipment, first obtain the anomaly sample data set P and the unlabeled sample data set U; 2) Perform random uniform sampling on the unlabeled sample set U to generate an unlabeled sample subset U. t It is considered a normal sample set, and the size of this subset is the same as the size of the abnormal sample data set; 3) Combine the abnormal sample dataset P and the unlabeled sample subset U t Generate a training set and use it as training data for the integrated Long Short-Term Memory-Fully Convolutional LSTM-FCN network model to obtain a classifier; 4) Data that was not sampled during this sampling is called out-of-package data U. oob =UU t ; 5) Transfer the external data U oob Input the trained classifier to obtain the predicted classification probability s(x); 6) Repeat steps 2)-5) to perform T samplings, and finally obtain T classifiers. Accumulate the predicted classification probability s(x) of each sample in the T classifiers and average them to obtain the final classification probability. 7) If the final classification probability is greater than 0.5, it is determined to be an abnormal sample; otherwise, it is a normal sample.

2. The method according to claim 1, characterized in that: The Long Short-Term Memory (LSTM) network module is composed of multiple layers of stacked LSTM basic units. The LSTM basic units preserve long-term memory features through gate structures. The LSTM basic units consist of forget gates, input gates, output gates, and cell states.

3. The method according to claim 1, characterized in that: The fully convolutional network module consists of a stack of basic network structures composed of convolutional layers, batch normalization (BN) layers, and a ReLU function. The output of the convolutional layers is symmetrical and non-sparse. The BN layers are used to accelerate the training and convergence of the network, control gradient explosion to prevent gradient vanishing, and prevent overfitting. Finally, the ReLU function is added to increase the non-linear relationship between the layers of the network.

Images