A method for industrial process imbalance fault classification based on time series generation

Abstract

The present invention discloses a method for classifying unbalanced faults in industrial processes based on time series generation. The method comprises: constructing and training a deep generative model for each fault based on time series industrial process data to generate a certain amount of fault time series data, thereby balancing the normal time series data and fault time series data in the industrial process; the deep generative model comprises a convolutional decoder, a convolutional decoder, a trend multilayer perceptron, and a seasonal multilayer perceptron; the output of the deep generative model is the sum of the outputs of the convolutional decoder and all the multilayer perceptrons; and constructing and training a classifier based on the balanced industrial process time series data to perform fault classification; the classifier comprises a long short-term memory network for extracting high-dimensional spatial features, and a multilayer perceptron and a softmax function for fault classification. The present invention can classify rare faults that occur in dynamic processes, and has high stability, good robustness, and excellent generalization performance.

Description

Technical Field

[0001] The present invention belongs to the field of industrial processes and relates to an industrial process imbalance fault classification method based on time series generation. Background Art

[0002] In today's industrial landscape, with the rapid development of information technology and big data, the monitoring and management of industrial processes are becoming increasingly intelligent and automated. However, due to the complexity and variability of industrial processes, failures remain a major factor affecting production efficiency and quality. Therefore, developing an efficient and accurate fault classification method is crucial.

[0003] Traditional fault classification methods are often based on manual experience and rules, limited by expert knowledge and experience, and difficult to adapt to the diverse and rapidly changing industrial environment. However, with the continuous accumulation of data in industrial processes, the use of big data analysis and artificial intelligence technologies can extract potential correlations and patterns from massive amounts of industrial data, providing a more reliable basis for fault classification. However, existing classification methods for image processing and signal processing are difficult to directly apply to industrial process fault classification because these methods are mostly constructed in scenarios with uniform samples across categories. In fault classification, the amount of fault data is often much smaller than that of normal data, making it difficult to directly apply conventional classification methods. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention proposes an industrial process imbalance fault classification method based on time series generation. It utilizes the time series data of serialized rare faults, maintains the dynamic relationship of data, reuses and generates the time series data of faults, etc., and can classify rare faults occurring in the dynamic process. It has high stability, good robustness and excellent generalization performance.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for classifying industrial process imbalance faults based on time series generation, the method comprising the following steps:

[0007] Step 1: Based on the time series industrial process data, for each fault, a deep generative model is constructed and trained to generate a certain amount of time series data of faults to balance the normal time series data and the fault time series data of the industrial process. The deep generative model includes:

[0008] A convolutional decoder based on a convolutional neural network that maps input samples to a latent space;

[0009] A convolutional decoder based on a convolutional neural network that reduces the latent variables in the latent space to the basic output of time series decomposition;

[0010] A trend multilayer perceptron that converts latent variables in the latent space into trend outputs;

[0011] Converting the latent variables in the latent space into multiple seasonal intermediate vectors to obtain multiple seasonal multi-layer perceptrons with seasonal outputs;

[0012] The output of the deep generative model is the sum of the outputs of the convolutional decoder and all multi-layer perceptrons;

[0013] Step 2: Based on the balanced industrial process time series data, a classifier is constructed and trained to perform fault classification; the classifier includes a long short-term memory network for extracting high-dimensional spatial features, and a multi-layer perceptron and softmax function for fault classification.

[0014] Furthermore, the step 1 includes the following sub-steps:

[0015] Step (1.1): Divide the serialized industrial process data into training set input samples X train ∈R t×n And the test set input sample X test ∈R t×n , the training set input sample corresponds to the output sample set Y train ∈R 1×1 , the test set input sample corresponds to the output sample set Y test ∈R 1×1 , where t represents the length of the time series and n represents the sample dimension. The training set is used for subsequent training iterations of the deep generative model, and the test set is used for subsequent evaluation of the deep generative model;

[0016] Step (1.2): Map the input sample to the hidden space through the convolutional encoder based on the convolutional neural network to obtain the hidden variable feature mean z mean and variance z sigma , the mean and variance are reparameterized to obtain the latent variable z = z mean +e×z sigma ,where e~N(0,I);

[0017] Step (1.3): Perform the following three operations in parallel:

[0018] (1) The latent variable z is restored to the basic output V of the time series decomposition through the convolution decoder based on the convolutional neural network base ;

[0019] (2) Convert the latent variable z into a trend intermediate vector U using a trend multilayer perceptron trend ; Define the time vector Time matrix R = [1, r, r 2 ,…,r p], get the trend output V of time series decomposition trend =U trend ×R, where t represents the length of the time series and p is the polynomial coefficient;

[0020] (3) Convert the latent variable z into a seasonal intermediate vector using m seasonal multilayer perceptrons Among them, m is the number of seasonal patterns; for each seasonal intermediate vector Index array A by season i =[B i ,B i ,…,B i ]Slice and expand to get seasonal output The final summation gives the seasonal output Among them, B i =[0,0…,0,1,1,…,1,…,ns i -1,ns i -1,…,ns i -1],ns i is the season number of the i-th seasonal pattern; B i By ls i 0,ls i 1,…,ls i ns i -1 composition, ls i is the seasonal length of the i-th seasonal pattern; A i Depend on B i composition;

[0021] Step (1.4): Obtain the reconstructed value of the input sample X

[0022] Step (1.5): Define the loss function of the deep generative model constructed from steps (1.2) to (1.4) as Among them L X represents the reconstruction loss, D KL Represents the latent variable loss obtained based on KL divergence;

[0023] Step (1.6): Use the training set data in step (1.1) to train the deep generative model constructed in steps (1.2) to (1.4). Use the stochastic gradient iterative algorithm to perform gradient iterative updates on each network parameter. The specific strategy is as follows:

[0024] Convolutional encoder network parameters Network parameters of the convolutional decoder Network parameters of trend multilayer perceptron Network parameters of seasonal multilayer perceptron

[0025] Step (1.7): Based on the collected fault data, a deep generation model is constructed for each fault to generate a certain amount of time series data of the fault as needed.

[0026] Furthermore, in step (1.7), the time series data generation process for a certain type of fault is as follows: a set of latent variables z is obtained by random sampling from a Gaussian distribution, and steps (1.2) to (1.4) are repeated to generate time series data for the corresponding number of faults.

[0027] Furthermore, the step 2 includes the following sub-steps:

[0028] Step (2.1): The collected serialized industrial process time series data and the fault time series data generated on demand in step 1 are used as the class-balanced industrial process time series data; these data are divided into the training set input samples X teain ∈R t ×n And the test set input sample X test ∈R t×n , the training set input sample corresponds to the output sample set Y train ∈R 1×1 , the test set input sample corresponds to the output sample set Y test ∈R 1×1 , where t represents the length of the time series and n represents the sample dimension. The training set is used for subsequent classifier training iterations, and the test set is used for subsequent classifier evaluation;

[0029] Step (2.2): Use the long short-term memory network to extract the high-dimensional spatial features V of the input sample X LSTM ;

[0030] Step (2.3): Based on high-dimensional space features V LSTM , use multi-layer perceptron and softmax function to classify faults, and get the probability P = [p1, p2, ..., p M ], where M is the number of fault categories; let the classification result where i∈{1,2,…,M};

[0031] Step (2.4): Construct cross entropy L as the loss function of the classifier constructed in steps (2.2) to (2.3), use the training set data in step (2.1) to train the classifier constructed in steps (2.2) to (2.3), and use the stochastic gradient iterative algorithm to perform gradient iterative updates on each network parameter. The specific strategy is as follows:

[0032] Long Short-Term Memory Network Parameters Multilayer Perceptron network parameters

[0033] Step (2.5): Use the classifier trained in step (2.4) to classify the industrial process time series data into faults and obtain the fault classification results.

[0034] Furthermore, when the collected industrial process raw data is non-time series raw data, it is necessary to first serialize the collected industrial process raw data in sequence according to the time series length to generate time series data.

[0035] Furthermore, the raw data of the industrial process is {x1,x2,…x n} and {y1,y2,…y n}, the time series length is t, and t data are grouped together. The original industrial process data are serialized in sequence into n-t+1 groups of time series data {x1, x2, ...x t},{x2,x3,…x t+1},…,{x n-t+1 ,x n-t+2 ,…x n} and {y t ,y t+1 ,…y n}.

[0036] The beneficial effects of the present invention are as follows:

[0037] Compared to normal industrial process time series data, fault time series data is relatively scarce. Existing classifier technologies struggle to accurately classify imbalance faults. This paper proposes a method for classifying industrial process imbalance faults based on time series generation. By generating time series data through a deep generative model and maintaining dynamic data relationships, this method effectively addresses the problem of classifying industrial process time series data under conditions of scarce fault data, achieving high classification accuracy and good generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a process flow diagram for Tennessee Eastman (TE).

[0039] Figure 2 Schematic diagram of a deep generative network.

[0040] Figure 3 This is the network structure diagram of the fault classifier. DETAILED DESCRIPTION

[0041] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more apparent. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] To address the scarcity of fault data in industrial processes, this paper proposes a time-series-based method for classifying unbalanced faults in industrial processes, effectively improving the accuracy of fault classification. The following further illustrates the effectiveness of this method using the Tennessee Eastman process (TE process).

[0043] Figure 1 The process flow diagram for the TE process is shown. The TE process consists of five operating units: a reactor, a condenser, a separator, a compressor, and a stripper. The main products are gases A, C, D, and E, an inert gas B, liquids G and H, and a byproduct F. This process has a total of 41 measured variables and 11 manipulated variables. In this example, the first 22 process variables (numbered 1-22) and 11 manipulated variables (numbered 23-33) are selected as industrial process data for fault diagnosis, resulting in a sample dimension of n = 33. See Table 1 for details.

[0044] Table 1: TE variable description

[0045]

[0046]

[0047] The training data set selected in this example includes 500 samples of normal working conditions, 400 samples of fault 1, 300 samples of fault 2, 200 samples of fault 6, and 100 samples of fault 12; the test data set includes 100 samples of normal working conditions, 100 samples of fault 1, 100 samples of fault 2, 100 samples of fault 6, and 100 samples of fault 12. And the accuracy is used Recall F1 score The fault classification effect is evaluated, where the definitions of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) are shown in Table 2.

[0048] Table 2: Confusion Matrix

[0049]

[0050] The industrial process imbalance fault classification method based on time series generation of this embodiment includes the following steps:

[0051] Step (1) serializes the industrial process data. The time series length is t = 10. After serialization, 491 training samples of normal working conditions, 391 training samples of fault 1, 291 training samples of fault 2, 191 training samples of fault 6, 91 training samples of fault 12, 91 test samples of normal working conditions, 91 test samples of fault 1, 91 test samples of fault 2, 91 test samples of fault 6, and 91 test samples of fault 12 are obtained.

[0052] Step (2): Based on the industrial process time series data obtained by serialization in step (1), four time series data deep generation models are constructed on the Tensorflow platform. The network structure diagram is as follows: Figure 2 As shown in the figure, they are used to generate 100 classification training samples for fault 1, 200 classification training samples for fault 2, 300 classification training samples for fault 6, and 400 classification training samples for fault 12, respectively. This ensures that the number of classification training samples for all categories is 491, balancing the normal time series data and fault time series data of the industrial process.

[0053] In step (2.1), prepare the original training data and test data for building four deep generative models respectively;

[0054] Step (2.2) maps the input sample X to the hidden layer space through a convolutional encoder based on a convolutional neural network to obtain the hidden variable feature mean z mean and variance z sigma , the mean and variance are reparameterized to obtain the latent variable z = z mean +e×z sigma ,where e~N(0,I);

[0055] Step (2.3): Perform the following three operations in parallel:

[0056] (1) The latent variable z is restored to the basic output V of the time series decomposition through the convolution decoder based on the convolutional neural network base ;

[0057] (2) Convert the latent variable z into a trend intermediate vector U using a trend multilayer perceptron trend ; Define the time vector p=2,time matrix R=[1,r,r 2 ,…,r p ]=[1,r,r 2 ], get the trend output V of time series decomposition trend =U trend ×R.

[0058] (3) Define the number of seasonal patterns m = 4, the number of seasons of each seasonal pattern ns = [2, 3, 4, 5], and the length of each seasonal pattern ls = [1, 1, 1, 1]. Use 4 multi-layer perceptrons to convert the latent variable z into a seasonal intermediate vector Seasonal median vector Slice and expand the seasonal index array A1 = [0, 1, 0, 1, 0, 1, 0, 1] to get the seasonal output Seasonal median vector Slice and expand the seasonal index array A2 = [0, 1, 2, 0, 1, 2, 0, 1, 2, 0] to get the seasonal output Seasonal median vector Slice and expand the seasonal index array A3 = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1] to get the seasonal output Seasonal median vector Slice and expand the seasonal index array A4 = [0, 1, 2, 3, 4, 0, 1, 2, 3, 4] to get the seasonal output The final summation gives the seasonal output

[0059] Step (2.4), obtain the reconstructed value of the input sample X

[0060] Step (2.5), define the loss function of the deep generative model constructed in the above steps (2.2) to (2.4) as Among them L X represents the reconstruction loss, D KL Represents the latent variable loss obtained based on KL divergence.

[0061] In step (2.6), the four deep generative models constructed in steps (2.2) to (2.4) are trained separately using the training set data in step (2.1). The stochastic gradient iterative algorithm is used to perform gradient iterative updates on the network parameters. The specific strategy is as follows:

[0062] Network parameters of the convolutional encoder The network parameters θ of the convolutional decoder base → Network parameters of trend multilayer perceptron Network parameters of seasonal multilayer perceptron

[0063] In step (2.7), use the deep generative model for the time series data corresponding to fault 1 to randomly sample 100 latent variables z from a Gaussian distribution. Repeat steps (2.2) to (2.4) to generate 100 classification training samples for fault 1. Use the deep generative model for the time series data corresponding to fault 2 to randomly sample 200 latent variables z from a Gaussian distribution. Repeat steps (2.2) to (2.4) to generate 200 classification training samples for fault 2. Use the deep generative model for the time series data corresponding to fault 6 to randomly sample 300 latent variables z from a Gaussian distribution. Repeat steps (2.2) to (2.4) to generate 300 classification training samples for fault 6. Use the deep generative model for the time series data corresponding to fault 12 to randomly sample 400 latent variables z from a Gaussian distribution. Repeat steps (2.2) to (2.4) to generate 400 classification training samples for fault 12.

[0064] Step (3), build a classifier based on the balanced industrial process time series data, the network structure is as follows Figure 3 As shown, fault classification is performed.

[0065] In step (3.1), the original serialized industrial process time series data and the fault time series data generated on demand in step (2) are randomly mixed as category-balanced industrial process time series data.

[0066] Step (3.2), use the long short-term memory network to extract the high-dimensional spatial features V of the input sample X LSTM .

[0067] Step (3.3): Based on high-dimensional space features V LSTM , use the multilayer perceptron and softmax function to classify faults, and get the probability P = [p1, p2, p3, p4, p5] that the input sample X belongs to each type of fault, where p1 is the probability that the input sample X belongs to normal; p2 is the probability that the input sample X belongs to fault 1; p3 is the probability that the input sample X belongs to fault 2; p4 is the probability that the input sample X belongs to fault 6; and p5 is the probability that the input sample X belongs to fault 12. Let the classification result be where i∈{1,2,3,4,5}.

[0068] In step (3.4), cross entropy L is used as the loss function of the classifier constructed in steps (3.2) to (3.3). The classifier constructed in steps (3.2) to (3.3) is trained using the training set data described in step (3.1). The stochastic gradient iterative algorithm is used to perform gradient iterative updates on each network parameter. The specific strategy is as follows:

[0069] Long Short-Term Memory Network Parameters Multilayer Perceptron network parameters

[0070] In step (3.5), the classifier trained in step (3.4) is used to classify 91 test samples of normal working conditions, 91 test samples of fault 1, 91 test samples of fault 2, 91 test samples of fault 6, and 91 test samples of fault 12. The evaluation of the fault classification effect is shown in Table 3.

[0071] Table 3: Evaluation of the fault classification effect in this example

[0072]

[0073]

[0074] If the original fault imbalanced training data is directly used to train the classifier based on the softmax function, the evaluation of its fault classification effect on the same test set is shown in Table 4.

[0075] Table 4: Evaluation of the original imbalance fault classification effect

[0076]

[0077] Compared to other fault classification methods, the proposed method for classifying unbalanced industrial process faults based on time series generation balances the number of samples of normal and faulty data by using time series data generation. This achieves better fault classification results when fault time series data is scarce.

[0078] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. A method for classifying industrial process imbalance faults based on time series generation, characterized in that: The method comprises the following steps: Step 1: Based on the time series industrial process data, for each fault, a deep generative model is constructed and trained to generate time series data of the fault, which is used to balance the normal time series data and the fault time series data in the industrial process. The deep generative model includes: A convolutional decoder based on a convolutional neural network that maps input samples to a latent space; A convolutional decoder based on a convolutional neural network that reduces the latent variables in the latent space to the basic output of time series decomposition; A trend multilayer perceptron that converts latent variables in the latent space into trend outputs; Converting the latent variables in the latent space into multiple seasonal intermediate vectors to obtain multiple seasonal multi-layer perceptrons with seasonal outputs; The output of the deep generative model is the sum of the outputs of the convolutional decoder and all multi-layer perceptrons; Step 2: Based on the balanced industrial process time series data, a classifier is constructed and trained to classify faults. The classifier includes a long short-term memory network for extracting high-dimensional spatial features, and a multi-layer perceptron and softmax function for fault classification. The step 1 includes the following sub-steps: Step (1.1): Divide the serialized industrial process data into a training set and a test set, wherein the training set is used for training iterations of the deep generative model, and the test set is used for evaluating the deep generative model; Step (1.2): Map the input sample to the hidden space through the convolutional encoder based on the convolutional neural network to obtain the mean of the latent variable feature and variance , the mean and variance are reparameterized to obtain latent variables ,in ; Step (1.3): Perform the following three operations in parallel: (1) The hidden variable Restore the basic output of temporal decomposition through the convolutional decoder based on convolutional neural network ; (2) The hidden variable Use trend multilayer perceptron to convert to trend intermediate vector ; Define the time vector , time matrix , get the trend output of time series decomposition ,in represents the length of the time series, are the polynomial coefficients; (3) The hidden variable use Seasonal multilayer perceptron is converted into seasonal intermediate vector ,in, is the number of seasonal patterns; for each seasonal intermediate vector Indexing arrays by season Slice and expand to get seasonal output , and finally sum to get the seasonal output ;in, , For the the number of seasons in a seasonal pattern; In turn, 0, 1, ..., indivual composition, For the Season length of the seasonal pattern; Depend on indivual composition; Step (1.4): Obtain input samples The reconstruction value of ; Step (1.5): Define the loss function of the deep generative model constructed from steps (1.2) to (1.4) as ,in represents the reconstruction loss, Represents the latent variable loss obtained based on KL divergence; Step (1.6): Use the training set data in step (1.1) to train the deep generative model constructed in steps (1.2) to (1.4), and use the stochastic gradient iterative algorithm to perform gradient iterative updates on each network parameter. The specific strategy is as follows: Convolutional encoder network parameters ; Network parameters of convolutional decoder ; Network parameters of trend multilayer perceptron ; Network parameters of seasonal multilayer perceptron ; Step (1.7): Based on the collected fault data, a deep generation model is constructed for each fault to generate time series data of the fault as needed.

2. The method for classifying industrial process imbalance faults based on time series generation according to claim 1, characterized in that: In step (1.7), the time series data generation process for a certain type of fault is as follows: a set of latent variables is obtained by random sampling from the Gaussian distribution , repeat steps (1.2) to (1.4) to generate time series data of the corresponding number of faults.

3. The method for classifying industrial process imbalance faults based on time series generation according to claim 2, characterized in that: The second step includes the following sub-steps: Step (2.1): The collected serialized industrial process time series data and the fault time series data generated on demand in step 1 are used as class-balanced industrial process time series data; these data are divided into a training set and a test set. The training set is used for subsequent classifier training iterations, and the test set is used for subsequent classifier evaluation. Step (2.2): Extract input samples using long short-term memory network High-dimensional space features ; Step (2.3): Based on high-dimensional space features , use multi-layer perceptron and softmax function to classify faults and get input samples The probability of belonging to each type of failure ,in is the number of fault categories; let the classification result ,in ; Step (2.4): Construct cross entropy As the loss function of the classifier constructed from step (2.2) to step (2.3), the classifier constructed from step (2.2) to step (2.3) is trained using the training set data in step (2.1). The stochastic gradient iterative algorithm is used to perform gradient iterative updates on each network parameter. The specific strategy is as follows: Long Short-Term Memory Network Parameters ; Multilayer Perceptron network parameters ; Step (2.5): Use the classifier trained in step (2.4) to classify the industrial process time series data into faults and obtain the fault classification results.

4. The method for classifying industrial process imbalance faults based on time series generation according to claim 1, characterized in that: When the collected industrial process raw data is non-time series raw data, it is necessary to first serialize the collected industrial process raw data in sequence according to the time series length to generate time series data.

5. The method for classifying industrial process imbalance faults based on time series generation according to claim 4 is characterized in that: The raw data of industrial processes are and , the time series length is t, according to The data are grouped into a group, and the original industrial process data are serialized in sequence. Group time series data and .

Images