An industrial process fault diagnosis method based on near neighbor component loss optimization multi-scale convolutional neural network

Abstract

The application discloses an industrial process fault diagnosis method based on a near neighbor component loss optimized multi-scale convolutional neural network, which can be used for complex industrial process fault diagnosis. First, a multi-scale convolutional neural network framework is adopted to extract different scale space features between fault data from a large-scale range and a small-scale range, and a model is trained after multi-scale feature fusion; subsequently, a proxy near neighbor component loss function is used for training, so that the model is not only affected by positive samples in the training process, but also introduces negative samples to expand the distance between different fault categories and reduce the distance between the same fault categories; in order to verify the accuracy of the model fault diagnosis performance, the model is used for fault diagnosis of a wastewater treatment process. Experimental results show that the combination of the multi-scale convolutional neural network feature extraction method and the proxy near neighbor component loss function can obviously improve the accuracy of the model fault diagnosis, and is more suitable for complex industrial process fault diagnosis.

Description

Technical Field

[0001] This invention relates to a fault diagnosis method for wastewater treatment processes, specifically to an industrial process fault diagnosis method based on a multi-scale convolutional neural network optimized by nearest neighbor component loss. Background Technology

[0002] During long-term operation of industrial processes, various failures can occur due to equipment aging, environmental climate changes, and other factors. Since most industrial processes are continuous production processes characterized by large production volumes, complex processes, and numerous influencing factors, failure detection often involves a significant time lag. Therefore, failures can lead to serious consequences such as substandard processing results and unit failures. Timely identification of the cause of a failure and repair of the offending unit are crucial for maintaining the stability of the industrial process and saving on processing costs. Therefore, timely fault diagnosis of industrial production processes is essential.

[0003] Current fault diagnosis in industrial processes mainly falls into two categories: expert systems based on prior knowledge and data-driven model-based fault diagnosis. Because expert systems based on prior knowledge require summarizing a large amount of prior knowledge and are often difficult to establish for subtle or difficult-to-identify faults, recent research has primarily focused on data-driven fault diagnosis methods. Traditional statistical analysis methods can often only detect faults based on data distribution, making it difficult to distinguish fault types. Meanwhile, while traditional machine learning methods can differentiate fault types to some extent, they require significant effort in data feature engineering to distinguish the data features of different faults. Deep learning-based fault diagnosis methods, due to their adaptive learning of data features, have been widely applied in fault detection in recent years. However, single-input-channel models often only acquire data features within a fixed scale, making it difficult to balance the features of local and global faults.

[0004] Furthermore, traditional deep learning models for industrial process fault diagnosis use cross-entropy as the loss function for iterative updates of model weights. The cross-entropy function optimizes the model weights by approximating the correct label along the direction of minimum error, causing the model to focus only on correctly classified positive samples. While the cross-entropy loss function can achieve excellent classification accuracy in most cases, for industrial process fault data, some fault categories exhibit extremely small feature differences, making it difficult to determine the decision boundaries between different classes. Summary of the Invention

[0005] This invention addresses the problems existing in the prior art by providing a method for fault diagnosis of industrial processes based on a multi-scale convolutional neural network optimized by nearest neighbor component loss. The method includes the following steps:

[0006] S1. Data Preprocessing: Standardize the input data X and output data Y in the training and test sets to eliminate the dimensions of the data; transform the original data samples into feature map samples so that they can be used as input samples for the deep learning model.

[0007] S2. Construct a multi-scale convolutional neural network model to distinguish industrial process fault types: To address the limitations of traditional deep learning models in acquiring local and global information, a multi-scale convolutional neural network model is introduced as the main framework for fault diagnosis.

[0008] S3. Constructing a surrogate nearest neighbor component loss function: When optimizing the weights of a neural network model, the surrogate nearest neighbor component loss function is used instead of the cross-entropy function. By introducing negative samples, the model can more easily obtain the decision boundary between different faults.

[0009] S4. Train the model weights: Use the Adam optimizer to train the model weights until the model accuracy converges.

[0010] S5. Evaluate the model's fault diagnosis performance based on the difference between the model output and the actual fault categories: Input the test set data into the trained model for classification, calculate the recognition accuracy and average recognition accuracy of each type of fault based on the predicted and true values ​​of the input data, and complete the evaluation of the model's fault diagnosis performance.

[0011] The advantage of this method lies in its combination of multi-scale convolution strategy and surrogate nearest neighbor component loss optimization method on the basis of a convolutional neural network model. This allows the established nearest neighbor component loss optimized multi-scale convolutional neural network fault diagnosis model to more effectively target the hidden spatiotemporal features between data and the subtle differences between different faults. Therefore, when facing complex industrial production processes, this model has a high fault identification accuracy and higher reliability compared to traditional fault diagnosis models.

[0012] By adopting the above solution, the present invention has the following advantages compared with the prior art:

[0013] Compared with existing technologies, the proposed fault diagnosis model based on nearest neighbor component loss optimization of multi-scale convolutional neural networks has the following beneficial effects on fault diagnosis in industrial processes: by adopting deep learning modeling methods, it overcomes the shortcomings of statistical analysis models in distinguishing different fault features, and solves the problems of high feature engineering costs and time consumption in traditional machine learning models; and by introducing the surrogate nearest neighbor component loss function, the model training process is more inclined to distinguish faults at the decision boundary, thereby ensuring that the fault diagnosis model in industrial processes has high accuracy. Attached Figure Description

[0014] Figure 1 This outlines the process for establishing offline fault diagnosis models and applying them online.

[0015] Figure 2 The validation set loss changes during the model training process using the cross-entropy loss function and the surrogate nearest neighbor component loss function, respectively.

[0016] Figure 3 This is a heatmap of the confusion matrix of the model on the test set. Detailed Implementation

[0017] The present invention will now be described more clearly and completely. Obviously, the examples described are only a part of the examples of the present invention, and not all of the embodiments.

[0018] The technical solution adopted by this invention for fault diagnosis in wastewater treatment processes is as follows:

[0019] S1. Data preprocessing: Standardize the input data X and output data Y in the training set and test set according to formula (1), where the training set is used for model construction and the test set is used for model performance evaluation; and convert the original data samples into feature map samples according to formula (2).

[0020] S2. Construct a multi-scale convolutional neural network model: Construct feature extraction frameworks for convolutional neural networks of different scales using large-scale and small-scale convolutional kernels respectively, and then use a fully connected layer to link the feature maps of the last layer with the output nodes to achieve fault diagnosis in the process.

[0021] S3. Constructing the surrogate nearest neighbor component loss function: During model training, surrogate points are set for each type of fault, and a surrogate nearest neighbor component loss function is constructed. This reduces the distance between the sample and the positive surrogate and increases the distance between the sample and the negative surrogate during model training, thereby optimizing the decision boundary.

[0022] S4. Train the model weights: Use the Adam optimizer to train the model weights until the model loss converges, and obtain the trained model.

[0023] S5. Evaluate the model's fault diagnosis performance based on the difference between the model's output and the actual fault categories. Input the test set data into the model for classification, calculate the recognition accuracy and average recognition accuracy for each category based on the input data labels and the true labels, and complete the evaluation of the model's predictive ability.

[0024] In step S1, the data is standardized to standard data with a mean of 0 and a variance of 1. The standardization formula is:

[0025]

[0026] In the formula, X ＊ X represents the original data, while μ and σ represent the standardized data, with μ being the mean and σ being the variance, respectively.

[0027] The transformation formula for feature map samples is:

[0028]

[0029] In the formula, X n Let x represent the nth feature map sample after transformation. n This represents the nth sample point in the original data.

[0030] In step S2, the multi-scale convolutional neural network model is constructed as follows:

[0031] S21: Construct a large-scale convolutional neural network, using large-scale convolutional kernels in the network's convolutional layers to obtain data variation features within a large-scale spatiotemporal range. The first and third layers of the network are two-dimensional convolutional layers, and the second and fourth layers are max-pooling layers. Subsequently, feature mapping is performed using the Leaky-ReLU activation function and Dropout operation to obtain the data features extracted by the large-scale convolutional neural network. The expression for the Leaky-ReLU function is formula (3):

[0032]

[0033] In the formula, k is a very small value, representing the linear correction of the input negative signal.

[0034] The Dropout operation is defined as follows:

[0035]

[0036]

[0037]

[0038]

[0039] In the formula, N represents the number of samples in a batch, and x n and y n Let represent the input and output features of the nth sample in a batch, respectively. ε is a minimum value close to 0, used to prevent algorithm failure caused by zero-variance features in the feature map. γ and β represent two parameters used for scaling and offsetting the data, respectively.

[0040] S22: Construct a small-scale convolutional neural network, using small-scale convolutional kernels in the network's convolutional layers to obtain data variation features within a small spatiotemporal range. The first, second, fourth, and fifth layers of the network are two-dimensional convolutional layers, and the third and sixth layers are max pooling layers. Subsequently, the data features extracted by the small-scale convolutional neural network are mapped and obtained according to the Leaky-ReLU and Dropout operations in step S21.

[0041] S23: Let W be the feature extracted by a large-scale convolutional neural network. l The features extracted by the small-scale convolutional neural network are W. s The features are fused according to formula (8):

[0042]

[0043] In the formula, W n These are the features of the merged data.

[0044] S24: Fully connect the fused feature nodes to the output layer nodes to establish a multi-scale convolutional neural network model.

[0045] In step S3, the surrogate nearest neighbor component loss function is constructed according to formula (9) as follows:

[0046]

[0047] In the formula, X represents the matrix composed of input vectors, and P - Let represent the matrix composed of negative proxies, and s(a, b) represent the cosine similarity between two vectors. Meanwhile, LSE denotes the logarithmic exponential summation function, which is a smooth approximation of the maximization function max.

[0048] During gradient backpropagation, the gradient of the surrogate nearest neighbor component loss relative to s(x, p) is calculated using equation (10):

[0049]

[0050] In step S4, the Adam optimizer optimizes the weights as follows:

[0051] First, calculate the gradient at time t:

[0052]

[0053] Subsequently, the exponential moving average of the gradient is calculated, taking into account the previously accumulated gradient momentum:

[0054] m t =β1m t-1 +(1-β1)g t (12)

[0055] In the formula, m0 is initialized to 0, and β1 is the exponential decay rate, which is used to control the weight distribution of momentum and the current gradient, and is usually taken as 0.9.

[0056] Then, the exponential moving average of the squared gradient is calculated:

[0057]

[0058] In the formula, v0 is initialized to 0, and β2 is the exponential decay rate, which controls the effect of the previous squared gradient, and is usually taken as 0.999.

[0059] For m t and v t Perform deviation correction:

[0060]

[0061]

[0062] Finally, the weight update formula (16) is obtained:

[0063]

[0064] In the formula, α is the learning rate, which defaults to 0.001, and ε is a very small number used to prevent division by zero.

[0065] In step S5, the difference between the true label and the predicted label is statistically analyzed, and the model's fault diagnosis capability is evaluated.

[0066] The test set data is fed into the model for prediction. The classification accuracy for each category and the overall average classification accuracy are calculated based on the predicted and true labels of the input data. The accuracy is calculated according to formula (17).

[0067]

[0068] In the formula, TP and TN represent the number of positive and negative examples that are correctly classified, respectively, and P and N represent the total number of positive and negative examples, respectively.

[0069] Example 1:

[0070] Taking 10 common faults occurring in a wastewater treatment process and 11 types of data, including normal operation data, as an example, the wastewater treatment process data used for fault diagnosis includes 15 monitoring variables: influent flow rate, influent ammonia concentration, nitrate concentration in the second reaction tank, dissolved oxygen content in the third reaction tank, dissolved oxygen content in the fourth reaction tank, dissolved oxygen content in the fifth reaction tank, total suspended solids concentration in the fourth reaction tank, total suspended solids concentration in the fifth reaction tank, total suspended solids concentration in the effluent, total suspended solids concentration in the residual sludge, total suspended solids concentration in the external circulation, oxygen conversion coefficient in the fifth reaction tank, internal circulation flow rate, effluent ammonia concentration, and effluent nitrate concentration. Specific information for each category of data is listed in Table 1.

[0071] Table 1. Data Types and Information for Fault and Normal Categories

[0072]

[0073] Combination Figure 1 The present invention will be further described in detail below:

[0074] Step 1: Convert both the training set data and the test set data into feature map type data to facilitate input into the model for feature extraction.

[0075] Step 2: Referring to the network structures of VGG-net and LeNet, we designed the network structures and parameters of large-scale convolutional neural networks and small-scale convolutional neural networks respectively. After multiple experiments, the optimal structures were determined and are listed in Table 2.

[0076] Table 2. Convolutional Neural Network Model Structure and Parameters

[0077]

[0078] Step 3: Construct the surrogate nearest neighbor component loss function. Since this case contains 11 classes of data, the number of surrogates is set to 11.

[0079] Step 4: Using the network model structure from Step 2 and the loss function from Step 3, train the model weights using the Adam optimizer on the training set data until the accuracy converges.

[0080] Step 5: Save the trained model and weight parameters, and use the test set data for classification to verify the model's performance. Compare the fault diagnosis accuracy of the multi-scale convolutional neural network optimized with surrogate nearest neighbor loss, large-scale and small-scale convolutional neural networks using nearest neighbor loss and cross-entropy loss respectively, the multi-scale convolutional neural network using cross-entropy loss, linear discriminant analysis, and support vector machines.

[0081] Table 3 compares the fault diagnosis performance of eight models on the test set for wastewater treatment processes. The results show that the multi-scale convolutional neural network with surrogate nearest neighbor component loss optimization achieves the best results in average accuracy and for most categories. Especially for category 0 (normal samples), where traditional methods have a 100% false positive rate, the proposed method achieves an accuracy of nearly 64%. For category 9, the method also shows a significant improvement in accuracy compared to other methods. Compared to the classical linear fault diagnosis method, linear discriminant analysis, the multi-scale convolutional neural network with surrogate nearest neighbor component loss optimization improves the average accuracy of fault diagnosis by nearly 25 percentage points. Simultaneously, compared to the classical nonlinear fault diagnosis method, support vector machine, it improves the average accuracy of fault diagnosis by nearly 21 percentage points.

[0082] Figure 2 This paper demonstrates the changes in validation set loss during training for models using the traditional cross-entropy loss function and those using surrogate nearest neighbor component loss. It shows that the cross-entropy function leads to poor stability in the later stages of training and a rapid tendency towards overfitting, while the surrogate nearest neighbor component loss used in this approach results in a more stable training process. The multi-scale convolutional neural network model optimized using surrogate nearest neighbor component loss achieves excellent fault diagnosis results in wastewater treatment processes. Figure 3 The results are shown in Figure 3 and Table 3. According to these figures, the categories with poor identification accuracy are mainly concentrated between normal data (Category 0), nitrate concentration actuator failure (Category 5), rainy weather (Category 6), stormy weather (Category 7), and dissolved oxygen sensor drift failure (Category 8). Among these, rainy and stormy weather failures exhibit strong random fluctuations, making it difficult to capture their fault characteristics. Dissolved oxygen sensor drift failures have small fault offsets in the early stages, making them difficult to detect. Both nitrate concentration actuator failures and dissolved oxygen sensor offset failures are step-like faults, and their data characteristics are identical to normal data for a period after the failure occurs, making them difficult to identify. In conclusion, although the accuracy of the proposed method still has room for improvement, it is reasonable.

[0083] Considering the nonlinearity, time-varying nature, and difficulty in distinguishing fault characteristics in wastewater treatment data, traditional fault diagnosis models struggle to achieve satisfactory identification accuracy. This invention employs a multi-scale convolutional neural network to extract nonlinear spatiotemporal transformation features from multiple dimensions. Furthermore, by optimizing the training process using surrogate nearest neighbor component loss, the multi-scale convolutional neural network model optimized by surrogate nearest neighbor component loss is better adapted to fault diagnosis in actual industrial processes.

[0084] The basic principles, main features, and advantages of this invention have been described above. The above description is merely a preferred embodiment of the invention, and the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope demonstrated by this invention should be included within the scope of protection of this invention. Therefore, the scope of protection of this invention should be defined by the appended claims and their equivalents.

[0085] Table 3 Fault diagnosis accuracy of different models

[0086]

Claims

1. A method for fault diagnosis in industrial processes based on multi-scale convolutional neural networks optimized by nearest neighbor component loss, characterized in that, The modeling method includes the following steps: S1. Data Preprocessing: The data comes from industrial process monitoring data. The input data X and output data Y in the training set and test set data are standardized to eliminate the dimensions of the data. The original data samples are transformed into feature map samples to serve as input samples for the deep learning model. S2. Construct a multi-scale convolutional neural network model to distinguish industrial process fault types: In order to solve the limitations of traditional deep learning models in acquiring local and global information, a multi-scale convolutional neural network model is introduced as the main framework for fault diagnosis. S3. Constructing the surrogate nearest neighbor component loss function: When optimizing the weights of the neural network model, the surrogate nearest neighbor component loss function is used instead of the cross-entropy function. By introducing negative samples, the model can more easily obtain the decision boundary between different faults. The surrogate nearest neighbor component loss function is expressed as follows: X represents the matrix composed of the output vectors of the multi-scale convolutional neural network, P - p represents the matrix composed of negative proxies. + As a positive agent, p - For negative surrogate, s(a, b) represents the cosine similarity between two vectors; LSE represents the logarithmic exponential summation function, which is a smooth approximation of the maximization function max. S4. Train the model weights: Use the Adam optimizer to train the model weights until the model accuracy converges.

2. The industrial process fault diagnosis method based on nearest neighbor component loss optimization of multi-scale convolutional neural networks according to claim 1, characterized in that, The specific implementation process is as follows: In step S1, the data is standardized to standard data with a mean of 0 and a variance of 1. The standardization formula is: In the formula, X ＊ X represents the original data, μ represents the standardized data, and σ represents the mean and variance, respectively. The transformation formula for feature map samples is: In the formula, X n Let x represent the nth feature map sample after transformation. n This represents the nth sample point in the original data.

3. The industrial process fault diagnosis method based on nearest neighbor component loss optimization of multi-scale convolutional neural networks according to claim 1, characterized in that, The specific process of step S2 is as follows: S21: Construct a large-scale convolutional neural network, using large-scale convolutional kernels in the network's convolutional layers to obtain data change features within a large-scale spatiotemporal range; the first and third layers of the network are two-dimensional convolutional layers, and the second and fourth layers are max pooling layers. Subsequently, feature mapping is performed using the Leaky-Relu activation function and Dropout operation to obtain the data features extracted by the large-scale convolutional neural network. The expression of the Leaky-Relu function is formula (3): In the formula, k is a very small value, representing the linear correction of the input negative signal; The Dropout operation is defined as follows: In the formula, N represents the number of samples in a batch, and x n and y n ε and β represent the input and output features of the nth sample in a batch, respectively. ε is a minimum value close to 0, used to prevent algorithm failure caused by zero variance features in the feature map. γ and β represent two parameters used for scaling and offsetting the data, respectively. S22: Construct a small-scale convolutional neural network and use small-scale convolutional kernels in the convolutional layers of the network to obtain data change features in a small-scale spatiotemporal range; the first, second, fourth, and fifth layers of the network are two-dimensional convolutional layers, and the third and sixth layers are max pooling layers. Then, the data features extracted by the small-scale convolutional neural network are obtained by mapping the Leaky-ReLU and Dropout operations in step S21. S23: Let W be the feature extracted by a large-scale convolutional neural network. l The features extracted by the small-scale convolutional neural network are W. s The features are fused according to formula (8): In the formula, W n Features of the merged data; S24: Fully connect the fused feature nodes to the output layer nodes to establish a multi-scale convolutional neural network model.

4. The industrial process fault diagnosis method based on nearest neighbor component loss optimization of multi-scale convolutional neural networks according to claim 1, characterized in that, The specific process of step S3 is as follows: During gradient backpropagation, the gradient of the surrogate nearest neighbor component loss relative to s(x, p) is calculated using equation (9):

5. The industrial process fault diagnosis method based on nearest neighbor component loss optimization of multi-scale convolutional neural networks according to claim 1, characterized in that, The specific process of step S4 is as follows: First, calculate the gradient at time t: Subsequently, the exponential moving average of the gradient is calculated, taking into account the previously accumulated gradient momentum: m t =β1m t-1 +(1-β1)g t (11) In the formula, m0 is initialized to 0, and β1 is the exponential decay rate, which is used to control the weight distribution of momentum and current gradient, and is usually taken as 0.9; Then, the exponential moving average of the squared gradient is calculated: In the formula, v0 is initialized to 0, and β2 is the exponential decay rate, which controls the effect of the previous squared gradient, and is usually taken as 0.

999. For m t and v t Perform deviation correction: Finally, update the weights: In the formula, α is the learning rate, which defaults to 0.001, and ε is a very small number used to prevent division by zero.

6. The industrial process fault diagnosis method based on nearest neighbor component loss optimized multi-scale convolutional neural network according to any one of claims 1 to 5, characterized in that, The modeling method also includes a model prediction capability evaluation process, which involves: inputting test set data into the trained model for prediction, calculating the average accuracy and the accuracy of each category based on the predicted labels and true labels of the input data, and completing the evaluation of the model's fault diagnosis performance.

Images