Industrial process fault diagnosis method, device and system, and storage medium

Abstract

The invention discloses an industrial process fault diagnosis method, device and system and a storage medium, pseudo labels are distributed and updated for all target domain samples based on a self-entropy and neighborhood structure pseudo label propagation mode, and the availability of target data is improved; a center loss is added to constrain an inherent structure of target domain data, it is ensured that the model can capture the inter-class relation, the relative distance between samples can be kept, and the model performance is further optimized; knowledge of a source domain is used for guiding the learning process of a target domain model, more stable and robust model learning is achieved, and the over-fitting risk is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of industrial process fault diagnosis, and specifically relates to an industrial process fault diagnosis method and device, a system, and a storage medium. Background Art

[0002] Modern industrial processes are developing rapidly, showing the characteristics of large-scale, complex and nonlinear. Equipment aging, improper operation, environmental factors, changes in production load, etc. in industrial processes may lead to equipment performance degradation, abnormal operation or complete failure, thus affecting production efficiency and safety. Through timely fault diagnosis, potential problems can be discovered, thereby improving the reliability of equipment, which is extremely important for improving process operation safety, reducing economic losses, optimizing resource allocation, and improving overall production efficiency and economic benefits.

[0003] Although data-driven methods (such as deep learning and support vector machines) have achieved remarkable results in fault diagnosis, they usually rely on a large amount of labeled data for training and require that the test data have the same data distribution as the training data. However, the acquisition and labeling of fault data is always difficult in the industrial field, and the fluctuation of working conditions will also cause changes in data distribution. In order to adapt to the distribution differences between different data sets, transfer learning has been proposed and applied to fault diagnosis of industrial processes. However, traditional transfer learning methods usually require access to both the source domain and target domain datasets during model training. However, in practical applications, due to data privacy protection or limited data transmission, source domain samples may not be accessible during model training, and traditional transfer learning cannot meet such scenarios. Therefore, passive domain transfer learning, which does not access source domain samples during target domain model training and only uses source domain models, has become an effective solution. It can achieve effective fault diagnosis and prediction in the absence of source domain data, and improve the generalization ability and adaptability of the model.

[0004] Existing passive domain transfer learning methods are all based on Source Hypothesis Transfer (SHOT), that is, if the source domain-like feature representation of the target domain samples can be learned, then the classification output of the source domain classifier for the target domain data should be similar to the classification output of the source domain data. However, due to the existence of domain bias, the pseudo-labels assigned to the target domain based on the source domain model may not always be accurate, resulting in a decrease in the performance of the target domain model. How to improve the reliability of pseudo-labels and avoid incorrect pseudo-labels from degrading the model is an urgent problem to be solved in passive domain transfer learning. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an industrial process fault diagnosis method and device, system, and storage medium, which can achieve effective fault diagnosis and prediction in the absence of source domain data and improve the generalization ability and adaptability of the model.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] An industrial process fault diagnosis method, comprising:

[0008] Step 1: Perform supervised training based on the source domain fault data set to obtain a source domain model;

[0009] Step 2: Initialize the target domain model according to the source domain model parameters;

[0010] Step 3: Input the unlabeled target domain samples into the source domain model to obtain the classification and classification uncertainty;

[0011] Step 4: Select the category prototype of the target domain sample based on the classification result and uncertainty;

[0012] Step 5: Input the target domain samples into the feature extractor of the target domain model to obtain sample features, and assign pseudo labels to all target domain samples based on the distance between features and the category prototype;

[0013] Step 6: Classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results.

[0014] Step 7: Calculate the category loss based on the features and predicted labels

[0015] Step 8: Input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss

[0016] Step 9: Classify loss using cross entropy Class loss and knowledge distillation loss Jointly train the target domain model;

[0017] Step 10: Repeat the above steps 5 to 9 to update the target domain model and obtain a trained fault diagnosis model;

[0018] Step 11: Use the trained fault diagnosis model to perform fault diagnosis on the operating condition data of the target domain.

[0019] As a preferred method, in step 1, the AO process commonly used in sewage treatment is selected as the source domain, and the A2O process is selected as the target domain. The fault data set of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. Indicates the fault type. They respectively represent normal operating conditions, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

[0020] Preferably, in step 10, pseudo labels are assigned to target domain samples according to different feature differences so that the pseudo labels are closer to real labels. When the loss function value of the model training meets the setting or reaches the maximum number of iterations, the training is stopped to obtain a trained fault diagnosis model of the target domain A2O process.

[0021] The present invention also provides an industrial process fault diagnosis device, comprising:

[0022] The first processing module is used to perform supervised training based on the source domain fault data set to obtain a source domain model;

[0023] A second processing module is used to initialize the target domain model according to the source domain model parameters;

[0024] The third processing module is used to input the unlabeled target domain samples into the source domain model to obtain the classification and classification uncertainty;

[0025] The fourth processing module is used to select the category prototype of the target domain sample according to the classification result and uncertainty;

[0026] The fifth processing module is used to input the target domain samples into the feature extractor of the target domain model to obtain sample features, and to label all target domain samples with pseudo labels according to the distance between the features and the category prototype;

[0027] The sixth processing module is used to classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results.

[0028] The seventh processing module is used to calculate the category loss based on the features and predicted labels.

[0029] The eighth processing module is used to input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss

[0030] The ninth processing module is used to classify the loss with cross entropy Class loss and knowledge distillation loss Jointly train the target domain model;

[0031] The tenth processing module is used to update the target domain model to obtain a trained fault diagnosis model;

[0032] The eleventh processing module is used to perform fault diagnosis on the operating condition data of the target domain using the trained fault diagnosis model.

[0033] As a preferred method, the AO process commonly used in sewage treatment is selected as the source domain, and the A2O process is selected as the target domain. The fault dataset of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. Indicates the fault type. They respectively represent normal operating conditions, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

[0034] Preferably, the tenth processing module is used to assign pseudo labels to target domain samples according to different feature differences, so that the pseudo labels are closer to the real labels, until the loss function value of the model training meets the setting or reaches the maximum number of iterations, then the training is stopped to obtain a trained fault diagnosis model of the target domain A2O process.

[0035] An embodiment of the present invention further provides an industrial process fault diagnosis system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes an industrial process fault diagnosis method when executed by the processor.

[0036] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the industrial process fault diagnosis method when running.

[0037] The present invention allocates and updates pseudo labels for all target domain samples based on the pseudo label propagation method of self-entropy and neighborhood structure, thereby improving the availability of target data; adding the center loss to constrain the inherent structure of the target domain data not only ensures that the model can capture the relationship between classes, but also maintains the relative distance between samples, further optimizing the model performance; using the knowledge of the source domain to guide the learning process of the target domain model, achieving more stable and robust model learning and reducing the risk of overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 is a flow chart of an industrial process fault diagnosis method according to an embodiment of the present invention;

[0040] Figure 2 It is a flow chart of the sewage treatment process based on AO process; among them, is used as source domain data;

[0041] Figure 3 is a flow chart of a wastewater treatment process based on the A2O process; wherein, is used as target domain data;

[0042] Figure 4 is a schematic diagram of the input variables selected in this example;

[0043] Figure 5 It is a fault diagnosis network structure diagram common to both source and target domains;

[0044] Figure 6 This is the experimental result diagram. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Embodiment 1:

[0048] like Figure 1 As shown, the embodiment of the present invention provides an industrial process fault diagnosis method, which is based on fault diagnosis of passive domain transfer learning, can solve the domain shift problem under privacy protection, effectively transfer the knowledge of the source domain model to the target domain, and realize fault diagnosis on the target domain. The specific implementation steps are as follows:

[0049] Step 1: Perform supervised training on a source domain fault dataset with sufficient sample size and labels to obtain a well-trained source domain model;

[0050] In step 1, the sewage treatment process is used as the background, and the AO process commonly used in the sewage treatment process is selected as the source domain, such as Figure 2 As shown, the A2O process is used as the target domain, such as Figure 3As shown in the figure, three fault conditions were designed, including abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow, including four types of normal conditions. The specific description is shown in Table 1. 30 process variables were selected as input variables, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. These variables can be measured in real time during the actual sewage treatment process and can reflect the operating status of the sewage treatment process. Specifically, Figure 4 shown.

[0051] Table 1

[0052]

[0053] Assume that the fault dataset of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. Figure 4 shown. Indicates the fault category. In this example, according to Table 1, They represent normal working condition, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow. The source domain AO process fault diagnosis model is in represents the feature extractor of the source domain, Represents the classifier of the source domain, such as Figure 5 In this example, The number of neurons is set to {30, 20, 10} respectively, The number of neurons is set to {10, 8, 4} respectively, and the source domain model is supervised by minimizing the cross entropy loss function. The specific formula is:

[0054]

[0055] in, Represents the label y s One-hot encoding of Then q c =1, indicating correct classification, if Then q c =0 indicates wrong classification. σ c Represents the c-th element of the softmax output of a C-dimensional vector:

[0056]

[0057] Among them, α c The cth element of the vector α output by the classifier is represented, and C represents the total number of fault categories. Thus, the fault diagnosis model F of the well-trained source domain AO process is obtained:s .

[0058] Step 2: Use the fault diagnosis model F of the source domain AO s Parameter initialization of the fault diagnosis model of the target domain A2O, including feature extractor and classifier;

[0059] In step 2, it is assumed that the target domain model and the source domain model have the same model structure as Figure 5 As shown, the fault diagnosis model of the target domain is in represents the feature extractor of the target domain, Represents the classifier of the target domain. The target domain model and the source domain model should have the same number of neurons. Using the source domain model to initialize the target domain model means assigning all the parameters of the source domain model to the target domain model, that is, φ t =φ s ,φ represents its network parameters.

[0060] Step 3: Input the unlabeled target domain A2O process samples into the source domain model, obtain the classification results, and calculate the classification uncertainty;

[0061] In step 3, the parameters of the target domain model are first fixed to classify the target domain samples, and the certainty of the model output is measured by calculating the self-entropy of the classification output. The self-entropy is calculated as follows:

[0062]

[0063] The smaller the value of self-entropy is, the smaller the uncertainty of classification is, and the model is more confident in the classification result.

[0064] Step 4: Select the category prototypes of the four working conditions in the target domain samples: normal working condition, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow according to the classification results and uncertainty;

[0065] In step 4, by sorting the self-entropy of the classification results, samples with smaller self-entropy are selected as category prototypes, which can be expressed as:

[0066] M c ={x i |x i ∈sort(H c (x t )),i≤N}

[0067] Among them, H c (x t ) represents the sample x belonging to category c tThe self-entropy of, sort(·) indicates the smooth arrangement from small to large, N indicates the number of prototypes of each type, which needs to be set manually, usually set to a value related to the sample size and category size of the target domain:

[0068]

[0069] Therefore, the pseudo labels for these category prototypes can be set as:

[0070]

[0071] in They respectively indicate that the samples belong to four operating conditions: normal operating condition, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

[0072] Step 5: Input the samples of the target domain A2O process into the feature extractor of the target domain model to obtain the sample features. Pseudo-label all target domain samples according to the distance between the features and the category prototype to indicate different fault types.

[0073] In step 5, firstly, all target domain samples are input into the feature extractor Calculate the Euclidean distance between the features of the class prototype and other features as the distance between samples:

[0074]

[0075] Among them, N z Represents the dimension of the feature. All target domain samples are assigned pseudo labels through the Euclidean distance and nearest neighbor algorithm based on sample features.

[0076] Step 6: Classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results

[0077] In step 6, the cross entropy loss between the pseudo label and the classification result The calculation formula is:

[0078]

[0079] Step 7: Calculate the class loss based on the features and predicted labels

[0080] In step 7, the category loss is reduced to maintain the internal structure of the target domain samples. Specifically, samples with similar categories in the feature space are grouped together while ensuring that samples with different categories are separated from each other. Therefore, the center loss is introduced to minimize the intra-class distance by concentrating the features of samples of the same type on a class center while keeping the distance between class centers large. Therefore, the category loss The calculation formula is:

[0081]

[0082] in, Indicates category The category center is obtained by calculating the average value of the sample features of this category: The category center is continuously updated during the training process to dynamically reflect the distribution of the current category in the feature space.

[0083] Step 8: Input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss

[0084] In step 8, the purpose of designing and reducing the knowledge distillation loss is to prevent the target domain model from self-biasing during learning and training. Therefore, the source domain model is introduced as a constraint, and its calculation formula is:

[0085]

[0086] Step 9: Classify the loss using the cross entropy above Class loss and knowledge distillation loss Jointly train the target domain model;

[0087] In step 9, the total loss function is:

[0088]

[0089] Among them, λ 1 and λ 2 is a pre-set trade-off parameter used to balance the various losses. Using neural network training methods such as gradient descent to minimize The target domain model is trained and updated. The general grid search method is used to set λ 1 =0.2,λ 2 =0.3.

[0090] Step 10: Repeat steps 5 to 9 above to update the target domain model and realize transfer learning of source domain data.

[0091] In step 10, when steps 5 to 9 are repeated, pseudo labels are assigned to samples in the target domain according to different feature differences, so that the pseudo labels are closer to the real labels, until the loss function value of the model training meets the setting or reaches the maximum number of iterations, and then the training is stopped. The trained fault diagnosis model F of the target domain A2O process is obtained. t* .

[0092] Step 11: Use the trained A2O process fault diagnosis model F t* Perform fault diagnosis on the target domain’s operating condition data:

[0093] In step 11, the process variables in the A2O sewage treatment process are first collected online. The collected process variables should be consistent with the 30 process variables in step 1, and are recorded as Then the process data Input the trained fault diagnosis model F t* , and get the fault diagnosis result:

[0094]

[0095] Then, the classification accuracy of the test data is used as an indicator to evaluate the effectiveness of the model, which is defined as:

[0096]

[0097] in, represents the test data collected in the target domain A2O sewage treatment process, y(x) represents the real fault category label, Represents the fault category label predicted by the classifier. The experimental result is Accuracy = 0.8813, indicating that the classification accuracy of the target domain fault diagnosis model constructed by the method of the present invention using only the source domain fault diagnosis model and the target domain data without the direct participation of the source domain data reaches 0.8813. The visualization results of the fault category classification are shown in Figure 6 The effectiveness of the present invention is illustrated.

[0098] Embodiment 2:

[0099] The embodiment of the present invention further provides an industrial process fault diagnosis device, comprising:

[0100] The first processing module is used to perform supervised training based on the source domain fault data set to obtain a source domain model;

[0101] A second processing module is used to initialize the target domain model according to the source domain model parameters;

[0102] The third processing module is used to input the unlabeled target domain samples into the source domain model to obtain the classification and classification uncertainty;

[0103] The fourth processing module is used to select the category prototype of the target domain sample according to the classification result and uncertainty;

[0104] The fifth processing module is used to input the target domain samples into the feature extractor of the target domain model to obtain sample features, and to label all target domain samples with pseudo labels according to the distance between the features and the category prototype;

[0105] The sixth processing module is used to classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results.

[0106] The seventh processing module is used to calculate the category loss based on the features and predicted labels.

[0107] The eighth processing module is used to input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss

[0108] The ninth processing module is used to classify the loss with cross entropy Class loss and knowledge distillation loss Jointly train the target domain model;

[0109] The tenth processing module is used to update the target domain model to obtain a trained fault diagnosis model;

[0110] The eleventh processing module is used to perform fault diagnosis on the operating condition data of the target domain using the trained fault diagnosis model.

[0111] As an implementation method of the present invention, the AO process commonly used in sewage treatment is selected as the source domain, and the A2O process is selected as the target domain. The fault data set of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. Indicates the fault type. They respectively represent normal operating conditions, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

[0112] As an implementation mode of an embodiment of the present invention, the tenth processing module is used to assign pseudo labels to target domain samples according to different feature differences, so that the pseudo labels are closer to the real labels, until the loss function value of the model training meets the setting or reaches the maximum number of iterations, the training is stopped to obtain a trained fault diagnosis model of the target domain A2O process.

[0113] Embodiment 3:

[0114] An embodiment of the present invention further provides an industrial process fault diagnosis system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes an industrial process fault diagnosis method when executed by the processor.

[0115] Embodiment 4:

[0116] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the industrial process fault diagnosis method when running.

[0117] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An industrial process fault diagnosis method, characterized in that: include: Step 1: Perform supervised training based on the source domain fault data set to obtain a source domain model; Step 2: Initialize the target domain model according to the source domain model parameters; Step 3: Input the unlabeled target domain samples into the source domain model to obtain the classification and classification uncertainty; Step 4: Select the category prototype of the target domain sample based on the classification result and uncertainty; Step 5: Input the target domain samples into the feature extractor of the target domain model to obtain sample features, and assign pseudo labels to all target domain samples based on the distance between features and the category prototype; Step 6: Classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results. Step 7: Calculate the category loss based on the features and predicted labels Step 8: Input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss Step 9: Classify loss using cross entropy Class loss and knowledge distillation loss Jointly train the target domain model; Step 10: Repeat the above steps 5 to 9 to update the target domain model and obtain a trained fault diagnosis model; Step 11: Use the trained fault diagnosis model to perform fault diagnosis on the operating condition data of the target domain.

2. The industrial process fault diagnosis method according to claim 1, characterized in that: In step 1, the AO process commonly used in sewage treatment is selected as the source domain, and the A2O process is selected as the target domain. The fault dataset of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. Indicates the fault type. They respectively represent normal operating conditions, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

3. The industrial process fault diagnosis method according to claim 2, characterized in that: In step 10, pseudo labels are assigned to target domain samples according to different feature differences, so that the pseudo labels are closer to the real labels. When the loss function value of the model training meets the setting or reaches the maximum number of iterations, the training is stopped to obtain a trained fault diagnosis model of the target domain A2O process.

4. An industrial process fault diagnosis device, characterized in that: include: The first processing module is used to perform supervised training based on the source domain fault data set to obtain a source domain model; A second processing module is used to initialize the target domain model according to the source domain model parameters; The third processing module is used to input the unlabeled target domain samples into the source domain model to obtain the classification and classification uncertainty; The fourth processing module is used to select the category prototype of the target domain sample according to the classification result and uncertainty; The fifth processing module is used to input the target domain samples into the feature extractor of the target domain model to obtain sample features, and to label all target domain samples with pseudo labels according to the distance between the features and the category prototype; The sixth processing module is used to classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results. The seventh processing module is used to calculate the category loss based on the features and predicted labels. The eighth processing module is used to input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss The ninth processing module is used to classify the loss with cross entropy Class loss and knowledge distillation loss Jointly train the target domain model; The tenth processing module is used to update the target domain model to obtain a trained fault diagnosis model; The eleventh processing module is used to perform fault diagnosis on the operating condition data of the target domain using the trained fault diagnosis model.

5. The industrial process fault diagnosis device according to claim 4, characterized in that: The AO process commonly used in sewage treatment is selected as the source domain, and the A2O process is selected as the target domain. The fault dataset of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. Indicates the fault type. They respectively represent normal operating conditions, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

6. The industrial process fault diagnosis device according to claim 5, characterized in that: The tenth processing module is used to assign pseudo labels to target domain samples according to different feature differences, so that the pseudo labels are closer to the real labels. When the loss function value of the model training meets the setting or reaches the maximum number of iterations, the training is stopped to obtain the trained fault diagnosis model of the target domain A2O process.

7. An industrial process fault diagnosis system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the industrial process fault diagnosis method according to any one of claims 1 to 3 is executed.

8. A storage medium, characterized in that: The storage medium stores a computer program, which executes the industrial process fault diagnosis method according to any one of claims 1 to 3 when running.

Images