Industrial process fault diagnosis method and device, system, storage medium

Abstract

The application discloses an industrial process fault diagnosis method and device, system and storage medium. Pseudo labels are assigned and updated for all target domain samples based on self-entropy and neighborhood structure pseudo label propagation, improving the availability of target data. A center loss constraint is added to the inherent structure of target domain data, which not only ensures that the model can capture the inter-class relationship, but also maintains the relative distance between samples, further optimizing the model performance. The learning process of the target domain model is guided by the knowledge of the source domain, realizing more stable and robust model learning and reducing the risk of overfitting.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial process fault diagnosis, and particularly relates to an industrial process fault diagnosis method and device, system, and storage medium. BACKGROUND

[0002] Modern industrial processes are developing rapidly and exhibit large-scale, complexity, and nonlinearity. In industrial processes, equipment aging, improper operation, environmental factors, and production load changes can all lead to equipment performance degradation, abnormal operation, or complete failure, thereby affecting production efficiency and safety. Through timely fault diagnosis, potential problems can be found, thereby improving equipment reliability, which is extremely important for improving process operation safety, reducing economic losses, optimizing resource allocation, and improving overall production efficiency and economic benefits.

[0003] Although data-driven methods such as deep learning and support vector machines have achieved remarkable results in fault diagnosis, they usually rely on a large amount of labeled data for training and require test data to have the same data distribution as training data. However, the acquisition and labeling of fault data are always difficult in the industrial field, and fluctuations in working conditions can also cause changes in data distribution. In order to adapt to the distribution differences between different data sets, transfer learning is proposed and applied to fault diagnosis of industrial processes. However, traditional transfer learning methods usually need to access both source domain and target domain data sets during model training. However, in practical applications, due to data privacy protection or limited data transmission, source domain samples may not be accessible during model training, and traditional transfer learning cannot meet such scenarios. Therefore, source-free transfer learning of target domain model training without accessing source domain samples becomes an effective solution, which can still achieve effective fault diagnosis and prediction in the absence of source domain data, and improve the generalization ability and adaptability of the model.

[0004] Existing source-free transfer learning methods are all based on source hypothesis transfer (SHOT), that is, if the class source domain feature representation of the target domain sample can be learned, the classification output of the source domain classifier for the target domain data should be similar to the classification output of the source domain data. However, due to the existence of domain bias, the pseudo-labels assigned to the target domain by the source domain model are not always accurate, leading to a decline in the performance of the target domain model. How to improve the reliability of pseudo-labels and avoid the deterioration of the model by false pseudo-labels is a problem to be solved in source-free transfer learning. SUMMARY

[0005] The technical problems to be solved by the present application are to provide an industrial process fault diagnosis method and device, system and storage medium, which can still achieve effective fault diagnosis and prediction and improve the generalization ability and adaptability of the model in the absence of source domain data.

[0006] To achieve the above object, the present application adopts the following technical scheme:

[0007] An industrial process fault diagnosis method comprises the following steps:

[0008] Step 1, supervised training is performed according to a source domain fault data set to obtain a source domain model;

[0009] Step 2, a target domain model is initialized according to the source domain model parameters;

[0010] Step 3, a no-label target domain sample is input into the source domain model to obtain classification and classification uncertainty;

[0011] Step 4, a class prototype of the target domain sample is selected according to the classification result and the uncertainty;

[0012] Step 5, the target domain sample is input into a feature extractor of the target domain model to obtain sample features, and pseudo labels are attached to all target domain samples according to the distance between the features and the class prototype;

[0013] Step 6, the target domain model is used to classify the target domain samples with pseudo labels, and a cross-entropy classification loss between the pseudo labels and the classification results is calculated

[0014] Step 7, a class loss is calculated based on the features and the predicted labels

[0015] Step 8, the target domain sample features are input into a source domain classifier and a target domain classifier respectively, and a knowledge distillation loss is calculated

[0016] Step 9, the target domain model is trained together with the cross-entropy classification loss the class loss and the knowledge distillation loss

[0017] Step 10, the above steps 5-9 are repeated to update the target domain model to obtain a trained fault diagnosis model;

[0018] Step 11, the trained fault diagnosis model is used to perform fault diagnosis on the running condition data of the target domain.

[0019] As a preferred, in step 1, the AO process commonly used in the sewage treatment process is selected as the source domain, the A2O process is selected as the target domain, and the fault data set of the source domain AO is​ represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. represents a fault category, respectively represent normal working condition, biological pool abnormality, secondary sedimentation tank abnormality, pipeline flow abnormality.

[0020] Preferably, in step 10, pseudo labels are assigned to target domain samples according to different feature differences, so that the pseudo labels are increasingly close to the true labels, and when the loss function value of model training meets the setting or reaches the maximum iteration number, the training is stopped, and a trained fault diagnosis model of the target domain A2O process is obtained.

[0021] The application also provides an industrial process fault diagnosis device, comprising:

[0022] The first processing module is used for supervised training according to the source domain fault data set to obtain a source domain model;

[0023] The second processing module is used for initializing a target domain model according to the source domain model parameters;

[0024] The third processing module is used for inputting the unlabeled target domain samples into the source domain model to obtain classification and classification uncertainty;

[0025] The fourth processing module is used for selecting a category prototype of the target domain samples according to the classification results and the uncertainty;

[0026] The fifth processing module is used for inputting the target domain samples into a feature extractor of the target domain model to obtain sample features, and assigning pseudo labels to all target domain samples according to the distances between the features and the category prototype;

[0027] The sixth processing module is used for classifying the target domain samples with pseudo labels based on the target domain model, and calculating a cross-entropy classification loss between the pseudo labels and the classification results

[0028] The seventh processing module is used for calculating a category loss based on the features and the predicted labels

[0029] The eighth processing module is used for inputting the target domain sample features into a source domain classifier and a target domain classifier respectively, and calculating a knowledge distillation loss

[0030] The ninth processing module is used for training the target domain model based on the cross-entropy classification loss the category loss and the knowledge distillation loss

[0031] ​The tenth processing module is configured to update the target domain model to obtain a trained fault diagnosis model.

[0032] The eleventh processing module is configured to perform fault diagnosis on the operation condition data of the target domain by using the trained fault diagnosis model.

[0033] As a preferred example, the AO process commonly used in the sewage treatment process is selected as the source domain, the A2O process is selected as the target domain, the fault data set of the source domain AO is represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc. represents a fault category, respectively represent normal operation condition, abnormal biological pool, abnormal secondary sedimentation tank and abnormal pipeline flow.

[0034] As a preferred example, the tenth processing module is configured to assign pseudo labels to the target domain samples according to different feature differences, so that the pseudo labels are increasingly close to the real labels, and the training is stopped until the loss function value of the model training meets the setting or the maximum iteration number is reached, to obtain the trained fault diagnosis model of the target domain A2O process.

[0035] The embodiment of the present application also provides an industrial process fault diagnosis system, comprising a memory and a processor, the memory stores a computer program run by the processor, and the computer program executes the industrial process fault diagnosis method when run by the processor.

[0036] The embodiment of the present application also provides a storage medium, which stores a computer program, and the computer program executes the industrial process fault diagnosis method when run.

[0037] The present application assigns and updates the pseudo labels for all target domain samples based on the self-entropy and neighborhood structure pseudo label propagation method, improves the availability of the target data, adds the center loss constraint to the inherent structure of the target domain data, not only ensures that the model can capture the inter-class relationship, but also maintains the relative distance between the samples, further optimizes the model performance, uses the knowledge of the source domain to guide the learning process of the target domain model, realizes more stable and robust model learning, and reduces the risk of overfitting. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0039] Figure 1 is a flow chart of an industrial process fault diagnosis method according to an embodiment of the present application;

[0040] Figure 2 is a flow chart of a sewage treatment process based on an AO process; wherein the AO process is used as a source domain data;

[0041] Figure 3 is a flow chart of a sewage treatment process based on an A2O process; wherein the A2O process is used as a target domain data;

[0042] Figure 4 is a schematic diagram of input variables selected in the present example;

[0043] Figure 5 is a network structure diagram of a fault diagnosis common to the source domain and the target domain;

[0044] Figure 6 is a graph of experimental results. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0046] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0047] Embodiment 1:

[0048] As shown in Figure 1 , the present embodiment provides an industrial process fault diagnosis method, which is based on passive domain transfer learning fault diagnosis and can solve the domain shift problem under privacy protection, effectively transfer the knowledge of the source domain model to the target domain, and realize fault diagnosis on the target domain. The specific implementation steps are as follows:

[0049] Step 1: Supervised training is performed on a sample sufficient and labeled source domain fault data set to obtain a well-trained source domain model;

[0050] In step 1, the sewage treatment process is taken as the background, and the AO process commonly used in the sewage treatment process is selected as the source domain, as shown in Figure 2 , the A2O process is taken as the target domain, as shown in Figure 3As shown in Table 1, three fault conditions, i.e., biological tank abnormality, secondary sedimentation tank abnormality, and pipeline flow abnormality, are designed, and four conditions including the normal condition are included. The details are shown in Table 1. Figure 4

[0051] Table 1

[0052]

[0053] The fault data set of the source domain AO is assumed to be represents the input process data, including the active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, etc., as shown in Table 1. Figure 4 represents the fault category. In this example, according to Table 1, respectively represent the normal condition, biological tank abnormality, secondary sedimentation tank abnormality, and pipeline flow abnormality. The process fault diagnosis model of the source domain AO is wherein represents the feature extractor of the source domain, represents the classifier of the source domain, as shown in Table 1. In this example, the number of neurons of Figure 5 is respectively set to {30, 20, 10}, and the number of neurons of is respectively set to {10, 8, 4}. The source domain model is supervised trained by minimizing the cross-entropy loss function, and the specific formula is as follows:

[0054]

[0055] wherein, represents the one-hot encoding of the label y s . If q c = 1, indicating correct classification, and if q c = 0, indicating incorrect classification. Wherein, σ c represents the c-th element of the softmax output of the C-dimensional vector:

[0056]

[0057] wherein, α c represents the c-th element of the vector α output by the classifier, and C represents the total number of fault categories. Thus, the trained source domain AO process fault diagnosis model F is obtained.​​​s .

[0058] Step 2: use the fault diagnosis model F of the source domain AO to initialize the fault diagnosis model of the target domain A2O s The fault diagnosis model of the target domain A2O is initialized, including the feature extractor and the classifier;

[0059] In step 2, it is assumed that the target domain model and the source domain model have the same model structure, as shown in the formula: Figure 5 That is, the fault diagnosis model of the target domain is Wherein represents the feature extractor of the target domain, represents the classifier of the target domain. The number of neurons of the target domain model and the source domain model should be consistent. The target domain model is initialized by using the source domain model, that is, the parameters of the source domain model are all assigned to the target domain model, that is, φ t = φ s , φ represents the network parameters thereof.

[0060] Step 3: input the unlabeled target domain A2O process sample into the source domain model to obtain the classification result and calculate the uncertainty of classification;

[0061] In step 3, the parameters of the target domain model are first fixed for target domain sample classification, and the self-entropy of the classification output is calculated to measure the certainty of the model output. The calculation method of the self-entropy is as follows:

[0062]

[0063] The smaller the value of the self-entropy, the smaller the uncertainty of the classification, and the model has more confidence in the classification result.

[0064] Step 4: select the class prototypes of four types of working conditions, i.e., normal working condition, biological pool abnormality, secondary sedimentation tank abnormality and pipeline flow abnormality, in the target domain sample according to the classification result and the uncertainty;

[0065] In step 4, the self-entropy of the classification result is sorted, and the samples with smaller self-entropy are selected as the class prototypes, which is expressed by the formula:

[0066] M c = {x i |x i ∈ sort(H c (x t )), i≤N}

[0067] Wherein, H c (x t ) represents the sample x tSelf-entropy of the class prototype, sort(·) represents the arrangement in the order of small to large, N represents the number of each class prototype, which needs to be manually set, usually set to a value related to the sample size and the number of categories of the target domain:

[0068]

[0069] Therefore, the pseudo label of these class prototypes can be set as:

[0070]

[0071] Wherein respectively represent that the sample belongs to normal condition, biological pool abnormality, secondary sedimentation tank abnormality, and pipeline flow abnormality respectively.

[0072] Step 5: input the sample of the target domain A2O process into the feature extractor of the target domain model to obtain the sample features, and label all target domain samples with pseudo labels according to the distance between the features and the class prototypes, indicating different fault types;

[0073] In step 5, first input all target domain samples into the feature extractor Calculate the Euclidean distance between the features of the class prototype and other features as the distance between samples:

[0074]

[0075] Wherein, N z represents the dimension of the feature. By calculating the Euclidean distance based on the sample features and the nearest neighbor algorithm, pseudo labels are given to all target domain samples.

[0076] Step 6: classify the target domain samples with pseudo labels based on the target domain model, and calculate the cross entropy classification loss between the pseudo labels and the classification results

[0077] In step 6, the cross entropy loss between the pseudo label and the classification result is The calculation formula is:

[0078]

[0079] Step 7: based on the features and the predicted label, calculate the category loss

[0080] In step 7, the category loss is reduced to maintain the internal structure of the target domain samples, specifically, to concentrate samples with similar categories in the feature space, while ensuring that samples with different categories are separated from each other. Therefore, a center loss is introduced to minimize the intra-class distance by concentrating features of samples of the same class around a class center, while maintaining a large distance between class centers. Thus, the category loss The calculation formula is:

[0081]

[0082] wherein, represents the class center of the category , which is obtained by calculating the average of the features of the samples of the category: The class center is updated during the training process to dynamically reflect the distribution of the current category in the feature space.

[0083] Step 8: input the features of the target domain samples into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss

[0084] In step 8, the knowledge distillation loss is designed and reduced to prevent the target domain model from having a self-bias problem when learning and training. Therefore, the source domain model is introduced as a constraint, and the calculation formula is:

[0085]

[0086] Step 9: train the target domain model with the above cross-entropy classification loss category loss and knowledge distillation loss

[0087] In step 9, the total loss function is:

[0088]

[0089] wherein, λ1 and λ2 are pre-set weighting parameters for balancing each loss. The neural network training method such as gradient descent is used to minimize to train and update the target domain model. The general grid search method is used to set λ1 = 0.2 and λ2 = 0.3.

[0090] Step 10: repeat the above steps 5-9 to update the target domain model and realize transfer learning without source domain data.

[0091] ​In step 10, when steps 5-9 are repeated, pseudo labels are assigned to target domain samples according to different feature differences, so that the pseudo labels are increasingly close to the true labels, and the training of the model is stopped when the loss function value meets the setting or reaches the maximum iteration number. A trained A2O process fault diagnosis model F is obtained t* .

[0092] In step 11, the trained A2O process fault diagnosis model F is used t* Fault diagnosis is performed on the operating condition data of the target domain:

[0093] In step 11, process variables in the A2O sewage treatment process are first collected online, and the collected process variables should be consistent with the 30 process variables in step 1, denoted as Then the process data are input into the trained fault diagnosis model F t* to obtain the fault diagnosis result:

[0094]

[0095] Subsequently, the classification accuracy of the test data is used as an evaluation index of the effectiveness of the model, which is defined as:

[0096]

[0097] Wherein, represents the test data collected from the target domain A2O sewage treatment process, y(x) represents the true fault class label, represents the fault class label predicted by the classifier. The experimental result is Accuracy=0.8813, indicating that the classification accuracy of the fault diagnosis model of the target domain constructed by the source domain fault diagnosis model and the target domain data directly participating in the source domain reaches 0.8813. The fault class classification visualization result is shown in Figure 6 , which illustrates the effectiveness of the present application.

[0098] Example 2:

[0099] The present application also provides an industrial process fault diagnosis device, comprising:

[0100] A first processing module is configured to perform supervised training according to the source domain fault data set to obtain a source domain model;

[0101] A second processing module is configured to initialize a target domain model according to the source domain model parameters;

[0102] A third processing module is configured to input the unlabeled target domain samples into the source domain model to obtain the classification and the classification uncertainty;

[0103] a fourth processing module configured to select a class prototype of the target domain sample according to the classification result and the uncertainty;

[0104] a fifth processing module configured to input the target domain sample into a feature extractor of the target domain model to obtain a sample feature, and label all the target domain samples with pseudo labels according to distances between the features and the class prototypes;

[0105] a sixth processing module configured to classify the target domain sample with the pseudo label based on the target domain model, and calculate a cross-entropy classification loss between the pseudo label and a classification result

[0106] a seventh processing module configured to calculate a class loss based on the feature and the predicted label

[0107] an eighth processing module configured to input the target domain sample feature into a source domain classifier and a target domain classifier respectively, and calculate a knowledge distillation loss

[0108] a ninth processing module configured to train the target domain model based on the cross-entropy classification loss the class loss and the knowledge distillation loss

[0109] a tenth processing module configured to update the target domain model to obtain a trained fault diagnosis model;

[0110] an eleventh processing module configured to perform fault diagnosis on the running condition data of the target domain by using the trained fault diagnosis model.

[0111] As an embodiment of the present application, the AO process commonly used in the sewage treatment process is selected as the source domain, the A2O process is selected as the target domain, the fault data set of the source domain AO is represents the input process data, including the concentration of active heterotrophic biomass, the concentration of autotrophic biomass, the concentration of nitrite, the concentration of dissolved oxygen, the alkalinity, etc. represents the fault class, respectively represent the normal working condition, the abnormal biological pool, the abnormal secondary sedimentation tank and the abnormal pipeline flow.

[0112] As an embodiment of the present application, the tenth processing module is configured to assign pseudo labels to the target domain samples according to different feature differences, so that the pseudo labels are increasingly close to the real labels, and the training is stopped when the loss function value of the model training meets the setting or reaches the maximum iteration number, to obtain the trained fault diagnosis model of the target domain A2O process.

[0113] Example 3: ​

[0114] The embodiment of the present application also provides an industrial process fault diagnosis system, comprising a memory and a processor, the memory storing a computer program run by the processor, the computer program performing the industrial process fault diagnosis method when run by the processor.

[0115] Embodiment 4:

[0116] The embodiment of the present application also provides a storage medium, the storage medium storing a computer program, the computer program performing the industrial process fault diagnosis method when run.

[0117] The above-described embodiments are only descriptions of the preferred modes of the present application, and do not limit the scope of the present application, and various modifications and improvements to the technical solutions of the present application made by those skilled in the art without departing from the design spirit of the present application shall fall within the protection scope of the present application defined by the claims.

Claims

1. A method for diagnosing industrial process faults, characterized in that: include: Step 1: Perform supervised training based on the source domain fault dataset to obtain the source domain model; Step 2: Initialize the target domain model according to the source domain model parameters; Step 3: Input the unlabeled target domain samples into the source domain model to obtain the classification and classification uncertainty; Step 4: Select the category prototype of the target domain sample based on the classification results and uncertainty; Step 5: Input the target domain samples into the feature extractor of the target domain model to obtain sample features, and assign pseudo labels to all target domain samples based on the distance between features and the category prototype; Step 6: Classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results. Step 7: Calculate the category loss based on the features and predicted labels Step 8: Input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss Step 9: Classify loss using cross entropy Class loss and knowledge distillation loss Jointly train the target domain model; Step 10: Repeat steps 5 to 9 above to update the target domain model and obtain a trained fault diagnosis model. Step 11: Use the trained fault diagnosis model to perform fault diagnosis on the operating condition data of the target domain.

2. The industrial process fault diagnosis method according to claim 1, characterized in that: In step 1, the AO process commonly used in sewage treatment is selected as the source domain, and the A2O process is selected as the target domain. The fault dataset of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, Indicates the fault type, They respectively represent normal operating conditions, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

3. The industrial process fault diagnosis method according to claim 2, characterized in that: In step 10, pseudo labels are assigned to target domain samples according to different feature differences, so that the pseudo labels are closer to the true labels. When the loss function value of the model training meets the setting or reaches the maximum number of iterations, the training is stopped to obtain a trained fault diagnosis model of the target domain A2O process.

4. An industrial process fault diagnosis device, characterized in that: include: The first processing module is used to perform supervised training based on the source domain fault dataset to obtain a source domain model; A second processing module is used to initialize the target domain model according to the source domain model parameters; The third processing module is used to input the unlabeled target domain samples into the source domain model to obtain the classification and classification uncertainty; The fourth processing module is used to select the category prototype of the target domain sample according to the classification result and uncertainty; The fifth processing module is used to input the target domain samples into the feature extractor of the target domain model to obtain sample features, and to label all target domain samples with pseudo labels based on the distance between features and the category prototype; The sixth processing module is used to classify the target domain samples with pseudo labels based on the target domain model and calculate the cross entropy classification loss between the pseudo labels and the classification results The seventh processing module is used to calculate the category loss based on features and predicted labels The eighth processing module is used to input the target domain sample features into the source domain classifier and the target domain classifier respectively, and calculate the knowledge distillation loss The ninth processing module is used to classify the loss with cross entropy Class loss and knowledge distillation loss Jointly train the target domain model; The tenth processing module is used to update the target domain model to obtain a trained fault diagnosis model; The eleventh processing module is used to perform fault diagnosis on the operating condition data of the target domain using the trained fault diagnosis model.

5. The industrial process fault diagnosis device according to claim 4, characterized in that: The AO process commonly used in sewage treatment is selected as the source domain, and the A2O process is selected as the target domain. The fault dataset of the source domain AO is Represents input process data, including active heterotrophic biomass concentration, autotrophic biomass concentration, nitrite concentration, dissolved oxygen concentration, alkalinity, Indicates the fault type, They respectively represent normal operating conditions, abnormal biological pool, abnormal secondary sedimentation tank, and abnormal pipeline flow.

6. The industrial process fault diagnosis device according to claim 5, characterized in that: The tenth processing module is used to assign pseudo labels to target domain samples based on different feature differences, so that the pseudo labels are closer to the true labels. Training is stopped until the loss function value of the model training meets the setting or reaches the maximum number of iterations, and a trained fault diagnosis model for the target domain A2O process is obtained.

7. An industrial process fault diagnosis system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the industrial process fault diagnosis method according to any one of claims 1 to 3 is executed.

8. A storage medium, characterized in that: The storage medium stores a computer program, which executes the industrial process fault diagnosis method according to any one of claims 1 to 3 when running.

Images