A fault detection method for long-tailed noisy distribution data

Abstract

The present invention discloses a fault detection method suitable for long-tailed noisy distributed data. In real-world applications, fault detection data often contains noise labels and follows a long-tailed distribution. To mitigate the impact of noise labels on model performance, the present method uses sample selection techniques to separate the noisy labeled data into clean labeled data sets and unlabeled data sets, and then updates the model using a semi-supervised learning approach. To mitigate the impact of the long-tailed distribution on model performance, the present method uses a novel balancing loss based on model preference to correct the model. Compared to existing technologies, the present method has greater robustness for fault detection of long-tailed noisy distributed data.

Description

Technical Field

[0001] A fault detection method suitable for long-tail noisy distributed data can be used to train a deep neural network model using fault detection data with noise labels and a long-tail distribution, and achieve good fault detection performance. The method belongs to the field of computer artificial intelligence data analysis technology. Background Art

[0002] Deep learning training typically requires large, well-organized datasets. In fault detection, data on different faults often have varying frequencies, and the collected data often exhibits a long-tail distribution. When training neural networks with long-tail data, the performance of the neural network is easily affected by the head class, while performance on the tail class is poor. Furthermore, labeled fault data often contains noise, and unprocessed training using this noisy labeled data can severely impact the performance of deep neural network models. Therefore, training fault detection models based on long-tail, noisy data is a challenging problem. Summary of the Invention

[0003] Purpose of the invention: Fault detection data in the real world usually contains noise labels and obeys a long-tail distribution. It is difficult to learn a deep neural network model with good performance using this type of data for training. To address this problem, the present invention proposes a fault detection method suitable for long-tail noisy distributed data. Specifically, in order to reduce the impact of noise labels on model performance, the method of the present invention uses sample selection technology to divide the noise labeled data into a clean labeled data set and an unlabeled data set, and then updates the neural network in a semi-supervised learning manner; in order to reduce the impact of the long-tail distribution on model performance, the method of the present invention uses a new balanced loss based on model preference to correct the deep neural network model. The present invention can be trained based on fault detection data with a long-tail noisy distribution to obtain a fault detection model with good performance, and has strong practicality.

[0004] The present invention provides a fault detection method applicable to long-tail noisy distribution data, comprising the following steps:

[0005] (1) For a fault detection database containing noise labels and obeying a long-tail distribution Specifying hyperparameters such as the deep neural network model f(·; Θ) and the number of training rounds at each stage and preheating the deep neural network model for a predetermined number of rounds on all training data using the cross entropy loss function and the regularization loss proposed in the present invention;

[0006] (2) Using a class-by-class selection method, samples with smaller cross-entropy loss values ​​are selected from the training data as the clean labeled data set, and the remaining unselected data are regarded as the unlabeled data set. A new pseudo-labeled data set is generated based on the clean labeled data set and the unlabeled data set obtained by screening;

[0007] (3) Using the Cross-Entropy loss function (i.e., cross entropy loss function), the MSE loss function, and the regularization loss proposed by the present invention to calculate the overall loss of the model on the pseudo-labeled data generated in step (2) and update the deep neural network model, while estimating the model preference matrix of this round by averaging the model output on each class of clean labeled samples;

[0008] (4) looping steps (2) and (3) for a certain number of rounds to update the deep neural network and calculate the model preference matrix for each round; using the exponential moving average method to weighted average the model preference matrix in each round to obtain the final model preference matrix;

[0009] (5) Using step (2) to select samples and generate pseudo-labeled data, using the mean square error loss function and the balanced loss function and regularization loss proposed in the present invention to calculate the overall loss of the model on the generated pseudo-labeled data set and update the deep neural network model;

[0010] (6) Repeat step (5) for a certain number of rounds to obtain the final deep neural network model; in the testing phase, the user inputs the sample to be tested into the deep neural network model, and then selects the category with higher prediction confidence as the prediction result of the model.

[0011] Furthermore, first for a fault detection database containing noise labels and obeying a long-tail distribution The number of data categories is C, and the data subset marked as c is The number of samples labeled c is n c ; Select a deep neural network model and record it as f(·;Θ).

[0012] Furthermore, combining the cross entropy loss function and the regularization loss L reg Preheat the deep neural network model, the regularization loss L reg is the following formula, where i, j, and c are category numbers

[0013]

[0014] Furthermore, the class-by-class sample selection process based on the cross entropy loss value of the sample in step (2) is:

[0015] Step 100) Based on the current deep neural network model, the cross entropy loss function l is used to calculate the loss value set of each class of sample set, which is recorded as in

[0016] Step 101) For any category c, the loss value set L c , use the binary Gaussian mixture model (g1, g0) to fit its distribution and get any sample of this category The posterior probability of belonging to the Gaussian component g1 with a smaller mean And according to whether the value of the posterior probability is greater than 0.5, the sample is divided into a clean labeled sample or an unlabeled sample;

[0017] Step 102) Apply step 101) to each type of data to obtain the final labeled data and unlabeled data

[0018] Furthermore, based on the labeled data and unlabeled data The process of generating pseudo-labeled data is as follows:

[0019] Step 200) For each labeled sample and unlabeled data Do K data enhancements respectively to get

[0020] and Generate pseudo labels for unlabeled samples based on deep neural network models and augmented unlabeled data in T is a hyperparameter;

[0021] Step 201) Finally, there is a pseudo-labeled data set

[0022] Furthermore, the cross entropy loss function, MSE loss function and regularization loss L reg Applied to pseudo-labeled datasets and Get the overall loss of the model And update the deep neural network model, where

[0023]

[0024]

[0025]

[0026] λ u and λ reg is a hyperparameter,

[0027] Furthermore, the process of estimating the model preference matrix by the average output of the deep neural network model on each class of clean labeled samples in the clean labeled data set is as follows:

[0028] make Represents the data set labeled i obtained by sample selection, then the model preference matrix of the deep neural network model in the current round t is in i, j, and c are category numbers.

[0029] Furthermore, the process of using the exponential moving average method to weight the average model preference matrix in each round to obtain the final model preference matrix is ​​as follows:

[0030] Step 301) Initialize the overall model preference matrix to an all-zero matrix Round number t = 0;

[0031] Step 302) Update the overall model preference matrix based on the model preference matrix at round t: Where δ is a hyperparameter; let the number of rounds t = t + 1;

[0032] Step 303) Repeat step 302 for a certain number of rounds to obtain the final overall model preference matrix

[0033] Furthermore, we use the MSE loss function, the balanced loss function, and the regularization loss L reg Calculate the overall loss of the model and update the deep neural network model,

[0034]

[0035]

[0036]

[0037] in λ u ,λ reg , γ sup and γ rel is a hyperparameter, is the indicator function, i, j, and c are category numbers.

[0038] Beneficial effects: Compared with the prior art, the fault detection method proposed in the present invention, which is applicable to long-tailed noisy distribution data, takes into account the long-tail distribution of different types of faults in fault detection and the noise problem of fault labeling. In order to reduce the impact of noise labels on model performance, the method of the present invention uses sample selection technology to divide the noise labeled data into a clean labeled data set and an unlabeled data set, and then updates the neural network in a semi-supervised learning manner; in order to reduce the impact of long-tail distribution on model performance, the method of the present invention uses a new balanced loss based on model preference to correct the deep neural network model. Therefore, compared with the prior art, the present invention has stronger robustness to long-tail distribution and noise labels, and can learn a deep neural network model with good fault detection performance from fault detection data with long-tailed noisy distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the present invention;

[0040] Figure 2 4 is a flow chart of the method for selecting samples by category in the present invention. DETAILED DESCRIPTION

[0041] The present invention is further illustrated below with reference to specific examples. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0042] like Figure 1 As shown in FIG, a fault detection method applicable to long-tailed noisy distribution data includes the following steps:

[0043] Step 100: for a fault detection database containing noise labels and obeying a long-tail distribution The number of data categories is C, and the data subset marked as c is The number of samples labeled c is n c ;

[0044] Step 101: Select a deep neural network model and record it as f(·; Θ); set the total number of model training rounds to T and the number of preheating rounds to T w , model preference matrix estimation round number T m , current round number t=0;

[0045] Step 102: Use the cross entropy loss function and the regularization loss L proposed in the present invention. reg In the training data Calculate the overall loss of the current deep neural network model and update the model, where L reg As shown below, i, j, and c are category numbers.

[0046]

[0047] Step 103, set t=t+1; if t=T w , then jump to step 104, otherwise jump to step 102;

[0048] Step 104: Initialize the overall model preference matrix to an all-zero matrix

[0049] Step 105, based on the current deep neural network model, the cross entropy loss function l is used to calculate the loss value set of each type of sample set, which is recorded as in

[0050] Step 106: For any category c, the loss value set L c , use the binary Gaussian mixture model (g1, g0) to fit its distribution and get any sample of this category The posterior probability of belonging to the Gaussian component g1 with a smaller mean And according to whether the value of the posterior probability is greater than 0.5, the sample is divided into a clean labeled sample or an unlabeled sample;

[0051] Step 107: Apply step 106 to each type of data to obtain the final labeled data and unlabeled data

[0052] Step 108: for each labeled sample and unlabeled data Do K data enhancements respectively to get and Generate pseudo labels for unlabeled samples based on deep neural network models and augmented unlabeled data in T is a pre-set hyperparameter, i, j, and c are category numbers;

[0053] Step 109, let Finally, we get a pseudo-labeled data set

[0054] Step 110: Apply the Cross-Entropy loss function, the MSE loss function, and the regularization loss proposed by the present invention to the pseudo-labeled data set. and Get the overall loss of the model and update the model, where

[0055]

[0056]

[0057]

[0058] λ u and λ reg are pre-set hyperparameters,

[0059] Step 111: Represents the data set marked as i obtained by sample selection, then the model preference matrix of the model in the current round t is in i, j, and c are category numbers;

[0060] Step 112: Update the overall model preference matrix based on the model preference matrix at round t: Where δ is a pre-set hyperparameter;

[0061] Step 113, set t=t+1; if t=t w +T m , then jump to step 114, otherwise jump to step 105;

[0062] Step 114: Execute steps 105 to 109 in sequence to select samples and obtain a pseudo-labeled data set.

[0063]

[0064] Step 115: Use the MSE loss function and the balance loss function and regularization loss proposed in the present invention to calculate the overall loss of the model. And update the deep neural network model, where

[0065]

[0066]

[0067]

[0068] λ u ,λ reg , γ sup and γ rel is a pre-set hyperparameter, is the indicator function, i, j, and c are category numbers;

[0069] Step 116, set t = t + 1; if t = T, the model training is completed, otherwise jump to step 114.

[0070] The above description is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principles of the present invention. These improvements should also be regarded as the scope of protection of the present invention.

Claims

1. A fault detection method for long-tail noisy distribution data, comprising the following steps: (1) In a fault detection database containing noisy labels and obeying a long-tail distribution , selecting a deep neural network model, the fault detection database includes a number of samples; (2) Combined with cross entropy loss function and regularization loss Perform a predetermined number of training rounds on all samples in the deep neural network model on all training data in the fault detection database; (3) Based on the cross entropy loss value of the samples, samples with smaller loss are selected from all the training data class by class as the clean labeled data set, and the data of the remaining unselected samples are regarded as the unlabeled data set, and a new pseudo-labeled data set is generated based on the screened labeled data set and the unlabeled data set; (4) The cross entropy loss function , MSE loss function L u and regularization loss Apply the pseudo-labeled data set generated in step (3) to obtain the final loss of the model, and update the deep neural network model. At the same time, the model preference matrix is ​​estimated by the average output of the updated deep neural network model on each class of clean labeled samples in the clean labeled data set; (5) Repeat steps (3) and (4) for a certain number of rounds, and use the exponential moving average method to weight the average model preference matrix in each round to obtain the final model preference matrix; (6) Use step (3) to select samples and generate pseudo-labeled data, and combine the model preference matrix to calculate the balanced loss function, respectively, and use the MSE loss function, the balanced loss function, and the regularization loss function. The final loss is obtained by applying it to the generated pseudo-labeled data and updating the deep neural network model again. The specific calculation formula of the balanced loss function is: ， in, , and is the model preference matrix; , and is a hyperparameter, is the indicator function, , All are category numbers; (7) Repeat step (6) for a certain number of rounds to obtain the final deep neural network model; in the testing phase, the deep neural network model takes the sample to be tested as input and outputs the category with the highest confidence as the prediction result.

2. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: First, for a fault detection database containing noise labels and obeying a long-tail distribution , the number of categories of data is , marked as The data subset is , the number of samples marked as c is ; Select the deep neural network model and record it as .

3. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: Combined with cross entropy loss function and regularization loss Preheat the deep neural network model, the regularization loss is the following formula, ， in , All are category numbers.

4. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: The class-by-class sample selection process based on the cross entropy loss value of the sample in step (3) is: Step 100) Using the cross entropy loss function based on the current deep neural network model Calculate the loss value set of each type of sample set separately, recorded as ,in ; Step 101) For any category c, the loss value set , using a bivariate Gaussian mixture model ( 、 ) fits its distribution and obtains any sample of this category Belongs to the Gaussian component with a smaller mean The posterior probability , and according to whether the value of the posterior probability is greater than 0.5, the sample is divided into a clean labeled sample or an unlabeled sample; Step 102) Apply step 101) to each type of data to obtain the final labeled data and unlabeled data .

5. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: Based on labeled data and unlabeled data The process of generating pseudo-labeled data is as follows: Step 200) For each labeled sample and unlabeled data Do it separately Second data enhancement and , generating pseudo labels for unlabeled samples based on deep neural network models and enhanced unlabeled data ,in , T is a hyperparameter; Step 201) Order , , , and finally there is a pseudo-labeled data set , .

6. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: The cross entropy loss function , MSE loss function L u and regularization loss Applied to pseudo-labeled datasets and Get the overall loss of the model And update the deep neural network model, where ， ， ， and is a hyperparameter, , All are category numbers.

7. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: The process of estimating the model preference matrix by the average output of the deep neural network model on each class of clean labeled samples in the clean labeled data set is as follows: make Indicates that the mark obtained by sample selection is The data set of , then the model preference matrix of the deep neural network model in the current round t is ,in , All are category numbers.

8. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: The process of using the exponential moving average method to weight the average model's preference matrix in each round to obtain the final model preference matrix is ​​as follows: Step 301) Initialize the overall model preference matrix to an all-zero matrix , round number t=0; Step 302) Update the overall model preference matrix based on the model preference matrix at round t: ,in is a hyperparameter; the number of rounds ; Step 303) Repeat step 302 for a certain number of rounds to obtain the final overall model preference matrix .

9. The fault detection method applicable to long-tail noisy distribution data according to claim 1, characterized in that: Use MSE loss function and balanced loss function and regularization loss Calculate the overall loss of the model and update the deep neural network model, ， ， Among them, λ u ,λ reg is a hyperparameter.

Images