Label data cleaning method and system based on failure sample contribution explanation

Abstract

The application provides a label data cleaning method and system based on fault sample contribution explanation, and the method comprises the following steps: step S1: dividing fault sample data to form a plurality of fault sample clustering clusters; step S2: using a simple model to perform preliminary screening of the fault sample clustering clusters to form a set of fine screening fault sample clusters; step S3: enumerating all combination methods of the clusters in the set of fine screening fault sample clusters, training a fault detection model, and obtaining the effect score of each model under various combinations; and step S4: based on the evaluation effect of each model, calculating the average marginal contribution value of each fine screening sample cluster to all combinations of the fault detection model, and cleaning data based on the average marginal contribution value. The application improves the efficiency of cleaning abnormal "label data", can effectively clean "label" samples irrelevant to "features", and has good interpretability.

Description

Technical Field

[0001] This invention relates to the technical field of data cleaning, and more specifically, to a method and system for cleaning labeled data based on the contribution interpretation of faulty samples. Background Technology

[0002] Using machine learning models for fault detection is a key development direction in AIOps (Intelligent Operations and Maintenance). The quality of the data used to train these models has a significant impact on the training results, and "data cleaning" is the process of improving data quality. Data used for supervised machine learning consists of two parts: "feature data" and "label data." Currently, most data cleaning methods primarily focus on "feature data," performing operations such as "missing value imputation," "outlier correction," and "duplicate value removal." Methods for handling unreasonable data in "label data" often involve:

[0003] 1) Manual processing: This method has the disadvantages of large workload and inability to effectively clean "label data" that is unrelated to "feature data" (labels that are unrelated to "features" are also a kind of dirty data, which will seriously affect the quality of the model).

[0004] 2) The method of discarding a portion of data that is unclear in the model by proportion is implemented. However, this method lacks a basis for cleaning and is random.

[0005] A data cleaning method is disclosed in patent document CN115481114A, applied in the field of data processing technology. The method provided in this application includes: acquiring target data cleaning tasks containing a target database of a target system and target data tables of the target database; acquiring the number of deployment environments of the target system; splitting the target data cleaning tasks into a data acquisition task list based on the number of deployment environments and the hardware resource performance of the deployment environments; executing the data acquisition tasks according to preset data acquisition rules and adding them to a set of data tables to be processed; generating a data processing task list for the target data tables based on the target data cleaning tasks; executing the data processing tasks in the data processing task list, using the processed target data tables as data tables to be updated; acquiring the data tables to be updated according to preset data update rules, and updating the data tables to be updated in the deployment environment of the target system. This patent mainly emphasizes the automation and formalization of data cleaning tasks using databases as data sources, but does not address the cleaning of problematic "tagged data."

[0006] Therefore, a new technical solution is needed to improve the above-mentioned technical problems. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a label data cleaning method and system based on the contribution interpretation of fault samples.

[0008] According to the present invention, a label data cleaning method based on the contribution interpretation of fault samples is provided, the method comprising the following steps:

[0009] Step S1: Divide the fault sample data into multiple fault sample clusters;

[0010] Step S2: Use a simple model to perform initial screening of fault sample clusters to form a set of finely screened fault sample clusters;

[0011] Step S3: Exhaustively enumerate all combinations of clusters in the set of all finely screened fault sample clusters, train the fault detection model, and obtain the performance score of each model under various combinations;

[0012] Step S4: Based on the evaluation effect of each model, calculate the average marginal contribution value of each fine-screen sample cluster to all combinations of the fault detection model, and clean the data based on the average marginal contribution value.

[0013] Preferably, step S1 includes the following steps:

[0014] Step S1.1: Cluster the fault sample data;

[0015] Step S1.2: Based on experience, re-divide the sample clusters with fewer samples, and set the minimum number of samples for each sample cluster, denoted as min_cluster_size. Sample clusters with fewer samples than min_cluster_size are called sample clusters with fewer samples.

[0016] Preferably, step S2 includes the following steps:

[0017] Step S2.1: Use a simple model to perform a one-to-one fitting on all sample clusters one by one, and obtain the gain score of each sample cluster;

[0018] Step S2.2: Obtain a set of finely screened sample clusters based on the gain scores. Sort the gain scores of each cluster from low to high, and select the x sample clusters with the lowest scores, or select the sample clusters with gain scores less than thre_lift and less than or equal to 1 floating-point numbers, to form a set of finely screened sample clusters.

[0019] Step S2.1 includes the following steps:

[0020] Step S2.1.1: Train a simple fault classification model using all fault sample clusters and non-fault sample data, and calculate the AUC value of the model, which is denoted as auc_all;

[0021] Step S2.1.2: Select sample cluster c from the set of sample clusters to be screened without repetition, remove sample cluster c from the set of sample clusters to be screened, and merge the data with the non-fault sample data to obtain the defitting training data of sample cluster c.

[0022] Step S2.1.3: Train a simple fault classification model using the unfit training data of sample cluster c, and then calculate the AUC value of the model, denoted as auc_remove_c;

[0023] Step S2.1.4: Calculate the gain score of sample cluster c, denoted as lift_score_c:

[0024] lift_score_c=auc_all-auc_remove_c;

[0025] Step S2.1.5: Repeat steps S2.1.2 to S2.1.4 to obtain the gain score for each cluster.

[0026] Preferably, step S3 includes the following steps:

[0027] Step S3.1: Generate all possible cluster combinations in the set of finely screened sample clusters;

[0028] Step S3.2: Train the model for each combination of training clusters to obtain the corresponding fault detection model and obtain the model's evaluation score.

[0029] Preferably, step S4 includes the following steps:

[0030] Step S4.1: Calculate the average marginal contribution of each cluster under various combinations of training models, denoted as d_i-j, where i and j are the training set numbers. Calculate the marginal gain of each cluster under all scenarios and calculate the mean of the marginal gain.

[0031] Step S4.2: Remove training clusters with negative marginal gain from the training data to complete data cleaning.

[0032] This invention also provides a labeled data cleaning system based on the contribution interpretation of fault samples, the system comprising the following modules:

[0033] Module M1: Divides the fault sample data into multiple fault sample clusters;

[0034] Module M2: Uses a simple model to perform initial screening of fault sample clusters, forming a set of finely screened fault sample clusters;

[0035] Module M3: Exhaustively enumerate all combinations of clusters in the set of all finely screened fault sample clusters, train the fault detection model, and obtain the performance score of each model under various combinations;

[0036] Module M4: Based on the evaluation performance of each model, calculate the average marginal contribution value of each fine-screen sample cluster to all combinations of the fault detection model, and clean the data based on the average marginal contribution value.

[0037] Preferably, module M1 includes the following modules:

[0038] Module M1.1: Clusters fault sample data;

[0039] Module M1.2: Based on experience, re-divide the sample clusters with fewer samples, and set the minimum number of samples for each sample cluster, denoted as min_cluster_size. Sample clusters with fewer samples than min_cluster_size are called sample clusters with fewer samples.

[0040] Preferably, module M2 includes the following modules:

[0041] Module M2.1: Uses a simple model to perform one-to-one fitting on all sample clusters and obtains the gain score for each sample cluster;

[0042] Module M2.2: Based on the gain score, obtain a set of finely screened sample clusters, sort the gain scores of each cluster from low to high, select the x sample clusters with the lowest scores, or select the sample clusters with gain scores less than thre_lift and less than or equal to 1 floating-point numbers, and together form the set of finely screened sample clusters.

[0043] Module M2.1 includes the following modules:

[0044] Module M2.1.1: Train a simple fault classification model using all fault sample clusters and non-fault sample data, and calculate the AUC value of the model, which is denoted as auc_all;

[0045] Module M2.1.2: Select sample cluster c from the set of sample clusters to be screened without repetition, remove sample cluster c from the set of sample clusters to be screened, and merge the data with the non-fault sample data to obtain the defitting training data of sample cluster c.

[0046] Module M2.1.3: Train a simple fault classification model using the unfit training data of sample cluster c, and then calculate the AUC value of the model, denoted as auc_remove_c;

[0047] Module M2.1.4: Calculates the gain score of sample cluster c, denoted as lift_score_c:

[0048] lift_score_c=auc_all-auc_remove_c;

[0049] Module M2.1.5: Repeat modules M2.1.2 to M2.1.4 to obtain the gain score for each cluster.

[0050] Preferably, module M3 includes the following modules:

[0051] Module M3.1: Generates all possible cluster combinations in the set of clusters for the finely screened sample;

[0052] Module M3.2: Trains the model for each combination of training clusters to obtain the corresponding fault detection model and obtains the model's evaluation score.

[0053] Preferably, module M4 includes the following modules:

[0054] Module M4.1: Calculate the average marginal contribution of each cluster under various combinations of training models, denoted as d_i-j, where i and j are the training set numbers, calculate the marginal gain of each cluster under all scenarios, and calculate the mean of the marginal gain;

[0055] Module M4.2: Removes training clusters with negative marginal gain mean from the training data, completing the data cleaning process.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. This invention improves the efficiency of cleaning abnormal "label data", effectively cleans "label" samples that are unrelated to "features", and has good interpretability;

[0058] 2. This invention quantifies the contribution of faulty samples to the machine learning model, finds the sample set that has a negative contribution to the model, and thus completes the cleaning process, which has good interpretability.

[0059] 3. This invention can clean up abnormal "tag data" and quantifies the degree of abnormality during the cleaning process, demonstrating good interpretability.

[0060] 4. This invention uses two steps, primary screening and fine screening, to complete data cleaning, which improves the efficiency of data cleaning and can effectively adapt to sample data cleaning in the case of big data.

[0061] 5. This invention only requires human intervention for processing a small number of samples, thus reducing the labor costs of data cleaning. Attached Figure Description

[0062] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0063] Figure 1 This invention provides a partitioning diagram of the fault sample data;

[0064] Figure 2 This is a cluster diagram of fault samples initially screened using the simple model of this invention;

[0065] Figure 3 This is a performance score chart for each combination of finely screened sample clusters obtained in this invention;

[0066] Figure 4 This is a data cleaning graph based on the average marginal contribution value of the present invention. Detailed Implementation

[0067] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0068] Example 1:

[0069] According to the present invention, a label data cleaning method based on the contribution interpretation of fault samples is provided, the method comprising the following steps:

[0070] Step S1: Divide the fault sample data into multiple fault sample clusters;

[0071] Step S1.1: Cluster the fault sample data;

[0072] Step S1.2: Based on experience, re-divide the sample clusters with fewer samples, and set the minimum number of samples for each sample cluster, denoted as min_cluster_size. Sample clusters with fewer samples than min_cluster_size are called sample clusters with fewer samples.

[0073] Step S2: Use a simple model to perform initial screening of fault sample clusters to form a set of finely screened fault sample clusters;

[0074] Step S2.1: Use a simple model to perform a one-to-one fitting on all sample clusters one by one, and obtain the gain score of each sample cluster;

[0075] Step S2.1.1: Train a simple fault classification model using all fault sample clusters and non-fault sample data, and calculate the AUC value of the model, which is denoted as auc_all;

[0076] Step S2.1.2: Select sample cluster c from the set of sample clusters to be screened without repetition, remove sample cluster c from the set of sample clusters to be screened, and merge the data with the non-fault sample data to obtain the defitting training data of sample cluster c.

[0077] Step S2.1.3: Train a simple fault classification model using the unfit training data of sample cluster c, and then calculate the AUC value of the model, denoted as auc_remove_c;

[0078] Step S2.1.4: Calculate the gain score of sample cluster c, denoted as lift_score_c:

[0079] lift_score_c=auc_all-auc_remove_c;

[0080] Step S2.1.5: Repeat steps S2.1.2 to S2.1.4 to obtain the gain score for each cluster.

[0081] Step S2.2: Obtain a set of finely screened sample clusters based on the gain scores. Sort the gain scores of each cluster from low to high, and select the x sample clusters with the lowest scores, or select the sample clusters with gain scores less than thre_lift and less than or equal to 1 floating-point numbers, to form a set of finely screened sample clusters.

[0082] Step S3: Exhaustively enumerate all combinations of clusters in the set of all finely screened fault sample clusters, train the fault detection model, and obtain the performance score of each model under various combinations;

[0083] Step S3.1: Generate all possible cluster combinations in the set of finely screened sample clusters;

[0084] Step S3.2: Train the model for each combination of training clusters to obtain the corresponding fault detection model and obtain the model's evaluation score.

[0085] Step S4: Based on the evaluation effect of each model, calculate the average marginal contribution value of each fine-screen sample cluster to all combinations of the fault detection model, and clean the data based on the average marginal contribution value;

[0086] Step S4.1: Calculate the average marginal contribution of each cluster under various combinations of training models, denoted as d_i-j, where i and j are the training set numbers. Calculate the marginal gain of each cluster under all scenarios and calculate the mean of the marginal gain.

[0087] Step S4.2: Remove training clusters with negative marginal gain from the training data to complete data cleaning.

[0088] The present invention also provides a label data cleaning system based on fault sample contribution interpretation. The label data cleaning system based on fault sample contribution interpretation can be implemented by executing the process steps of the label data cleaning method based on fault sample contribution interpretation. That is, those skilled in the art can understand the label data cleaning method based on fault sample contribution interpretation as a preferred embodiment of the label data cleaning system based on fault sample contribution interpretation.

[0089] Example 2:

[0090] This invention also provides a labeled data cleaning system based on the contribution interpretation of fault samples, the system comprising the following modules:

[0091] Module M1: Divides the fault sample data into multiple fault sample clusters;

[0092] Module M1.1: Clusters fault sample data;

[0093] Module M1.2: Based on experience, re-divide the sample clusters with fewer samples, and set the minimum number of samples for each sample cluster, denoted as min_cluster_size. Sample clusters with fewer samples than min_cluster_size are called sample clusters with fewer samples.

[0094] Module M2: Uses a simple model to perform initial screening of fault sample clusters, forming a set of finely screened fault sample clusters;

[0095] Module M2.1: Uses a simple model to perform one-to-one fitting on all sample clusters and obtains the gain score for each sample cluster;

[0096] Module M2.1.1: Train a simple fault classification model using all fault sample clusters and non-fault sample data, and calculate the AUC value of the model, which is denoted as auc_all;

[0097] Module M2.1.2: Select sample cluster c from the set of sample clusters to be screened without repetition, remove sample cluster c from the set of sample clusters to be screened, and merge the data with the non-fault sample data to obtain the defitting training data of sample cluster c.

[0098] Module M2.1.3: Train a simple fault classification model using the unfit training data of sample cluster c, and then calculate the AUC value of the model, denoted as auc_remove_c;

[0099] Module M2.1.4: Calculates the gain score of sample cluster c, denoted as lift_score_c:

[0100] lift_score_c=auc_all-auc_remove_c;

[0101] Module M2.1.5: Repeat modules M2.1.2 to M2.1.4 to obtain the gain score for each cluster.

[0102] Module M2.2: Based on the gain score, obtain a set of finely screened sample clusters, sort the gain scores of each cluster from low to high, select the x sample clusters with the lowest scores, or select the sample clusters with gain scores less than thre_lift and less than or equal to 1 floating-point numbers, and together form the set of finely screened sample clusters.

[0103] Module M3: Exhaustively enumerate all combinations of clusters in the set of all finely screened fault sample clusters, train the fault detection model, and obtain the performance score of each model under various combinations;

[0104] Module M3.1: Generates all possible cluster combinations in the set of clusters for the finely screened sample;

[0105] Module M3.2: Trains the model for each combination of training clusters to obtain the corresponding fault detection model and obtains the model's evaluation score.

[0106] Module M4: Based on the evaluation performance of each model, calculate the average marginal contribution value of each fine-screen sample cluster to all combinations of the fault detection model, and clean the data based on the average marginal contribution value;

[0107] Module M4.1: Calculate the average marginal contribution of each cluster under various combinations of training models, denoted as d_i-j, where i and j are the training set numbers, calculate the marginal gain of each cluster under all scenarios, and calculate the mean of the marginal gain;

[0108] Module M4.2: Removes training clusters with negative marginal gain mean from the training data, completing the data cleaning process.

[0109] Example 3:

[0110] This paper proposes a label data cleaning method based on the interpretation of the contributions of faulty samples. This method identifies the sample set that negatively contributes to the machine learning model by interpreting the contribution of faulty samples, thus completing the cleaning process. This method improves the efficiency of cleaning label data containing anomalous "labels," effectively cleans "labels" samples unrelated to "features," and has good interpretability.

[0111] This invention provides a label data cleaning method based on the contribution interpretation of fault samples, comprising the following steps:

[0112] Step 1: Divide the fault sample data into multiple fault sample clusters. Clusters containing a small number of samples are then manually merged based on empirical evidence.

[0113] Step 2: Use a simple model to perform initial screening of fault sample clusters, forming a set of finely screened fault sample clusters;

[0114] Step 3: Exhaustively enumerate all combinations of clusters in the set of all finely screened fault sample clusters, train the fault detection model, and obtain the performance score of each model under various combinations;

[0115] Step 4: Based on the evaluation results of each model, calculate the average marginal contribution value of each fine-screen sample cluster to all combinations of the fault detection model, and clean the data based on the average marginal contribution value.

[0116] Step 1 includes the following steps:

[0117] Step 1.1: Cluster the fault sample data. The sample data used to train the fault classification model consists of two parts: "feature data" and "label data". The "feature data" part is used as a vector for clustering. First, the CANOPY algorithm is used to cluster, obtaining the possible number of clusters, denoted as num_cluster; then, the KMeans algorithm is used, where the number of clusters k is selected as num_cluster. The final clustering result has num_cluster clusters. For example, the sample data (label 1 represents "fault", 0 represents "normal", and all clustered data are fault samples):

[0118] Sample 1, Features: [0.23, 0.25, 0.001, 3.01…7.65], Label: 1

[0119] Sample 2, features: [0.18, -0.76, 0.0, 8.56…2.33], label: 1

[0120] …

[0121] Sample n, features: [0.28, 0.82, -3.27, 9.86…5.86], label: 1

[0122] The "sample clusters" of the clustering results are as follows:

[0123] Cluster 1: Sample 12, Sample 25, Sample 36… Sample 103

[0124] Cluster 2: Sample 22, Sample 35, Sample 78… Sample 100

[0125] …

[0126] Cluster num_cluster: Sample 1, Sample 3, Sample 7... Sample 95

[0127] Step 1.2: Re-divide the "few-sample" clusters based on expert experience. Set a minimum number of samples for each cluster, denoted as `min_cluster_size` (e.g., 5). Clusters with fewer samples than `min_cluster_size` are called "few-sample" clusters. Collect samples from all "few-sample" clusters and then re-divide them according to expert experience until the partitioning stops when either of the following rules is met: "All rules have been executed" or "The cluster size is greater than the minimum value of `min_cluster_size`". Examples of expert experience include:

[0128] The anomalies occurred within a 1-minute interval and belonged to the same IP address and port.

[0129] The anomalies occurred within one minute of each other;

[0130] The anomaly belongs to the same IP and port.

[0131] The sample clusters are divided sequentially based on expert experience until the stopping rule is met. All sample clusters that are not "few samples" and the sample clusters re-divided based on expert experience together constitute the "set of sample clusters to be screened".

[0132] Step 2 includes the following steps:

[0133] Step 2.1: Use a simple model to perform "one-to-one fitting" on all sample clusters one by one, and obtain the gain score for each sample cluster. The process is as follows:

[0134] Step 2.1.1: Train a simple fault classification model (e.g., logistic regression model) using all fault sample clusters and "non-fault sample data", and calculate the AUC value (Are Under Curve, a classification model evaluation metric) of the model, and denote the AUC value as auc_all;

[0135] Step 2.1.2: Select sample cluster c from the "set of sample clusters to be screened" without repetition, remove sample cluster c from the "set of sample clusters with screening", and merge this data with the "non-fault sample data" to obtain the "one-fit" training data of sample cluster c.

[0136] Step 2.1.3: Train a simple fault classification model (e.g., a logistic regression model, which must be the same type as in Step 2.1.1) using the "remove one fit" training data of sample cluster c, and then calculate the AUC value of the model, denoted as auc_remove_c;

[0137] Step 2.1.4: Calculate the gain score of sample cluster c, denoted as lift_score_c:

[0138] lift_score_c=auc_all-auc_remove_c

[0139] Step 2.1.5: Repeat steps 2.1.2 to 2.1.4 to obtain the gain score for each cluster.

[0140] Step 2.2: Obtain the set of finely screened sample clusters based on the gain scores. Sort the gain scores of each cluster from low to high, and select the x sample clusters with the lowest scores, or select the sample clusters with gain scores less than thre_lift (floating-point numbers less than or equal to 1), to form the set of finely screened sample clusters.

[0141] Step 3 includes the following steps:

[0142] Step 3.1: Generate all possible cluster combinations in the refined sample cluster set. For example, if there are n_final sample clusters in the refined sample cluster set, then the total number of training combinations that can be generated is:

[0143]

[0144] For example, if there are 3 sample clusters c1, c2, and c3 in the set of finely screened sample clusters, then there are a total of C(3,1)+C(3,2)+C(3,3)=7 combinations of training clusters, namely: c1, c2, c3, {c1,c2}, {c1,c3}, {c2,c3}, {c1,c2,c3}.

[0145] Step 3.2: Train the model for each training cluster combination to obtain the corresponding fault detection model and its evaluation score. For example, perform 3-fold cross-validation (a common model validation method in machine learning) on ​​each training set to obtain the evaluation score of the fault detection model. The score is calculated as f1 * num_anomaly_ratio, where f1 is a metric for evaluating classification models, representing the harmonic mean of precision and recall. Here, f1 can be replaced with any metric that can evaluate model performance, such as accuracy, cross-entropy, or mean squared error. num_anomaly_ratio represents the proportion of fault samples used to build the current model out of the total number of fault samples. Based on the example in Step 3.1, the result of this step is as follows:

[0146] score_1=0.3, score_2=0.82, score_3=0.81,

[0147] score_1,2=0.62, score_1,3=0.56, score_2,3=0.87,

[0148] score_1,2,3=0.8

[0149] Where score_x,y… represents the evaluation score obtained after training the model using clusters x,y… together.

[0150] Step 4 includes the following steps:

[0151] Step 4.1: Calculate the average marginal contribution of each cluster trained under various combinations, denoted as d_i-j, where i and j are the training set numbers. For example, d_23-2 represents the difference between the "evaluation scores" obtained by training the model using cluster combinations {c2,c3} and c2 respectively, i.e., the marginal gain brought by c3 in this case. Calculate the marginal gain of each cluster under all scenarios and calculate the mean of the marginal gain. Based on the example in Step 3.2, the following examples of marginal gains can be obtained:

[0152] Cluster 1 Cluster 2 Cluster 3 d_1,2-2＝-0.2 d_1,2-1＝0.32 d_1,3-1＝0.26 d_1,3-3＝-0.25 d_2,3-3＝0.06 d_2,3-2＝0.05 d_1,2,3-2,3＝-0.07 d_1,2,3-1,3＝0.24 d_1,2,3-1,2＝0.18 d_1＝0.3 d_2＝0.82 d_3＝0.81 Mean = -0.055 Mean = 0.36 Mean = 0.325

[0153] Step 4.2: Remove training clusters with negative marginal gain from the training data to complete data cleaning.

[0154] Those skilled in the art can understand this embodiment as a more specific description of Embodiment 1 and Embodiment 2.

[0155] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0156] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A label data cleaning method based on failure sample contribution explanation, characterized in that, The method comprises the following steps: Step S1: dividing the fault sample data to form a plurality of fault sample clustering clusters; Step S2: using a simple model to perform preliminary screening of the fault sample clustering clusters to form a set of fine-screening fault sample clusters; Step S3: exhaustively listing all combination methods of the clusters in the set of fine-screening fault sample clusters, training a fault detection model, and obtaining an effect score of each model under various combinations; Step S4: based on the evaluation effect of each model, calculating the average marginal contribution value of each fine-screening sample cluster to all combinations of the fault detection model, and cleaning the data based on the average marginal contribution value. The step S1 comprises the following steps: Step S1.1: clustering the fault sample data; Step S1.2: re-dividing the sample clusters based on experience, setting the minimum number of samples of a sample cluster as min_cluster_size, and regarding a sample cluster with a sample quantity less than min_cluster_size as a sample cluster with few samples; The step S2 comprises the following steps: Step S2.1: using a simple model to perform one-leave-out fitting on all sample clusters one by one, and obtaining a gain score of each sample cluster; Step S2.2: obtaining a set of fine-screening sample clusters according to the gain score, sorting the gain scores of each cluster from low to high, selecting x sample clusters with the lowest scores, or selecting sample clusters with a gain score less than thre_lift, which is a floating-point number less than or equal to 1, to jointly constitute the set of fine-screening sample clusters; The step S2.1 comprises the following steps: Step S2.1.1: training a simple fault classification model using all fault sample clusters and non-fault sample data, and calculating the AUC value of the model, which is recorded as auc_all; Step S2.1.2: selecting a sample cluster c from the set of sample clusters to be screened without repetition, removing the sample cluster c from the set of sample clusters to be screened, and combining the data with non-fault sample data to obtain one-leave-out fitting training data of the sample cluster c; Step S2.1.3: training a simple fault classification model using the one-leave-out fitting training data of the sample cluster c, and then calculating the AUC value of the model, which is recorded as auc_remove_c; Step S2.1.4: calculating the gain score of the sample cluster c, which is recorded as lift_score_c: lift_score_c=auc_all-auc_remove_c; Step S2.1.5: repeating steps S2.1.2 to S2.1.4 to obtain the gain score of each cluster.

2. The label data cleaning method based on failure sample contribution explanation according to claim 1, characterized in that, The step S3 comprises the following steps: Step S3.1: generating all possible cluster combinations in the set of fine-screening sample clusters; Step S3.2: training a fault detection model for each combination of training clusters, and obtaining an evaluation score of the model.

3. The label data cleaning method based on failure sample contribution explanation according to claim 1, characterized in that, The step S4 comprises the following steps: Step S4.1: calculating the average marginal contribution of each cluster under various combinations of training models, which is recorded as d_i-j, wherein i and j are the numbers of training sets, calculating the marginal gain of each cluster under all scenarios, and calculating the mean of the marginal gain; Step S4.2: removing the training clusters with negative marginal gain mean from the training data to complete the data cleaning.

4. A label data cleaning system based on failure sample contribution explanation, characterized in that, The system comprises the following modules: Module M1: dividing the fault sample data to form a plurality of fault sample clustering clusters; Module M2: using a simple model to perform preliminary screening of the fault sample clustering clusters to form a set of fine-screening fault sample clusters; Module M3: training a fault detection model for all combinations of clusters in the set of fine-screening fault sample clusters to obtain an effect score of each model under various combinations; Module M4: calculating the average marginal contribution value of each fine-screening sample cluster to all combinations of fault detection models based on the evaluation effect of each model, and cleaning the data based on the average marginal contribution value; The module M1 comprises the following modules: Module M1.1: clustering the fault sample data; Module M1.2: re-dividing the sample clusters with few samples based on experience, setting the minimum number of samples of a sample cluster as min_cluster_size, and regarding a sample cluster with a sample quantity less than min_cluster_size as a sample cluster with few samples; The module M2 comprises the following modules: Module M2.1: using a simple model to perform one-by-one fitting on all sample clusters to obtain a gain score of each sample cluster; Module M2.2: obtaining a set of fine-screening sample clusters according to the gain score, sorting the gain scores of each cluster from low to high, selecting x sample clusters with the lowest scores, or selecting sample clusters with a gain score less than thre_lift, which is a floating point number less than or equal to 1, to jointly constitute the set of fine-screening sample clusters; The module M2.1 comprises the following modules: Module M2.1.1: training a simple fault classification model using all fault sample clusters and non-fault sample data, and calculating the AUC value of the model, which is recorded as auc_all; Module M2.1.2: selecting a sample cluster c from the set of sample clusters to be screened without repetition, removing the sample cluster c from the set of sample clusters to be screened, and combining the data with non-fault sample data to obtain one-by-one fitting training data of the sample cluster c; Module M2.1.3: training a simple fault classification model using the one-by-one fitting training data of the sample cluster c, and then calculating the AUC value of the model, which is recorded as auc_remove_c; Module M2.1.4: calculating the gain score of the sample cluster c, which is recorded as lift_score_c: lift_score_c=auc_all-auc_remove_c; Module M2.1.5: repeating modules M2.1.2 to M2.1.4 to obtain the gain score of each cluster.

5. The label data cleaning system based on failure sample contribution explanation of claim 4, wherein, The module M3 comprises the following modules: Module M3.1: generating all possible cluster combinations in the set of fine-screening sample clusters; Module M3.2: training a corresponding fault detection model for each combination of training clusters, and obtaining an evaluation score of the model.

6. The label data cleaning system based on failure sample contribution explanation of claim 4, wherein, The module M4 comprises the following modules: Module M4.1: Calculate the average marginal contribution of each cluster under various combinations of training models, denoted as d_i-j, where i, j are the numbers of training sets, calculate the marginal gain of each cluster under all scenarios, and calculate the mean of the marginal gain; Module M4.2: Remove the training clusters with negative mean marginal gain from the training data to complete data cleaning.

Images