One-way class label noise processing method, terminal and storage medium

Abstract

The present invention discloses a one-way class label noise processing method, terminal and storage medium, the method comprising: generating and maintaining multiple data clusters with class label distribution information based on a collected data set; calculating the class representative quantity of each data cluster, and determining the degree of the category represented by the corresponding data cluster based on the class representative quantity; evaluating the noise-free class label representative quantity of each data in the data set, and calibrating the noise-free class label representative quantity of each data in the data set based on the characteristics of the one-way class label noise; evaluating the credibility of the corresponding data class label based on the calibrated noise-free class label representative quantity, so as to guide model training or subsequent tasks of evaluating the quality of collected data. The present invention automatically aggregates and generates a series of data clusters based on the labeled data, and utilizes the data characteristics of the data clusters and the one-way class label noise to evaluate and obtain a credibility quantitative index of each data class label, thereby improving the prediction accuracy and robustness of the guidance model.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, terminal and storage medium for processing one-way class label noise. Background Art

[0002] Existing supervised classification learning based on artificial intelligence technology typically assumes that the class labels of training data are accurate and uncontaminated. Classification models are built and maintained based on the collected noise-free labeled data, thereby accurately classifying future unlabeled data. However, in actual production, due to uncertainty in the class label collection process (especially one-way uncertainty), the class labels of the collected labeled data are likely to be contaminated, that is, there is "one-way data noise." The source of data noise contamination is usually due to manual errors when manually collecting data class labels, or incorrect labeling caused by mechanical problems in the automatic class labeling mechanism, or both.

[0003] Specific application scenarios where unidirectional label noise exists include credit card application evaluation (pass vs. fail), credit fraud determination (fraud vs. non-fraud), autonomous driving (safe vs. unsafe), and software defect prediction (defective vs. non-defective). For example, for a piece of software change data that is labeled "defective" but has not yet caused a code defect, engineers typically construct training data labeled "non-defective" based on this data to build a classification model. Only long after the software defect caused by the code change is triggered do engineers realize that the relevant data was incorrectly labeled. On the other hand, for a code change where a defect has already been discovered, engineers can confidently label the corresponding data entry as "defective" without worrying about label noise.

[0004] When building classification models based on data with this type of (unidirectional) label noise, the accuracy and robustness of the model's predictions are significantly impacted. The larger the number of training samples containing label noise, the greater the negative impact on model performance. Furthermore, considering the problem of streaming data classification (i.e., online learning), since classification models continue to learn continuously after being deployed, the negative impact of this (unidirectional and bidirectional) label noise can continue to accumulate, resulting in even more profound adverse effects on model performance. Currently, neither academia nor industry has a specific solution to address this issue.

[0005] Therefore, the existing technology needs to be improved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that, in response to the defects of the existing technology, the present invention provides a one-way class label noise processing method, terminal and storage medium to solve the problem of low accuracy and low robustness of model prediction caused by one-way class label noise in the existing classification model.

[0007] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0008] In a first aspect, the present invention provides a method for processing one-way class-labeled noise, the method comprising the following steps:

[0009] Generate and maintain multiple data clusters with class label distribution information based on the collected data set; wherein each of the data clusters is a group of data with similar morphology in the input space;

[0010] Calculating the class representativeness of each of the data clusters, and determining the degree of the class represented by the corresponding data cluster according to the class representativeness;

[0011] Evaluating the noise-free class label representative quantity of each data in the data set, and calibrating the noise-free class label representative quantity of each data in the data set according to the characteristics of the unidirectional class label noise;

[0012] The credibility of the corresponding data class label is evaluated based on the representative quantity of the calibrated noise-free class label, which can be used to guide model training or subsequent tasks to assess the quality of collected data.

[0013] In one implementation, generating and maintaining multiple data clusters with class label distribution information based on the collected data includes:

[0014] Using a machine learning clustering algorithm to generate and maintain a plurality of said data clusters;

[0015] or generating and maintaining a plurality of said data clusters based on data analysis and expert knowledge;

[0016] The data cluster includes a fixed data cluster or a non-fixed data cluster.

[0017] In one implementation, calculating the class representativeness of each data cluster and determining the degree of the class represented by the corresponding data cluster according to the class representativeness includes:

[0018] Calculating the proportion of each category in each of the data clusters to obtain a first proportion;

[0019] Calculate the proportion of the corresponding category in the data set to obtain a second proportion;

[0020] The distance between the first ratio and the second ratio is calculated to obtain a quantitative index of the corresponding category in each of the data clusters.

[0021] In one implementation, the step of calculating the class representativeness of each data cluster and determining the degree of the class represented by the corresponding data cluster according to the class representativeness further includes:

[0022] Determining whether the data in the data set is streaming data;

[0023] If it is streaming data, the quantitative indicators of the corresponding categories in the corresponding data cluster are updated in real time according to the preset time interval.

[0024] In one implementation, evaluating the noise-free class label representative quantity of each data in the data set and calibrating the noise-free class label representative quantity of each data in the data set according to the characteristics of the unidirectional class label noise includes:

[0025] Calculating the contribution of each data cluster in the data set to each data class label based on data-driven or prior knowledge;

[0026] Calculate the noise-free class label representative quantity of each data in the data set according to the contribution;

[0027] According to the noise-free class label representative quantity of each data in the data set, data whose true class label is the noise-free class label is screened and the determined data is defined as source data;

[0028] The noise-free class label representative quantity of each data in the data set is adjusted according to the source data to obtain a calibrated noise-free class label representative quantity.

[0029] In one implementation, calculating the noise-free class label representative quantity of each data in the data set according to the contribution includes:

[0030] The contribution of each data cluster is combined with the corresponding class representative quantity, and the noise-free class representative quantity of each data in the data set is calculated:

[0031]

[0032] Wherein, rep is the representative quantity of the data cluster;

[0033] ω is the contribution of the data cluster to the data;

[0034] The superscript 1 is the noise-free class label, and 0 is the noise class label;

[0035] C j is the jth data cluster generated;

[0036] α (1) (X) is the representative value of the current data X for the noise-free class label.

[0037] In one implementation, adjusting the noise-free class label representative quantity of each data in the data set according to the source data to obtain a calibrated noise-free class label representative quantity includes:

[0038] Back propagation is performed using the residual between the calculated value and the true value of the noise-free class label representative quantity of the source data;

[0039] The noise-free class label representative quantity of each data in the data set is adjusted based on a preset gradient to obtain a calibrated noise-free class label representative quantity.

[0040] In one implementation, evaluating the credibility of the corresponding data class label based on the calibrated noise-free class label representative quantity includes:

[0041] According to the data class label and the noise-free class label representative quantity after calibration, the class label credibility of the corresponding data is calculated:

[0042]

[0043] Among them, y * is the class label of data X;

[0044] α (1) (X) + is the noise-free class standard after calibration;

[0045] cl(X,y * ) is the credibility of the class label of data X.

[0046] In a second aspect, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a one-way class-labeled noise processing program, and when the one-way class-labeled noise processing program is executed by the processor, it is used to implement the steps of the one-way class-labeled noise processing method as described in the first aspect.

[0047] In a third aspect, the present invention provides a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores a one-way class label noise processing program. When the one-way class label noise processing program is executed by a processor, it is used to implement the steps of the one-way class label noise processing method as described in the first aspect.

[0048] The present invention adopts the above technical solution to achieve the following effects:

[0049] The present invention automatically aggregates and generates a series of data clusters based on the labeled data, and uses the data characteristics of the data clusters and one-way class label noise to evaluate the credibility quantitative indicators of each data class label, thereby improving the prediction accuracy and robustness of the guidance model; and by fully exploring the one-way class label noise characteristics and using the source data of the data with accurate class label annotations, the class label evaluation is calibrated to obtain more accurate and practical data class label evaluation indicators; the present invention can condense a series of data statistical characteristics including data class label information without storing specific data, thereby supporting the maintenance and update of data clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0051] Figure 1 It is a flow chart of a one-way class label noise processing method in one implementation of the present invention.

[0052] Figure 2 It is an actual flow chart of a one-way class label noise processing method in one implementation of the present invention.

[0053] Figure 3 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0054] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] Exemplary Methods

[0057] Currently, there are methods for dealing with bidirectional label noise in academia and industry, but there are no methods for dealing with unidirectional label noise. In fact, most of the related solutions assume the "symmetry" of label noise, that is, the probability that positive data is incorrectly labeled as negative data is equal to the probability that negative data is incorrectly labeled as positive data. While the methods for dealing with bidirectional label noise utilize the bidirectional nature of label noise, they cannot be used to solve the problem of unidirectional label noise. For some methods for dealing with bidirectional label noise that do not make the above assumptions, since they do not utilize the specific data property of unidirectional label noise, directly applying this solution cannot effectively solve the problem of unidirectional label noise.

[0058] To address the above issues, this embodiment provides a one-way label noise processing method. To address the one-way label noise problem of labeled data faced in industrial practice, a series of data clusters are automatically generated based on the labeled data aggregation. Based on these data clusters and applying the data characteristic of one-way label noise, the credibility of each data label is evaluated to obtain a quantitative indicator of the credibility of the data label, thereby providing engineers with specific data usage guidance standards at the data application level.

[0059] like Figure 1 As shown, an embodiment of the present invention provides a method for processing one-way class-labeled noise, and the method for processing one-way class-labeled noise includes the following steps:

[0060] Step S100 : generating and maintaining a plurality of data clusters with class label distribution information based on the collected data set.

[0061] In this embodiment, the one-way class-labeled noise processing method is applied in a terminal, which includes but is not limited to data processing devices such as computers and mobile terminals.

[0062] In this embodiment, a one-way class label noise processing method is provided, and the one-way class label noise processing method is implemented by a class label evaluation model; in an application scenario of this embodiment, for the model training process, the class label evaluation model of this embodiment can give a class label confidence index for each data sample. When the class label confidence of the data reaches a certain value (for example, the class label confidence reaches 95%), it indicates that the class label is trustworthy, and thus the data can be relied upon and used more in the model training process; conversely, if the confidence obtained by the evaluation is low (for example, the class label confidence is lower than 10%), it is recommended to minimize the participation of this type of data in the model components during the subsequent model training process.

[0063] It is worth mentioning that the one-way class-labeled noise processing method of this embodiment is applicable not only to fixed data scenarios, but also to streaming data scenarios. Since streaming data is more consistent with the data acquisition method in actual industrial production, in streaming data scenarios, an appropriate data cluster generation and maintenance scheme can be selected based on actual storage capacity and computing power. For example, when storage space is limited and it is impossible to save all data, only basic data cluster information, such as statistical information such as mean and variance, can be stored, and this statistical information can be updated and maintained in real time after new data is obtained. When storage space is extremely large and computing costs are not a concern, all or most recent data information can be stored to construct data clusters.

[0064] In one implementation of this embodiment, step S100 includes the following steps:

[0065] Step S101: Generate and maintain a plurality of data clusters using a machine learning clustering algorithm.

[0066] In this embodiment, it is necessary to first collect a data set, and then generate and maintain multiple data clusters with class label distribution information based on the collected data set; when collecting the data set, the collected data set can be financial data (for example, data in financial investment fields such as medical care, education funds, sports events, etc.), or industrial data, etc., which is not limited in this embodiment.

[0067] In this embodiment, when generating and maintaining multiple data clusters, each data cluster is composed of a group of data with similar morphology in the input space (i.e., input feature space); specific examples of similar morphology include but are not limited to: data are similar in some features, for example, a subcategory of financial data.

[0068] Specifically, the number of data clusters generated depends on the data cluster generation method adopted. Depending on the generation method, it can be a fixed number or a non-fixed number; wherein, a fixed number of data clusters is referred to as a fixed data cluster, and a non-fixed number of data clusters is referred to as a non-fixed data cluster. The strategy of non-fixed data clusters has better applicability in industrial practice. It does not require model users (such as software developers or project engineers, etc.) to pre-set the hyperparameter of the number of data clusters, which reduces the engineer's need for prior knowledge of the model. The strategy of fixed number of data clusters is suitable for scenarios where engineers have good domain knowledge of the application background. Engineers can introduce domain knowledge into this embodiment, which may improve the performance of model evaluation. The specific number of data clusters can be selected accordingly according to the specific situation.

[0069] In an actual operation, the method for generating data clusters can adopt a machine learning clustering algorithm, wherein the machine learning clustering algorithm can be an offline clustering algorithm such as k-Means (i.e., an offline clustering algorithm based on mean) and DBSCAN (i.e., a spatial offline clustering algorithm based on density), or an online clustering algorithm such as DenStream and CluStream.

[0070] Offline clustering algorithms require storing all or part of the training data, which places a certain amount of storage space requirements on the network. Online clustering algorithms, on the other hand, do not require saving specific data; only statistical information needs to be saved and updated, resulting in lower storage space requirements. In industry practice, appropriate clustering algorithms can be selected based on specific circumstances to generate data clusters with class label distribution information. Specific machine learning clustering algorithms are not discussed here.

[0071] In one implementation of this embodiment, step S100 further includes the following steps:

[0072] Step S102 : generating and maintaining a plurality of data clusters based on data analysis and expert knowledge.

[0073] In this embodiment, the data cluster generation method can also be generated and maintained by data analysis or expert knowledge; wherein, the data analysis method is to analyze the collected data set based on historical data and / or real-time data, and generate data clusters with class label distribution information in the form of empirical data; and the expert knowledge method is to use manually input prior knowledge (class label information) as professional information to generate data clusters with class label distribution information.

[0074] In another implementation of this embodiment, the clustering algorithm generation method using machine learning in step S101 and the data analysis and expert knowledge generation method in step S102 can be combined or improved to obtain multiple data clusters; for example: automatically aggregating a series of data clusters based on labeled data, and / or artificially synthesizing a series of data clusters based on expert knowledge.

[0075] In one application scenario, the collected data set contains a large amount of financial data, including financial investment areas such as medical care, education funds, and sports events. These financial data can be divided according to their categories to form data clusters with category distribution information, such as medical care data clusters, education fund data clusters, and sports event data clusters.

[0076] It's worth noting that for all generated data clusters, each needs to store information about the distribution of class labels. Because the generated data clusters are grouped based on similar patterns, data assigned to the same cluster may not necessarily have the same class labels. For example, a piece of data in a healthcare cluster might be classified as "fraudulent information," while another piece of data in the same cluster might be classified as "non-fraudulent information." Class label information might include information such as the number of positive and negative data in a cluster.

[0077] In one implementation of this embodiment, when the data in the processed data set is streaming data, the constructed data cluster also needs to be updated in real time based on the continuously received data, so as to better reflect the latest data characteristics.

[0078] Furthermore, when processing streaming data, the data clusters constructed and maintained in this embodiment can extract a series of data statistical characteristics, including data classifier information, without storing specific data, thereby supporting data cluster maintenance and updates. The extracted specific information that can be used to describe the data cluster may include classifier distribution statistics, geometric descriptions of the data feature space, and real-time data indicators.

[0079] This embodiment can obtain morphologically similar data clusters by clustering the data of the data set, so as to calculate the corresponding class representativeness according to each data cluster, thereby determining the degree of the category represented by the corresponding data cluster.

[0080] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the one-way class label noise processing method further includes the following steps:

[0081] Step S200 , calculating the class representativeness of each data cluster, and determining the degree of the class represented by the corresponding data cluster according to the class representativeness.

[0082] In this embodiment, after generating multiple different data clusters, it is necessary to calculate the class representativeness of each data cluster to determine the degree to which the category represented by the corresponding data cluster is determined based on the class representativeness; for binary classification problems (for example, positive and negative classes) with unidirectional class label noise, the representativeness corresponding to the positive class and the representativeness corresponding to the negative class of each data cluster can be calculated to measure the degree to which the data cluster represents the positive class and the negative class respectively according to different representativeness.

[0083] In one implementation of this embodiment, it can be assumed that unidirectional class labeling noise exists in the negative class, that is, the data labeled as the negative class may be contaminated data (the true class label of this data is the positive class); in this case, the system's estimated value of the statistic for the positive class may be less than its true value, and correspondingly, the system's estimated value of the statistic for the negative class may be greater than its true value. In this case, the sum of the representative values ​​of each class in each data cluster does not necessarily meet the normalization principle, that is, the sum of the representative values ​​of each class in the data cluster may be greater than 1 or less than 1; for this case, it is necessary to determine the degree to which each class is represented in each data cluster.

[0084] In one implementation of this embodiment, step S200 includes the following steps:

[0085] Step S201, calculating the proportion of each category in each data cluster to obtain a first proportion;

[0086] Step S202, calculating the proportion of the corresponding category in the data set to obtain a second proportion;

[0087] Step S203 : calculating the distance between the first ratio and the second ratio to obtain a quantitative index of the corresponding category in each of the data clusters.

[0088] In this embodiment, when calculating the class representative quantity of each data cluster, the class representative quantity of each data cluster may be calculated in a data-driven manner or in a statistical manner.

[0089] In this embodiment, calculations are mainly performed in a data-driven manner. Data-driven means collecting data and organizing it into information flows. When making decisions or optimizing products and operations, the information flows are refined and summarized according to different needs, so that scientific actions are taken under the support or guidance of data. This is called data-driven.

[0090] Specifically, when performing calculations in a data-driven manner, the following steps may be included:

[0091] First, the proportion of each category in each data cluster is calculated to obtain a first proportion; during the calculation process, statistics are performed based on the labeling information stored in each data cluster to obtain the proportion of the positive class in each data cluster, and the proportion of the negative class in each data cluster is obtained; that is, the first proportion can be the proportion of the positive class in each data cluster, or the proportion of the negative class in each data cluster.

[0092] Secondly, determine the proportion of the category corresponding to the first proportion in the overall data (i.e., the data set) to obtain the second proportion; that is, calculate the proportion of the positive class in the entire data set, and obtain the proportion of the negative class in the entire data set.

[0093] Finally, by calculating the distance between the two ratios, we can obtain a quantitative indicator of the representativeness of the data cluster to the class.

[0094] In one implementation of this embodiment, if the distribution of positive data in a data cluster is 0.6, and the amount of positive data in all data clusters is 0.5, then the positive representative value of the data cluster should be a value greater than 0.6 (the ratio of the data cluster in the positive class is greater than the ratio of the positive class in the overall data, so the representative value of the data cluster for the positive class will be adjusted upward based on 0.6).

[0095] It is worth mentioning that in the actual process of calculating class representativeness, there can be many calculation methods, and the above calculation is just an example.

[0096] In industry practice, if engineers or project managers have relevant domain knowledge, they can also set representative values ​​for corresponding categories based on that knowledge. For example, software developers know in advance that the software code developed during this period has undergone multiple rounds of testing, modification, and integration by the testing and development teams, and that the code quality can be fully guaranteed. Therefore, for the "data cluster" formed by the data generated by the code changes during this period, the representative value for the "no defects" category can be set to a very large value (such as 1.2), while the representative value for the "defective" category can be set to a very small value (such as 0.1). The specific values ​​need to be set based on the relative values ​​of the data volumes of the two categories in all data clusters.

[0097] In one implementation of this embodiment, step S200 further includes the following steps:

[0098] Step S204, determining whether the data in the data set is streaming data;

[0099] Step S205: If the data is streaming data, the quantitative index of the corresponding category in the corresponding data cluster is updated in real time according to a preset time interval.

[0100] In this embodiment, different data update processing processes are performed according to different data types; if all data in the data set are streaming data, it is necessary to update the quantitative indicators of the corresponding categories in the corresponding data cluster according to the data update frequency of the streaming data; if all data in the data set are non-streaming data, no subsequent update processing is performed.

[0101] Specifically, we can first determine whether the data in the dataset is streaming data based on the time point at which each data is acquired. If it is streaming data, the calculation of the class representativeness of each data cluster needs to be adjusted and updated according to time to better describe the latest data class label distribution. The storage information required for streaming data processing should be based on the actual storage devices and storage capacity in industry practice. When the storage capacity of the device is limited, only the data of the most recent period (for example, one day) can be stored to calculate the representativeness of each data cluster in each category at the current moment. When there is no need to worry about the storage capacity of the device, engineers can choose to record the original values ​​of all data to improve the calculation performance of the model for quantitative indicators.

[0102] In this embodiment, by calculating the quantitative index of each category in each data cluster, the representativeness of each data in each data cluster on the noise-free class label can be evaluated based on the quantitative index, thereby accurately evaluating the class label of each data based on the characteristics of the one-way class label noise.

[0103] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the one-way class label noise processing method further includes the following steps:

[0104] Step S300 : evaluating the noise-free class label representative quantity of each data in the data set, and calibrating the noise-free class label representative quantity of each data in the data set according to the characteristics of unidirectional class label noise.

[0105] In this embodiment, after calculating and updating the class representative quantity in each data cluster, it is also necessary to evaluate the noise-free class label representative quantity of each data in the corresponding data cluster based on the class representative quantity; in this way, the noise-free class label representative quantity of each data in the data set is evaluated, and then the representative quantity of each data on the noise-free class label is calibrated based on the characteristics of the unidirectional class label noise.

[0106] Specifically, in the context of unidirectional label noise, a noise-free label refers to a category without label noise. That is, when data is labeled with this category, it is definitely the true category of the data. For example, in software defect prediction, if a piece of software change data is labeled as defective, the labeling is considered accurate and not mislabeled, because the defect has been discovered and corrected in subsequent code (at this point, any errors that may have occurred during the process of linking the two code changes are not considered).

[0107] In one implementation of this embodiment, step S300 includes the following steps:

[0108] Step S301 : Calculate the contribution of each data cluster in the data set to each data class label based on data drive or prior knowledge.

[0109] In this embodiment, when evaluating the representativeness of each data in the data set on the noise-free class label, the contribution of each data cluster in determining the credibility of the data class label may be calculated first.

[0110] Specifically, the contribution of a data cluster to the credibility measure of the data class label can be automatically calculated based on a data-driven approach, or it can be proactively designed based on the domain knowledge (i.e., prior knowledge) of industry developers. Using a data-driven approach, the geometric distance between the data point and each data cluster can be calculated. The farther the data cluster is from the data, the lower its contribution to the calculation of the representative quantity of the data class label, and vice versa. Based on domain knowledge, experts can pre-determine whether certain data clusters have greater or lesser contributions. This value can be set between 0 and 1 to facilitate subsequent calculations.

[0111] Step S302: Calculate the noise-free class label representative quantity of each data in the data set according to the contribution.

[0112] In this embodiment, after the contribution is calculated, the representativeness of the data on the noise-free class label is automatically calculated based on the contribution of all data clusters to the data point and the class representativeness of each data cluster obtained in step S200.

[0113] In one implementation of this embodiment, step S302 includes the following steps:

[0114] Step S302a: combining the contribution of each data cluster with the corresponding class representative quantity, and calculating the noise-free class representative quantity of each data in the data set.

[0115] In this embodiment, the specific calculation formula can refer to the following standards:

[0116]

[0117] Wherein, rep is the representative quantity of the data cluster;

[0118] ω is the contribution of the data cluster to the data;

[0119] The superscript 1 is the noise-free class label, and 0 is the noise class label;

[0120] C j is the jth data cluster generated;

[0121] α (1) (X) is the representative value of the current data X for the noise-free class label.

[0122] It's worth noting that the above formula is just an example. Engineers can customize their own formulas based on specific industrial practices to automatically measure the representativeness of each data point for the noise-free class. The provided metric only needs to be between 0 and 1. This requirement is primarily to ensure the physical meaning of the data and facilitate subsequent steps. A maximum value of 1 indicates that the model believes the data point is very likely to represent the noise-free class, while a minimum value of 0 indicates that the data point is least likely to represent the noise-free class.

[0123] For all the data of each data cluster in the data set, the above steps S301 to S302 are repeatedly performed to obtain a set of representative quantities of noise-free class labels, each representative quantity corresponding to a piece of data.

[0124] In one implementation of this embodiment, when the processed data is streaming data, information similar to a "timestamp" can be added to each data cluster to measure the degree to which each data cluster represents the latest data status, and this information can be incorporated into step S301 to calculate the contribution of each data cluster in the data set to each data class label.

[0125] The calculation principle based on timestamps is: for two data clusters at the same distance from a data point, the data cluster with a closer timestamp can be given a greater contribution; for example, determine all data clusters at the same distance from a data point, sort them according to the distance of their timestamps, and assign contributions from small to large according to the order of arrangement.

[0126] In one implementation of this embodiment, step S300 further includes the following steps:

[0127] Step S303 , based on the representative amount of the noise-free class label of each data in the data set, filter out data whose true class label is the noise-free class label, and define the determined data as source data.

[0128] In this embodiment, after obtaining the representative amount of the noise-free class label for each data, the representative amount of all data on the noise-free class label can be measured, and the representative amount is a representative amount with a value between 0 and 1; among them, the true class label of some data is the "noise-free class label", which can be called "source data"; by comparing and screening the representative amount of the noise-free class label of each data, the data whose true class label is the noise-free class label can be obtained, and the data is defined as the source data.

[0129] Step S304 : adjusting the noise-free class label representative quantity of each data in the data set according to the source data to obtain a calibrated noise-free class label representative quantity.

[0130] Theoretically, the noise-free class-label representative quantity of the source data should be close to a maximum value of 1. However, in practice, the estimated representative quantity of the noise-free class-label of the source data obtained through the derivation process of steps S301 to S302 may not meet this theoretical value. Therefore, it is necessary to compare the actual (calculated) value of the noise-free class-label representative quantity of the source data with the theoretical value to determine the error between the two. This error can then be used to "calibrate" the noise-free class-label representative quantity of all data.

[0131] In this embodiment, the criterion for the noise-free class label representative quantity of all data is: requiring the noise-free class label representative quantity of the source data to be the maximum value.

[0132] In one implementation of this embodiment, step S304 includes the following steps:

[0133] Step S304a, performing back propagation using the residual between the calculated value and the true value of the noise-free class label representative quantity of the source data;

[0134] Step S304b: adjusting the noise-free class label representative quantity of each data in the data set based on a preset gradient to obtain a calibrated noise-free class label representative quantity.

[0135] In this embodiment, in the process of calibrating the noise-free class label representative quantities of all data, the calibration method includes but is not limited to: using the residual between the theoretical value and the true value of the noise-free class label representative quantity of the source data to "back propagate" the error, and adjusting the noise-free class label representative quantity of each data based on the gradient.

[0136] In another implementation of this embodiment, a feasible solution is: modeling the mapping relationship between the theoretical value and the true value of the source data (i.e., constructing a loss function). For example, assuming that the mapping is a linear function, and obtaining the specific expression of the mapping function based on data fitting, the representative value of the non-source data is adjusted accordingly (i.e., back-propagation training is performed through the constructed loss function to calibrate the output data).

[0137] In the process of industry practice, engineers can choose a calibration method that meets the conditions of the data processing equipment based on actual conditions.

[0138] This embodiment uses an active data classification evaluation model to fully exploit the "one-way classification noise" characteristics, and uses source data with accurate classification labels to "calibrate" the classification evaluation, thereby obtaining more accurate and practical data classification evaluation indicators.

[0139] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the one-way class label noise processing method further includes the following steps:

[0140] Step S400 : The credibility of the corresponding data class label is evaluated based on the representative quantity of the calibrated noise-free class label, so as to be used for subsequent tasks such as guiding model training or evaluating the quality of collected data.

[0141] In this embodiment, based on the noise-free class label representative quantity of the calibrated data, the class label credibility of each data point can be simply calculated and evaluated, and the evaluated data can be used to guide model training or evaluate the quality of collected data and other subsequent tasks to avoid the problem of low accuracy and low robustness of model predictions due to one-way class label noise.

[0142] In one implementation of this embodiment, step S400 includes the following steps:

[0143] Step S401 : Calculate the reliability of the class label of the corresponding data based on the class label of the data and the calibrated noise-free class label representative quantity.

[0144] In this embodiment, based on the noise-free class label representative quantity of the calibrated data, the reliability of the class label of each data point can be simply calculated and evaluated. The measurement method is as follows:

[0145]

[0146] Among them, y * is the class label of data X;

[0147] α (1) (X) + is the noise-free class standard after calibration;

[0148] cl(X,y * ) is the credibility of the class label of data X.

[0149] Academia and industry currently lack methods for addressing unidirectional data label noise, yet this data problem is often encountered in real-world production. Methods for addressing bidirectional symmetric label noise are either not directly applicable to unidirectional label noise or, due to the lack of mechanisms for addressing unidirectional label noise, fail to achieve satisfactory results. This embodiment evaluates the labeling of data with unidirectional label noise. The resulting data can be used to guide model training or assess the quality of collected data, among other subsequent tasks.

[0150] The following combination Figure 2 , further illustrate the technical solution of this embodiment:

[0151] like Figure 2 As shown, in actual application, the one-way classification noise processing method includes the following steps:

[0152] Step S11, inputting a data set;

[0153] Step S12, constructing a data cluster with class label quantity information;

[0154] Step S13, calculating the representativeness of each data cluster for each type of target;

[0155] Step S14, selecting data points;

[0156] Step S15, calculating the representativeness of the data to the noise-free class label;

[0157] Step S16, determine whether there is unevaluated data; if yes, return to step S14; if no, execute step S17;

[0158] Step S17, calibrating the representative quantity according to the one-way class mark noise characteristics;

[0159] Step S18, evaluating the credibility of the classification label of each data;

[0160] Step S19, determining whether a new data set is received; if yes, returning to step S11; if no, executing step S20.

[0161] Step S20: Output the credibility evaluation of each data class label.

[0162] It's worth noting that steps S11 to S20 above are merely a practical application of this embodiment for a binary classification problem. However, this solution can be extended to multi-classification problems, specifically for applications with more than two class labels and in situations where there is some "unidirectional class label noise." A specific extension strategy might be to decompose the multi-classification problem into multiple binary classification problems with unidirectional class label noise, process them separately, and then perform fusion.

[0163] This embodiment achieves the following technical effects through the above technical solution:

[0164] This embodiment automatically aggregates and generates a series of data clusters based on the labeled data, and uses the data characteristics of the data clusters and one-way class label noise to evaluate and obtain the credibility quantitative indicators of each data class label, thereby improving the prediction accuracy and robustness of the guidance model; and by fully exploring the one-way class label noise characteristics and using the source data of the data with accurate class label annotations, the class label evaluation is calibrated to obtain more accurate and practical data class label evaluation indicators; this embodiment can condense a series of data statistical characteristics including data class label information without storing specific data, thereby supporting the maintenance and update of data clusters.

[0165] Exemplary devices

[0166] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 3 shown.

[0167] The terminal includes: a processor, memory, interface, display screen and communication module connected via a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external terminal devices, such as mobile terminals and computers; the display screen is used to display corresponding one-way class-labeled noise processing information; and the communication module is used to communicate with a cloud server or a mobile terminal.

[0168] When the computer program is executed by a processor, it is used to implement the steps of a one-way class-labeled noise processing method.

[0169] It will be understood by those skilled in the art that Figure 3 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0170] In one embodiment, a terminal is provided, comprising: a processor and a memory, wherein the memory stores a one-way classifier noise processing program, and when the one-way classifier noise processing program is executed by the processor, it is used to implement the steps of the one-way classifier noise processing method described above.

[0171] In one embodiment, a storage medium is provided, wherein the storage medium is a computer-readable storage medium, and the storage medium stores a one-way class label noise processing program, which, when executed by a processor, is used to implement the steps of the one-way class label noise processing method described above.

[0172] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other storage medium used in the various embodiments provided herein may include non-volatile and / or volatile memory.

[0173] In summary, the present invention provides a one-way class label noise processing method, terminal and storage medium, the method comprising: generating and maintaining multiple data clusters with class label distribution information based on the collected data set; calculating the class representative quantity of each data cluster, and determining the degree of the category represented by the corresponding data cluster based on the class representative quantity; evaluating the noise-free class label representative quantity of each data in the data set, and calibrating the noise-free class label representative quantity of each data in the data set based on the characteristics of the one-way class label noise; evaluating the credibility of the corresponding data class label based on the calibrated noise-free class label representative quantity, so as to guide model training or subsequent tasks of evaluating the quality of collected data. The present invention automatically aggregates and generates a series of data clusters based on the labeled data, and utilizes the data characteristics of the data clusters and the one-way class label noise to evaluate and obtain a quantitative indicator of the credibility of each data class label, thereby improving the prediction accuracy and robustness of the guidance model.

[0174] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for processing one-way class label noise, characterized in that: The one-way class mark noise processing method includes: Generate and maintain multiple data clusters with class label distribution information based on the collected data set; wherein each of the data clusters is a group of data with similar morphology in the input space; Calculating the class representativeness of each of the data clusters, and determining the degree of the class represented by the corresponding data cluster according to the class representativeness; Evaluating the noise-free class label representative quantity of each data in the data set, and calibrating the noise-free class label representative quantity of each data in the data set according to the characteristics of the unidirectional class label noise; The credibility of the corresponding data class labels is evaluated based on the representative quantity of the calibrated noise-free class labels, which can be used to guide model training or subsequent tasks to assess the quality of collected data; The evaluating the noise-free class label representative quantity of each data in the data set, and calibrating the noise-free class label representative quantity of each data in the data set according to the characteristics of the unidirectional class label noise, includes: Calculating the contribution of each data cluster in the data set to each data class label based on data-driven or prior knowledge; Calculate the noise-free class label representative quantity of each data in the data set according to the contribution; According to the noise-free class label representative quantity of each data in the data set, data whose true class label is the noise-free class label is screened and the determined data is defined as source data; Adjusting the noise-free class label representative quantity of each data in the data set according to the source data to obtain a calibrated noise-free class label representative quantity; Calculating the noise-free class label representative quantity of each data in the data set according to the contribution degree includes: The contribution of each data cluster is combined with the corresponding class representative quantity, and the noise-free class representative quantity of each data in the data set is calculated: ； in, is the representative quantity of the data cluster; is the contribution of the data cluster to the data; The superscript 1 is the noise-free class label, and 0 is the noise class label; For the generated data clusters; For current data Representative quantity for noise-free class; The step of adjusting the noise-free class label representative quantity of each data in the data set according to the source data to obtain a calibrated noise-free class label representative quantity includes: Back propagation is performed using the residual between the calculated value and the true value of the noise-free class label representative quantity of the source data; The noise-free class label representative quantity of each data in the data set is adjusted based on a preset gradient to obtain a calibrated noise-free class label representative quantity.

2. The method for processing one-way classifier noise according to claim 1, characterized in that: The method of generating and maintaining multiple data clusters with class label distribution information based on the collected data includes: Using a machine learning clustering algorithm to generate and maintain a plurality of said data clusters; or generating and maintaining a plurality of said data clusters based on data analysis and expert knowledge; The data cluster includes a fixed data cluster or a non-fixed data cluster.

3. The method for processing one-way classifier noise according to claim 1, characterized in that: The calculating of the class representativeness of each data cluster and determining the degree of the class represented by the corresponding data cluster according to the class representativeness includes: Calculating the proportion of each category in each of the data clusters to obtain a first proportion; Calculate the proportion of the corresponding category in the data set to obtain a second proportion; The distance between the first ratio and the second ratio is calculated to obtain a quantitative index of the corresponding category in each of the data clusters.

4. The method for processing one-way class label noise according to claim 1, characterized in that: The step of calculating the class representativeness of each data cluster and determining the degree of the class represented by the corresponding data cluster according to the class representativeness further includes: Determining whether the data in the data set is streaming data; If it is streaming data, the quantitative indicators of the corresponding categories in the corresponding data cluster are updated in real time according to the preset time interval.

5. The method for processing one-way class label noise according to claim 1, characterized in that: The step of evaluating the credibility of the corresponding data class label based on the calibrated noise-free class label representative quantity includes: According to the data class label and the noise-free class label representative quantity after calibration, the class label credibility of the corresponding data is calculated: ； in, For data Class mark; is the noise-free class standard after calibration; For data The credibility of the class label.

6. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a one-way class label noise processing program, and when the one-way class label noise processing program is executed by the processor, it is used to implement the steps of the one-way class label noise processing method according to any one of claims 1 to 5.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and the storage medium stores a one-way class mark noise processing program. When the one-way class mark noise processing program is executed by a processor, it is used to implement the steps of the one-way class mark noise processing method according to any one of claims 1 to 5.

Images