Method for eliminating influence of improper labeled samples, classification method, device and medium

Abstract

The application discloses a method for eliminating the influence of improper labeled samples, a classification method, equipment and a medium. The method comprises the following steps: obtaining the predicted categories of all samples through a prediction model, wherein the samples have labeled categories; distinguishing all samples into a correct labeled set, an uncertain labeled set and an error labeled set according to the similarities and differences between the predicted categories and the labeled categories; and giving different weights to the correct labeled set, the uncertain labeled set and the error labeled set when calculating a loss function, so as to eliminate the influence of samples in the uncertain labeled set and the error labeled set. The weight of the samples in the error labeled set is not more than the weight of the samples in the uncertain labeled set, and the weight of the samples in the uncertain labeled set is not more than the weight of the samples in the correct labeled set. The application does not need to make any prior assumption on the labeled noise, thereby fundamentally avoiding the deviation caused by impractical assumptions. After the samples are processed by the application, the classification accuracy is obviously improved.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a method, classification method, device, and medium for eliminating the influence of inappropriately labeled samples. Background Technology

[0002] With the development of artificial intelligence technology, models are becoming increasingly larger, leading to a greater demand for data, including labeled data. However, due to various subjective and objective reasons, data labeling is not always accurate, and incorrect labeling can negatively impact the model. Therefore, research has been conducted on identifying mislabeled samples. Relevant materials include Chinese patents CN202010327799.0, CN202210417059.5, CN201910133229.5, CN202110152413.1, Buyu Li, Yu Liu, and Xiaogang Wang's 2019 publication at AAAI, "Gradient Harmonized Single-stage Detector," and Curtis G. Northcutt, Lu Jiang, and Isaac L. Chuang's 2021 publication in the Journal of Artificial Intelligence Research, "Confident Learning: Estimating Uncertainty in Dataset Labels."

[0003] However, these methods are either based on specific models, such as the deep convolutional networks in Chinese patent CN202010327799.0; or computationally complex, such as the iterations in Chinese patent CN 202110152413.1; or involve numerous steps, such as the literature "Confident Learning: Estimating Uncertainty in Dataset Labels". Furthermore, these related patent documents simply categorize labeled data into correctly labeled data and incorrectly labeled data. However, accurately determining the correctness of a label based solely on the data itself is difficult. In other words, most labels can be statistically determined to be correct or incorrect, while the remaining portion is difficult to categorize as either correct or incorrect; these are called uncertain labels. To identify these two types of inappropriate labels—incorrect and uncertain labels—and eliminate their impact on the classification model's performance, this invention proposes a method to eliminate the influence of inappropriately labeled samples. Summary of the Invention

[0004] This invention provides a method, classification method, device, and medium for eliminating the influence of improperly labeled samples, thereby solving the technical problems in the prior art.

[0005] The technical solution adopted in this invention is:

[0006] In a first aspect, the present invention provides a method for eliminating the influence of improperly labeled samples, comprising:

[0007] The predicted categories of all samples are obtained through a prediction model, and the samples have labeled categories;

[0008] Based on the similarities and differences between the predicted and labeled categories, all samples are classified as the correct labeled set I. R Uncertain annotation set I W and error annotation set I E ;

[0009] When calculating the loss function, different weights are assigned to the correct label set, uncertain label set, and incorrect label set to eliminate the influence of samples in the uncertain label set and incorrect label set; the weight of samples in the incorrect label set does not exceed the weight of samples in the uncertain label set, and the weight of samples in the uncertain label set does not exceed the weight of samples in the correct label set.

[0010] Furthermore, the method for obtaining the predicted category of all samples through the prediction model includes:

[0011] The prediction model uses cross-validation to obtain the predicted class probability of each sample i for all samples. The class with the highest predicted probability value is the predicted class of that sample.

[0012] Furthermore, the method of obtaining the class prediction probability of each sample i by using cross-validation on all samples through the prediction model, and the class with the highest prediction probability value being the predicted class of that sample, includes:

[0013] All data is randomly divided into S mutually exclusive subsets of equal or similar size. These S subsets are iterated through sequentially, with each subset used as the validation set and the remaining subsets used as the training set. The prediction model then predicts the probability P of each sample in the validation set belonging to each class. n (i), where n represents the category number; the predicted category of sample i is the predicted probability value P. n (i) The category corresponding to the maximum value.

[0014] Furthermore, based on the similarities and differences between the predicted category and the labeled category, all samples are classified as the correct labeled set I. R Uncertain annotation set I W and error annotation set I E The methods include:

[0015] Calculate the average predicted probability of samples in each labeled category. Among them I n Let |I| represent the set of samples of class n. n | represents the number of samples in the nth class of samples;

[0016] If the predicted category of sample i is the same as the labeled category, then sample i belongs to the correctly labeled set I. R ;

[0017] If the predicted category of sample i is different from the labeled category, then sample i belongs to the inappropriate label set I. W ;

[0018] For I W Any sample i in W If P M (i W )≥C M , then i W Belongs to error annotation set I E Where M is I W The predicted category; if P M (i W ) < C M , then i W Belongs to the uncertain annotation set I U ;

[0019] Based on the above results, all samples are assigned to the correctly labeled set I. R Uncertain annotation set I U Or error annotation set I E middle.

[0020] Furthermore, the weight of the correct annotation set is 1, 0 < the weight of the uncertain annotation set ≤ 1, and the weight of the incorrect annotation set is 0.

[0021] Furthermore, the weight of uncertain labels is 0.6 ≤ 0.9.

[0022] Secondly, the present invention also provides a classification method, comprising:

[0023] After obtaining samples with labeled categories, the samples are processed using the method described in the first aspect for eliminating the influence of inappropriately labeled samples;

[0024] The processed samples are used as the training set to train the classification model;

[0025] The trained classification model is used to classify the samples that need to be classified.

[0026] Thirdly, the present invention also provides an apparatus for eliminating the influence of improperly labeled samples, comprising:

[0027] A prediction module is used to obtain the predicted category of all samples, which have labeled categories;

[0028] The discrimination module is used to classify all samples as the correct labeled set I based on the similarities and differences between the predicted category and the labeled category. R Uncertain annotation set I W and error annotation set I E ;

[0029] The weighting module is used to assign different weights to the correct label set, uncertain label set, and incorrect label set when calculating the loss function, and the weight of the samples in the incorrect label set does not exceed the weight of the samples in the uncertain label set, and the weight of the samples in the uncertain label set does not exceed the weight of the samples in the correct label set.

[0030] Fourthly, the present invention also provides an electronic device, the electronic device comprising:

[0031] One or more processors;

[0032] Storage device for storing one or more programs.

[0033] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for eliminating the influence of inappropriately labeled samples as described in the first aspect or the classification method as described in the second aspect.

[0034] Fifthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for eliminating the influence of inappropriately labeled samples as described in the first aspect or the classification method as described in the second aspect.

[0035] The beneficial effects of this invention are: it provides, for the first time, a feasible method for identifying and processing uncertain labeled samples; simultaneously, it does not require any prior assumptions about labeled noise, thus fundamentally avoiding bias caused by unrealistic assumptions; furthermore, it is model-independent and can use any classification model. Comparative experiments on publicly available datasets demonstrate the high classification accuracy of this invention. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the method for eliminating the influence of inappropriately labeled samples disclosed in this invention.

[0037] Figure 2 This is a structural diagram of the convolutional neural network disclosed in this invention;

[0038] Figure 3This is an example diagram of correctly labeled samples (first row), uncertainly labeled samples (second row), and incorrectly labeled samples (third row) in an embodiment of the present invention, wherein the labels of the first to sixth columns are 7, 3, 7, 4, 9, and 3, respectively;

[0039] Figure 4 This is a schematic diagram of the structure of the electronic device disclosed in this invention. Detailed Implementation

[0040] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0041] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0042] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0043] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0044] Example 1:

[0045] Figure 1 This invention discloses a method for eliminating the influence of inappropriately labeled samples, which includes the following steps:

[0046] S1: Obtain the predicted category of all samples through the prediction model, where the samples have labeled categories;

[0047] S2: Based on the similarities and differences between the predicted category and the labeled category, classify all samples as the correct labeled set I. R Uncertain annotation set I W and error annotation set I E ;

[0048] S3: When calculating the loss function, different weights are assigned to the correct label set, uncertain label set, and incorrect label set, and the weight of the samples in the incorrect label set does not exceed the weight of the samples in the uncertain label set, and the weight of the samples in the uncertain label set does not exceed the weight of the samples in the correct label set.

[0049] In some implementations, the method for obtaining the predicted class of all samples in S1 through the prediction model can be as follows: The prediction model uses cross-validation on all samples to obtain the predicted class probability of each sample i; the class with the highest predicted probability value is the predicted class of that sample. Specifically, all data is randomly divided into S mutually exclusive sample subsets of equal or similar size, and these S subsets are iterated sequentially. Each time, the current subset is used as the validation set, and all remaining sample subsets are used as the training set. The prediction model predicts the probability P of each sample in the validation set belonging to each class. n (i), where n represents the category number; the predicted category of sample i is the predicted probability value P. n (i) The category corresponding to the maximum. A mutually exclusive subset of samples is defined as a subset whose intersection is an empty set and whose union contains all samples. The prediction model can be any classification model.

[0050] In some implementations, in S2, all samples are classified as the correct labeled set I based on the similarities or differences between the predicted category and the labeled category. R Uncertain annotation set I W and error annotation set I E The methods include:

[0051] S2.1: Calculate the average predicted probability of samples in each labeled category. Among them I n Let |I| represent the set of samples of class n. n | represents the number of samples in the nth class of samples;

[0052] S2.2: If the predicted category of sample i is the same as the labeled category, then sample i belongs to the correct label set I. R ;

[0053] S2.3: If the predicted category of sample i is different from the labeled category, then sample i belongs to the inappropriate label set I. W ;

[0054] S2.4: For I W Any sample i inW If P M (i W )≥C M Then P M (i W ) is i W A consistent estimate of the true probability value is obtained, in which case the predicted class is the true class. However, by the definition of an inappropriate labeled set, the predicted class, i.e., the true class, differs from the labeled class. Therefore, i... W Belongs to error annotation set I E Where M is I W The predicted category; if P M (i W ) < C M P M (i W It's not necessarily i W A consistent estimate of the true probability value is needed, but the predicted class may not be the true class, therefore i W Belongs to the uncertain annotation set I U ;

[0055] S2.5: Based on the above results, all samples are assigned to the correctly labeled set I. R Uncertain annotation set I U Or error annotation set I E middle.

[0056] In some implementations, in S3, the weight of the correctly labeled set is preferably 1, and 0 < the weight of uncertain labels ≤ 1; the weight of the incorrectly labeled set is preferably 0, i.e., the incorrectly labeled set is directly excluded. Experiments have shown that when 0.6 ≤ the weight of uncertain labels ≤ 0.9, the prediction results are better, while values ​​below 0.4 result in slightly worse performance. Specifically, the loss function mentioned here can be cross-entropy loss or various classification variant losses, such as Focal Loss or Gradient Harmonizing Mechanism-Classification Loss.

[0057] It should be noted that the above-mentioned samples can be any kind of sample, including but not limited to image samples, text samples, sound samples, radar trajectory samples, tissue samples, blood samples, saliva samples, and urine samples.

[0058] The method of the present invention will be described below with reference to specific application scenarios.

[0059] The publicly available dataset MNIST was chosen to validate this invention. Samples corresponding to numbers 3, 4, 7, and 9 in the dataset were used. All samples are grayscale images with a resolution of 28*28 pixels, including 24,187 training samples and 4,029 test samples.

[0060] This invention constructs a convolutional neural network for classification, the network structure of which is as follows: Figure 2 As shown, cross-entropy loss is used. Four-fold cross-validation is applied to the training set samples to predict the probability P of each handwritten digit belonging to 3, 4, 7, or 9. n (i), and the category with the highest corresponding predicted probability value is the predicted category of the handwritten digit sample.

[0061] Calculate the average predicted probability of samples in the four labeled categories 3, 4, 7, and 9. Among them I n Let |I| represent the set of samples of class n. n | represents the number of samples in the nth class sample set; for classes 3, 4, 7, and 9, the sample numbers are 6131, 5842, 6265, and 5949 respectively; find all samples whose predicted class is the same as the labeled class, a total of 23713 handwritten digit samples, which constitute the correct labeled set I. R Find all samples whose predicted category differs from the labeled category, totaling 474 handwritten digit samples, which constitute the inappropriate label set I. W For I W Any sample i in W If P M (i W ) < C M , then i W Belongs to the uncertain annotation set I U If P M (i W )≥C M , then i W Belongs to error annotation set I E Where M is I W The predicted categories were calculated; a total of 435 unlabeled samples and 39 incorrectly labeled samples were obtained. Examples of correctly labeled samples, unlabeled samples, and incorrectly labeled samples are shown below. Figure 3 The first, second, and third rows are shown, with the labels for the first to sixth columns being 7, 3, 7, 4, 9, and 3, respectively.

[0062] Finally, the samples from the three sets are processed separately to eliminate the influence of inappropriate labeling. Specifically, when calculating the loss, set I... R The weight of the middle sample is 1, and the set I U The weight of the middle sample is set to 0.8, and the set I... E The weight of the middle sample is set to 0, so I is directly deleted. E For sample i, the expression for the loss function is:

[0063]

[0064] Where y n (i) represents the label, which is 1 when sample i belongs to class n and 0 otherwise; Represents set I R The indicator function, when i belongs to I R Select 1 otherwise select 0; Represents set I U The indicator function, when i belongs to I U Select 1 if the training set is set correctly, otherwise select 0. After processing the training set and training the network using the above steps, predictions were made on 4029 test samples from MNIST, with a prediction accuracy of 0.985.

[0065] To demonstrate the effectiveness of this invention, comparative experiments were conducted in this embodiment. The comparative experiments employed the controlled variable method, where the test set and model were identical, differing only in the samples within the training set. Comparative Experiment A involved training the model using all 24,187 training samples without any processing, achieving a final test set prediction accuracy of 0.981. Comparative Experiment B utilized the method described in the literature "Confident Learning: Estimating Uncertainty in Dataset Labels," removing I... E Using the samples in the test set and the remaining data to train the model, the final test set prediction accuracy was 0.982. In contrast, experiment C used only I... R The model was trained using samples in the dataset, and the final prediction accuracy on the test set was 0.977. This is because the accuracy of the labeled set I is uncertain. U The samples in the model include difficult samples, and directly deleting these difficult samples will lead to a decrease in the model's generalization ability. More detailed comparative experimental information is shown in Table 1. It can be seen that the samples weighted according to this invention achieved the highest accuracy in classification. This demonstrates the effectiveness of the method for identifying and processing uncertain labeled samples in this invention.

[0066] Table 1

[0067]

[0068]

[0069] Example 2:

[0070] This embodiment discloses a classification method, including the following steps:

[0071] After obtaining samples with labeled categories, the samples are processed using the method described in Example 1 for eliminating the influence of inappropriately labeled samples;

[0072] The processed samples are used as the training set to train the classification model;

[0073] The trained classification model is used to classify the samples that need to be classified.

[0074] Example 3:

[0075] This embodiment discloses an apparatus for eliminating the influence of improperly labeled samples, comprising:

[0076] A prediction module is used to obtain the predicted category of all samples, which have labeled categories;

[0077] The discrimination module is used to classify all samples as the correct labeled set I based on the similarities and differences between the predicted category and the labeled category. R Uncertain annotation set I W and error annotation set I E ;

[0078] The weighting module is used to assign different weights to the correct label set, uncertain label set, and incorrect label set when calculating the loss function, and the weight of the samples in the incorrect label set does not exceed the weight of the samples in the uncertain label set, and the weight of the samples in the uncertain label set does not exceed the weight of the samples in the correct label set.

[0079] Example 4:

[0080] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this embodiment. Figure 4 A block diagram is shown of an exemplary electronic device 40 suitable for implementing embodiments of the present invention. Figure 4 The electronic device 40 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0081] like Figure 4 As shown, electronic device 40 is represented in the form of a general-purpose computing device. The components of electronic device 40 may include, but are not limited to: one or more processors or processing units 401, system memory 402, and bus 403 connecting different system components (including system memory 402 and processing unit 401).

[0082] Bus 403 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0083] Electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0084] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0085] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 407 typically perform the functions and / or methods described in the embodiments of the present invention.

[0086] Electronic device 40 can also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), and with one or more devices that enable a user to interact with the electronic device 40, and / or with any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 411. Furthermore, electronic device 40 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 412. As shown, network adapter 412 communicates with other modules of electronic device 40 via bus 403. It should be understood that, although... Figure 4Not shown, other hardware and / or software modules may be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0087] The processing unit 401 executes various functional applications and data processing by running programs stored in the system memory 402, such as implementing the method for eliminating the influence of inappropriately labeled samples provided in Embodiment 1 of the present invention, or implementing the classification method provided in Embodiment 2.

[0088] Example 5:

[0089] This embodiment provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform either the method for eliminating the influence of improperly labeled samples as described in Embodiment 1, or the classification method as described in Embodiment 2.

[0090] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0091] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0092] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0093] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for eliminating the influence of improperly labeled samples, characterized in that, include: The predicted categories of all samples are obtained through a prediction model, and the samples have labeled categories; The samples include image samples or sound samples; Based on the similarities and differences between the predicted and labeled categories, all samples are classified as the correctly labeled set. Uncertain annotation set and error annotation set ; When calculating the loss function, different weights are assigned to the correct label set, uncertain label set, and incorrect label set to eliminate the influence of samples in the uncertain label set and incorrect label set; the weight of samples in the incorrect label set does not exceed the weight of samples in the uncertain label set, and the weight of samples in the uncertain label set does not exceed the weight of samples in the correct label set. Based on the differences between the predicted category and the labeled category, all samples are identified as the correctly labeled set. Uncertain annotation set and error annotation set The methods include: Calculate the average predicted probability of samples in each labeled category. ,in Indicates the first A collection of class samples, Indicates the first The number of samples in the class sample set. To represent the probability that each sample belongs to each category; If the sample If the predicted category is the same as the labeled category, then the sample Belongs to the correct annotation set ; If the sample If the predicted category is different from the labeled category, then the sample Belongs to the inappropriate annotation set ; for any sample in ,if ,but Belongs to the error annotation set ,in yes The predicted category; if ,but Belongs to the uncertain annotation set ; Based on the judgment results, all samples are assigned to the correctly labeled set. Uncertain annotation set Or error annotation set middle.

2. The method for eliminating the influence of inappropriately labeled samples according to claim 1, characterized in that, The method for obtaining the predicted category of all samples through the prediction model includes: Each sample is obtained by using cross-validation on all samples through the prediction model. The predicted probability of a category is calculated, and the category with the highest predicted probability value is the predicted category of the sample.

3. The method for eliminating the influence of inappropriately labeled samples according to claim 2, characterized in that, The method involves using cross-validation on all samples through a prediction model to obtain the results for each sample. Methods for predicting the category probability, where the category with the highest predicted probability value is the predicted category of the sample, include: Randomly divide all data into Each subset of samples is equal to or similar in size and mutually exclusive, and this subset is iterated through sequentially. Each time, a subset of samples is used as the validation set, and the remaining subsets are used as the training set. The prediction model then predicts the probability that each sample in the validation set belongs to each class. ,in Represents category number; sample The predicted category is the predicted probability value. The category corresponding to the maximum value.

4. The method for eliminating the influence of inappropriately labeled samples according to any one of claims 1-3, characterized in that, The weight of the correct annotation set is 1, 0 < the weight of the uncertain annotation set is ≤ 1, and the weight of the incorrect annotation set is 0.

5. The method for eliminating the influence of inappropriately labeled samples according to claim 4, characterized in that, 0.6≤weight of uncertain annotations≤0.

9.

6. A classification method, characterized in that, include: After obtaining samples with labeled categories, the samples are processed using the method for eliminating the influence of inappropriately labeled samples as described in any one of claims 1-5; The processed samples are used as the training set to train the classification model; The trained classification model is used to classify the samples that need to be classified.

7. An apparatus for eliminating the influence of improperly labeled samples, characterized in that, include: A prediction module is used to obtain the predicted category of all samples, which have labeled categories; The samples include image samples or sound samples; The discrimination module is used to determine all samples as the correct labeled set based on the similarities and differences between the predicted category and the labeled category. Uncertain annotation set and error annotation set ; Specifically, this involves calculating the average predicted probability of samples in each labeled category. ,in Indicates the first A collection of class samples, Indicates the first The number of samples in the class sample set. To represent the probability that each sample belongs to each category; If the sample If the predicted category is the same as the labeled category, then the sample Belongs to the correct annotation set ; If the sample If the predicted category is different from the labeled category, then the sample Belongs to the inappropriate annotation set ; for any sample in ,if ,but Belongs to the error annotation set ,in yes The predicted category; if ,but Belongs to the uncertain annotation set ; The weighting module is used to assign different weights to the correct label set, uncertain label set, and incorrect label set when calculating the loss function, and the weight of the samples in the incorrect label set does not exceed the weight of the samples in the uncertain label set, and the weight of the samples in the uncertain label set does not exceed the weight of the samples in the correct label set.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method for eliminating the influence of inappropriately labeled samples as described in any one of claims 1-5 or implement the classification method as described in claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the method for eliminating the influence of inappropriately labeled samples as described in any one of claims 1-5, or the classification method as described in claim 6.

Images