Classification model training method, apparatus, device, medium, and program product

Abstract

This application discloses a classification model training method, apparatus, device, medium, and program product. The method includes: acquiring data uncertainty information corresponding to each sample in the training dataset; performing sample decoupling processing on the data uncertainty information corresponding to each sample according to the source of data uncertainty, obtaining annotation uncertainty information and feature uncertainty information corresponding to each sample; calculating second loss information of the classification model based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, and the first loss information of the classification model; and training the classification model based on the second loss information to obtain the trained classification model. By distinguishing and processing the sources of data uncertainty, this application significantly improves model performance and robustness.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to the field of artificial intelligence, specifically to a classification model training method, a classification model training device, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Uncertainty estimation (UE) is an integral part of the field of machine learning. It helps to understand the reliability and uncertainty of machine learning model predictions, thereby enabling more informed decision-making in practical applications.

[0003] Currently, the methods for handling high data uncertainty during model training are rather simplistic and crude. For example, when the data uncertainty estimate of a machine learning model is high, either the problematic samples in the training dataset are directly discarded, or the weight of the problematic samples in the model training process is increased; both of these diametrically opposed approaches reduce the model performance and robustness of the machine learning model. Summary of the Invention

[0004] This application provides a classification model training method, apparatus, device, medium, and program product, which significantly improves model performance and robustness by distinguishing the sources of uncertainty in the processed data.

[0005] On one hand, embodiments of this application provide a classification model training method. The data uncertainty sources of the prediction results output by the classification model include: labeled noisy samples and hard samples in the training dataset. The method includes:

[0006] Obtain the data uncertainty information for each sample in the training dataset. The data uncertainty information is used to characterize: the degree of influence of the sample on the prediction uncertainty of the prediction result; the prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model.

[0007] Based on the source of data uncertainty, sample decoupling processing is performed on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample; the annotation uncertainty information is used to describe the probability information of the sample being a labeled noise sample, and the feature uncertainty information is used to describe the probability information of the sample being a hard sample;

[0008] Based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model, the second loss information of the classification model is calculated; the first loss information is calculated based on the prediction result of the classification model for each sample and the true class label of each sample.

[0009] The classification model is trained based on the second loss information to obtain the trained classification model. The trained classification model is applied to category classification scenarios, which include at least one of the following: image recognition scenario, autonomous driving scenario, risk assessment scenario, medical diagnosis scenario, and email classification scenario.

[0010] On the other hand, embodiments of this application provide a classification model training apparatus. The data uncertainty sources of the prediction results output by the classification model include: labeled noisy samples and hard samples in the training dataset. The apparatus includes:

[0011] The acquisition unit is used to acquire the data uncertainty information corresponding to each sample in the training dataset. The data uncertainty information is used to characterize: the degree of influence of the sample on the prediction uncertainty of the prediction result; the prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model.

[0012] The processing unit is used to perform sample decoupling processing on the data uncertainty information corresponding to each sample according to the source of data uncertainty, so as to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample; the annotation uncertainty information is used to describe the probability information of the sample being a labeled noise sample, and the feature uncertainty information is used to describe the probability information of the sample being a hard sample;

[0013] The processing unit is also used to calculate the second loss information of the classification model based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model; the first loss information is calculated based on the prediction result of the classification model for each sample and the true class label of each sample.

[0014] The processing unit is also used to train the classification model based on the second loss information to obtain the trained classification model. The trained classification model is applied to category classification scenarios, which include at least one of the following: image recognition scenario, autonomous driving scenario, risk assessment scenario, medical diagnosis scenario, and email classification scenario.

[0015] In one implementation, the training dataset includes samples and their corresponding true class labels; the processing unit, used to obtain uncertain information about the data for each sample in the training dataset, is specifically used for:

[0016] Obtain the model parameters of the classification model. The model parameters are the parameters used by the classification model to classify and identify samples in this round of model training.

[0017] Based on model parameters and samples, predict the probability distribution of the true class label corresponding to the sample;

[0018] The probability distribution of the true class label corresponding to the sample serves as the data uncertainty information corresponding to the sample.

[0019] In one implementation, the processing unit, used to perform sample decoupling processing on the data uncertainty information corresponding to each sample according to the source of data uncertainty, to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample, specifically is used for:

[0020] When the source of data uncertainty is labeled noisy samples, label prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information corresponding to each sample; and,

[0021] When the source of data uncertainty is difficult samples, parameter prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample.

[0022] In one implementation, the training dataset includes samples and their corresponding ground truth class labels; the processing unit, used to perform label prediction processing on the data uncertainty information corresponding to each sample, specifically performs the following when obtaining the annotation uncertainty information corresponding to each sample:

[0023] Obtain the predicted class label corresponding to the sample. The predicted class label is obtained by the classification model through the classification and recognition process of the sample.

[0024] Based on the sample and the predicted class label, predict the probability distribution of the true class label corresponding to the sample;

[0025] The probability distribution of the true class labels corresponding to the samples serves as the annotation uncertainty information for the samples.

[0026] In one implementation, the processing unit, used to perform parameter prediction processing on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample, specifically performs the following:

[0027] Obtain the predicted class label corresponding to the sample. The predicted class label is obtained by the classification model through the classification and recognition process of the sample.

[0028] Obtain the model parameters of the classification model. The model parameters are the parameters used by the classification model to classify and identify samples in this round of model training.

[0029] Based on model parameters and samples, predict the probability distribution of the predicted class label corresponding to the sample;

[0030] The probability distribution of the predicted class label corresponding to the sample serves as the feature uncertainty information corresponding to the sample.

[0031] In one implementation, the processing unit, when calculating the second loss information of the classification model based on the annotation uncertainty information corresponding to each sample, the feature uncertainty information corresponding to each sample, and the first loss information of the classification model, specifically performs the following:

[0032] The second loss information of the classification model is obtained by weighting the annotation uncertainty information, the feature uncertainty information, and the first loss information of the classification model for each sample.

[0033] In one implementation, the processing unit, used to train the classification model based on the second loss information to obtain the trained classification model, specifically performs the following:

[0034] Based on the second loss information, the model parameters of the classification model are adjusted so that the adjusted classification model pays more attention to difficult samples and less attention to samples with label noise.

[0035] The parameter-adjusted classification model is iteratively trained using the training dataset to obtain the trained classification model.

[0036] In one implementation, when the category classification scenario is an image recognition scenario, the training dataset includes image samples, and the classification model is an image classification model; the sources of data uncertainty in the prediction results output by the image classification model include: labeled noisy image samples and difficult image samples in the training dataset;

[0037] When the category classification scenario is email classification, the training dataset includes email text samples, and the classification model is an email classification model. The sources of data uncertainty in the prediction results output by the email classification model include: labeled noisy email text and difficult email text in the training dataset.

[0038] In one implementation, the category classification scenario is the image recognition scenario, and the processing unit is further configured to:

[0039] Acquire the image to be identified;

[0040] The image classification model trained by the model is invoked to perform classification and recognition processing on the image, and the recognition result is obtained. The recognition result is used to describe the category to which the image content belongs.

[0041] On the other hand, embodiments of this application provide a computer device, the device comprising:

[0042] A processor is used to load and execute computer programs;

[0043] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described classification model training method.

[0044] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the above-described classification model training method.

[0045] On the other hand, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements the above-described classification model training method.

[0046] In this embodiment, the data uncertainty source of the prediction result output by the classification model is decoupled into labeled noise samples and hard samples in the training dataset. After obtaining the prediction result of the sample output by the classification model in any round of model iteration training, the computer device obtains the data uncertainty information corresponding to each sample in the training dataset. This data uncertainty information is used to characterize the degree of influence of the sample on the prediction uncertainty of the prediction result output in this round of iteration training. The prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model, such as 80% unreliability. In order to distinguish, identify and process the impact of labeled noise samples and hard samples in the training dataset on the classification model during the model training phase, this embodiment supports the computer device to split the data uncertainty information corresponding to each sample into annotation uncertainty information and feature uncertainty information according to the source of data uncertainty after obtaining the data uncertainty information corresponding to each sample. Among them, the annotation uncertainty information characterizes the probability information that the sample is a labeled noise sample, while the feature uncertainty information characterizes the probability information that the sample is a hard sample, thereby realizing the effective identification of labeled noise samples and hard samples in the training dataset. Then, based on the annotation uncertainty and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model, the second loss information of the classification model is calculated. That is, the annotation uncertainty and feature uncertainty information corresponding to the sample are used as indicators of the first loss information during model training. Specifically, the annotation uncertainty information is used as the loss weight of the label noise samples relative to the first loss information, and the feature uncertainty information is used as the loss weight of the hard samples relative to the first loss information. In this way, when training the classification model based on the second loss information, the model training can be carried out in the direction of reducing the loss weight of the label noise samples relative to the first loss information and increasing the loss weight of the hard samples relative to the first loss information. The aim is to reduce the classification model's attention to the label noise samples, that is, to reduce the attention to the mislabeled samples, and to strengthen the classification model's attention to the hard samples, so as to help the classification model improve its prediction ability for hard samples, thereby improving the model performance and robustness of the classification model. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the architecture of a category classification system provided in an exemplary embodiment of this application;

[0049] Figure 2 This is a flowchart illustrating a classification model training method provided in an exemplary embodiment of this application;

[0050] Figure 3 This is a schematic diagram illustrating sample decoupling provided in an exemplary embodiment of this application;

[0051] Figure 4 This is a flowchart illustrating another classification model training method provided in an exemplary embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the structure of a classification model training device provided in an exemplary embodiment of this application;

[0053] Figure 6 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] This application proposes a classification model training scheme, specifically a scheme for improving the performance of classification models based on data uncertainty decoupling. The following is a brief introduction to the technical terms and related concepts involved in the classification model training scheme provided in this application embodiment:

[0056] I. Classification Models.

[0057] A classification model is a type of machine learning model within the field of machine learning in Artificial Intelligence (AI). Classification models can include, but are not limited to, machine learning models such as logistic regression, decision trees, random forests, Naive Bayes, and KNN (K-Nearest Neighbors). This application does not limit the type of classification model in its embodiments. Artificial intelligence utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to obtain optimal results—theories, methods, technologies, and application systems for this purpose. Machine learning models within the field of artificial intelligence are network models obtained through training using machine learning (ML). Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. Machine learning typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formulaic learning.

[0058] The classification model has a corresponding training dataset, which is used to train and optimize the model. The training dataset includes samples and their true class labels (i.e., the actual category to which the sample belongs). During the model training phase, the classification model learns the features of the samples in the training dataset to predict the predicted class label (or predicted category) of each sample. Based on the difference between the true and predicted class labels, the model is trained to accurately predict the category to which a sample belongs. During the model inference phase, the input data indicating the category to be predicted is fed into the trained classification model. The model performs classification processing on the input data to obtain a probability distribution, which includes the probability of the input data belonging to each of several predefined categories. The category with the highest probability is selected as the predicted category to which the input data belongs.

[0059] II. Uncertainty estimation (UE).

[0060] Uncertainty refers to the possibility that an event / outcome / situation cannot be accurately predicted or determined. Uncertainty estimation is an indicator for estimating the uncertainty of a machine learning model's predictions. By estimating the uncertainty of a machine learning model's predictions, we can help understand the reliability and uncertainty of the model's predictions, thereby making more informed decisions in practical applications.

[0061] For example, a machine learning model is a classification model capable of classifying images. Suppose the model can identify 100 image categories, excluding the category of "cat." If an image containing the word "cat" is input into the model, it will inevitably classify it into one of the 100 categories, such as "airplane," with a 90% probability. From the model's perspective, the probability of the image belonging to the "airplane" category is very high; the model firmly believes the image belongs to the "airplane" category. However, in reality, the image may not belong to the "airplane" category. Therefore, the classification model has high prediction uncertainty, meaning its prediction is unreliable. Thus, during the training phase of a classification model, it typically outputs not only a prediction score for the input data (the prediction result, such as a 90% probability that the input image belongs to the "airplane" category) but also a prediction uncertainty score to determine the reliability of that score (e.g., a 10% prediction uncertainty score means only 10% of respondents consider the prediction reliable).

[0062] Practical experience has shown that the predictive uncertainty of classification models mainly comprises two parts: data uncertainty (Aleatoric) and model uncertainty (Epistemic). ① Data uncertainty, also known as accidental uncertainty, describes the inherent noise in the training dataset of the classification model. In other words, data uncertainty posits that the noise in the samples themselves makes the predictions of the classification model unreliable. Data uncertainty is unavoidable; for example, it cannot be mitigated by increasing the number of samples. For instance, in scenarios where image samples are collected by taking photos, shaky hands can cause blurry images, and this uncertainty cannot be eliminated by increasing the number of photos taken, as hand tremors are difficult to control each time. ② Model uncertainty, also known as cognitive uncertainty, describes the noise inherent in the classification model itself. In other words, model uncertainty posits that the model parameters estimated during the model training phase lead to unreliable predictions. Model uncertainty is avoidable; for example, it can be overcome by selectively adjusting the model parameters (e.g., increasing the number of training iterations, increasing the number of samples in the training dataset) to optimize the model parameters multiple times.

[0063] To overcome the negative impact of data uncertainty on classification models, existing technologies propose two opposing solutions. One mainstream approach argues that high data uncertainty indicates contaminated samples in the training dataset (e.g., samples with noisy labels, referred to as "noisy labels"). These unreliable samples cause the classification model to overemphasize them, reducing model reliability. Therefore, suppressing or discarding these samples improves the model's reliability. The other mainstream approach argues that high data uncertainty stems from difficult samples in the training dataset, which provide more information and require increased weighting. These two solutions are contradictory. Discarding noisy labels during model training limits the model's predictive performance on contaminated samples, while increasing the weight of difficult samples during training severely interferes with the model's performance. Specifically, noisy labels...

[0064] The classification model training scheme provided in this application decouples data uncertainty into annotation uncertainty and feature uncertainty based on different sources of data uncertainty. Annotation uncertainty describes data uncertainty originating from label noise samples in the training dataset, while feature uncertainty describes data uncertainty originating from difficult samples in the training dataset. In other words, the sources of data uncertainty are decoupled into label noise samples and difficult samples in the training dataset. By distinguishing and processing these two sources of data uncertainty during the model training phase, the classification model can more efficiently focus on valuable information, thereby improving the performance and robustness of the classification model.

[0065] Specifically, this application's embodiments replace the traditional loss function (such as the cross-entropy loss function (CELoss)) of the original classification model with a new loss function, achieving differentiated processing of two sources of data uncertainty (i.e., annotation uncertainty and feature uncertainty), thus steadily improving the performance of the classification model. This not only enhances the reliability of the classification model but also strengthens its robustness, effectively promoting the development of uncertainty estimation research. In this application's embodiments, the newly constructed loss function is referred to as the target loss function. The target loss function is a component or plug-in used to replace the traditional loss function; it can be easily integrated / deployed into the classification model as a component without changing the original model structure, reducing the learning cost for developers and providing a simple and effective model training method for the classification model.

[0066] Taking a single round of model training (i.e., the process of a classification model performing a complete training using all samples in the training dataset, including forward propagation and backpropagation) as an example, the general process of training a classification model based on a target loss function provided in this application embodiment may include: inputting each sample in the training dataset into the classification model to obtain the prediction result of the classification model for each sample; the computer device, based on the prediction result of each sample, obtains the data uncertainty information corresponding to each sample, which is used to characterize the degree of influence of the sample on the prediction uncertainty of the prediction result, and the prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model, which is usually affected by data uncertainty and feature uncertainty. Then, the computer device performs sample decoupling processing on the data uncertainty information of each sample according to the source of data uncertainty, aiming to distinguish and process different sources of data uncertainty, and obtains the annotation uncertainty information and feature uncertainty information corresponding to each sample; the annotation uncertainty information is used to describe the probability information of the sample being a labeled noise sample, while the feature uncertainty information is used to describe the probability information of the sample being a hard sample. After obtaining the probability information of the samples belonging to two different types of data uncertainty, the second loss information of the classification model in this round of model training can be calculated based on the target loss function. Specifically, the second loss information is obtained based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model (the loss information calculated using the traditional loss function). Finally, the classification model is backpropagated based on the second loss information to achieve model training, so as to obtain the trained classification model.

[0067] Therefore, on the one hand, the embodiments of this application decouple data uncertainty into annotation uncertainty and feature uncertainty, that is, decouple the sources of data uncertainty into labeled noisy samples and hard samples in the training dataset. During the model training phase, by distinguishing and processing labeled noisy samples and hard samples, the classification model can effectively focus on valuable information in different samples, thereby improving the training effect and model performance of the classification model. In particular, the classification model can better adapt to and handle labeled noisy samples and hard samples, reducing misjudgments and misclassifications, and lowering the risk of prediction errors. On the other hand, the proposed objective loss function can be easily integrated into the classification model as a component or plugin without changing the original model structure, greatly reducing the learning cost for developers, providing a simple and effective optimization method for the classification model while improving model performance and robustness.

[0068] The classification model training scheme provided in this application, as a solution to the predictive uncertainty of a classification model, can be applied to various category classification scenarios where a classification model is used for category prediction or recognition. In other words, the classification model training scheme provided in this application is applicable to any category classification scenario that requires category recognition using a classification model. The category classification scenarios to which the trained classification model can be applied may include, but are not limited to, at least one of the following: ① Image recognition scenario: When the classification model is applied to image recognition, it can accurately identify the image content in an image and classify the image into different categories based on the identified image content. For example, when training a cat and dog classification model, inputting an image into the model allows it to automatically classify the image as having a probability of belonging to the cat category and a probability of belonging to the dog category. ② Customer classification scenario: When the classification model is applied to customer classification scenarios, it can automatically divide customers who have joined group chats or social software into different groups to better understand the needs and behaviors of different customers. ③ Risk assessment scenario: When the classification model is applied to risk assessment scenarios, such as financial risk assessment, it can learn from borrowers' historical behavioral data to classify borrowers into categories with different financial risk levels to better assess the financial risk of different borrowers. ④ Medical Diagnosis Scenarios: When applied to medical diagnosis scenarios, classification models can help analyze, interpret, and categorize medical images, assisting doctors in making more accurate diagnoses. ⑤ Email Classification Scenarios: When applied to email classification scenarios, models can automatically categorize emails in a user's inbox, accurately identifying spam and legitimate emails, and processing different categories separately, improving email organization efficiency. ⑥ Autonomous Driving Scenarios: When applied to the field of autonomous driving, classification models can classify objects in front of the driver (e.g., classifying various objects on the road ahead, such as people, vehicles, and traffic lights) and understand the driving environment (e.g., understanding whether the surrounding environment is safe and drivable), thereby helping to better achieve autonomous driving.

[0069] Based on the above introduction to the application of classification models in various category classification scenarios, the following two points need to be explained:

[0070] (1) When a classification model is applied to different category classification scenarios, the input data of the classification model is not the same. Specifically, the sample data input during the model training stage and the data to be identified input during the model inference stage are both adapted to or matched with the corresponding category classification scenario. For example, when the category classification scenario is an image recognition scenario, the training dataset input to the classification model during the model training stage includes image samples, and the classification model is an image classification model with image classification capabilities. In this case, the data uncertainty of the prediction results output by the image classification model includes: labeled noisy image samples and difficult image samples in the training dataset. As another example, when the category classification scenario is an email classification scenario, the training dataset input to the classification model during the model training stage includes email text samples, and the classification model is an email classification model with email classification capabilities. In this case, the data uncertainty of the prediction results output by the email classification model includes: labeled noisy email text and difficult email text in the training dataset. Considering that the category classification scenarios applied to the classification model are rich, the format of the input data input to the classification model under each category classification scenario will not be described.

[0071] (2) The above description is merely an exemplary product performance and category classification scenario given in the embodiments of this application, and does not limit the product performance and category classification scenario of the classification model training scheme provided in the embodiments of this application. The classification model training scheme provided in the embodiments of this application can provide efficient, accurate and convenient classification services in various category classification scenarios, and demonstrates high value and practicality in various category classification scenarios, which helps to improve user experience and satisfaction.

[0072] To facilitate understanding, the following will be combined with Figure 1 The category classification system shown illustrates the classification model training scheme proposed in the embodiments of this application. For example... Figure 1 As shown, the category classification system includes a terminal 101 and a server 102. This application embodiment does not limit the number and type of the terminal 101 and the server 102.

[0073] Terminal 101 is a user-held terminal device. This terminal device has an application with classification and recognition capabilities deployed or running on it. When the user wants to perform classification and recognition, they upload the data to be identified through this application, and the application outputs the recognition results for the data. An application is a computer program designed to perform one or more specific tasks. Classified by how it runs, applications can include, but are not limited to: ① Client-side applications, also known as application clients or APP (Application) clients, which are applications installed and running on the terminal. They need to work with a server to provide local services to the target object. For example, a client is a browser downloaded from an app store and installed on the computer device. ② Installation-free applications, meaning applications that can be used without downloading and installing. Specifically, this refers to any application running within a client. When an application runs within a client, the client and application have a parent-child relationship; the client is the parent program of the application, and the application is the child program of the client. Such installation-free applications are commonly known as mini-programs, and they typically run as child programs within the client. ③ An application can also refer to a web application opened through a browser. A web application is a program that requires internet access. Compared to a client application, it does not require downloading an installation package; access to the web application can be achieved simply through a browser. This application embodiment does not limit the operating mode of the application providing classification and identification functions in terminal 101, but this is only described here. The device type of terminal 101 may include, but is not limited to: smartphones (such as smartphones running the Android system, or smartphones running the Internetworking Operating System (IOS)), tablet computers, portable personal computers, mobile internet devices (MIDs), in-vehicle devices, head-mounted devices, intelligent chatbots, and aircraft, etc. This application embodiment does not limit the device type of the terminal device.

[0074] Server 102 is the server corresponding to terminal 101, used to interact with terminal 101 to provide computing and application service support, specifically providing application services and technical support for applications running on terminal 101. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The various devices included in the category classification system can communicate directly or indirectly through wired or wireless means. This embodiment does not limit the communication methods between devices; for example, communication methods between devices may include, but are not limited to, HTTP requests, remote procedure calls, sockets, and content sharing.

[0075] The classification model training scheme provided in this application embodiment can be executed by a computer device. The computer device can be... Figure 1 The system shown may contain terminal 101 or server 102, or both terminal 101 and server 102; that is, the embodiments of this application may be executed by terminal 101 and / or server 102. For example, model training and model inference of the classification model may both be executed by terminal 101, or both by server 102; as another example, model training of the classification model may be executed by server 102, and when the trained classification model is deployed to terminal 101 (specifically, deployed to an application), model inference of the classification model may be executed by terminal 101.

[0076] The following section uses a category classification scenario as the image recognition scenario, with both terminal 101 and server 102 working together, to introduce the general process of the classification model training scheme, including:

[0077] (1) Model training phase.

[0078] Server 102 is associated with a database containing a training dataset for model training. In the image recognition scenario, the training dataset consists of image samples, including noisy image samples with incorrect real-world class labels and difficult-to-predict image samples. In this case, when training an image classification model with image recognition capabilities, server 102 first retrieves the training data from the database. Then, server 102 inputs the image samples from the training dataset into the image classification model. The model performs classification processing on each image sample, obtaining a prediction result and probability distribution for each sample. The probability distribution includes the probability information of the image sample belonging to each class, and the prediction result indicates the probability information of the image sample belonging to the class with the highest probability information. After obtaining the prediction result and probability distribution for each image sample, server 102 can calculate and obtain the data uncertainty information corresponding to each image sample. Then, server 102 performs sample decoupling processing on the data uncertainty information corresponding to each image sample according to the source of data uncertainty, obtaining annotation uncertainty information and feature uncertainty information corresponding to each image sample. The annotation uncertainty information describes the probability information that the image sample is a labeled noisy image sample, while the feature uncertainty information describes the probability information that the image sample is a difficult image sample. Then, server 102 uses the target loss function designed in the embodiments of this application to calculate the second loss information of the image classification model based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the image classification model. Finally, server 102 adjusts the model parameters of the image classification model based on the second loss information to obtain the image classification model after model training. By iteratively training the image classification model multiple times using the training dataset according to the above model training process, a high-performance and robust image classification model can be obtained.

[0079] (2) Model reasoning stage.

[0080] Based on implementation method (1), after server 102 obtains a high-performance and robust image classification model, the image classification model can be deployed in terminal 101. In this case, if a user wants to classify and recognize an image, the user uploads the image to be recognized in terminal 101; terminal 101 (specifically, an application with image recognition capabilities in terminal 101) calls the image classification model trained by the deployed model to perform classification and recognition processing on the image, and obtains the recognition result (or prediction result) of the image; the recognition result is used to describe the category to which the image content belongs. For example, if the categories that the image classification model can recognize include category A, category B and category C, and the probability distribution obtained by the image classification model for classifying and recognizing the image to be recognized is (0.2, 0.5, 0.3), then the recognition result of the image is determined to be 0.5 (or expressed as 50%), indicating that the image has a 50% probability of belonging to category B.

[0081] Based on the above Figure 1 As can be seen from the category classification system shown, the embodiments of this application support the integration of the trained image classification model on the terminal device side. Relying on the advantages of low latency and stability of the terminal device, it can ensure that users are provided with a lower latency, higher stability and a smoother classification experience.

[0082] Based on the above brief introduction to the classification model training scheme and category classification system provided in the embodiments of this application, the following points should also be noted:

[0083] ①The embodiments of this application mentioned above Figure 1 The system architecture shown is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application and does not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will understand that, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems. That is to say, Figure 1 The schematic diagram of the category classification system shown is an exemplary schematic diagram; in actual applications, the number and distribution of computer devices included in the category classification system may change, and the schematic diagram of the category classification system in this application embodiment is not limited.

[0084] For example, server 102 in the category classification system can be a single server or a distributed system; this distributed system includes multiple servers, which allows the model training task to be split across different servers for execution, improving the efficiency of model training and reducing the load on a single server. Furthermore, the trained classification model can also be deployed on server 102; thus, during the model inference phase, terminal 101 only needs to transmit the image to be recognized to server 102 and display the recognition results returned by server 102.

[0085] ② In this application embodiment, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations. Obtaining personal information requires the knowledge or consent of the individual (or a legal basis for information acquisition), and subsequent data use and processing should be carried out within the scope of laws, regulations, and the authorization of the personal information subject. For example, when this application embodiment is applied to specific products or technologies, such as uploading an image to be identified, the user's permission or consent is required, and the collection, use, and processing of relevant data (such as classifying and identifying the image to be identified) must comply with relevant national and regional laws, regulations, and standards.

[0086] Based on the classification model training scheme described above, this application proposes a more detailed classification model training method. The classification model training method proposed in this application will be described in detail below with reference to the accompanying drawings.

[0087] Figure 2 The illustration shows a flowchart of a classification model training method provided in an exemplary embodiment of this application; the classification model training method can be executed by a computer device, which can be... Figure 1 The system shown includes terminals and / or servers. The classification model training method may include, but is not limited to, steps S201-S204:

[0088] S201: Obtain the data uncertainty information corresponding to each sample in the training dataset.

[0089] As described above, the prediction uncertainty of a classification model consists of data uncertainty and model uncertainty. Prediction uncertainty can be used to characterize the reliability of the prediction results output by the classification model. This reliability is mainly affected by data uncertainty and model uncertainty. For example, when data uncertainty and model uncertainty are large, the prediction uncertainty of the classification model is large.

[0090] Furthermore, the reliability of prediction uncertainty is specifically represented or described by prediction uncertainty information. For example, a prediction uncertainty of 10% indicates that the prediction result output by the classification model has only a 10% reliability or credibility. Similarly, data uncertainty can be represented or described by data uncertainty information, and model uncertainty can be represented or described by model uncertainty information. Data uncertainty information characterizes the degree of influence of a sample on the prediction uncertainty of the classification model's output for that sample; that is, data uncertainty information can be understood as a numerical value or mathematical expression that characterizes the contribution of data uncertainty to prediction uncertainty. Model uncertainty information characterizes the degree of influence of model parameters on the prediction uncertainty of the classification model's output; that is, model uncertainty information can be understood as a numerical value or mathematical expression that characterizes the contribution of model uncertainty to prediction uncertainty. When the prediction uncertainty information corresponding to the prediction uncertainty of the classification model is fixed, if the data uncertainty information value is large, it indicates that the prediction uncertainty is mainly affected by data uncertainty; if the data uncertainty information value is small, it indicates that the prediction uncertainty is mainly affected by model uncertainty.

[0091] The information corresponding to each type of uncertainty (such as the data uncertainty information) is calculated based on the classification model, which calculates the prediction results and probability distribution of the samples. Therefore, to better express and understand the information corresponding to each type of uncertainty, some symbol definitions related to the classification model are given below:

[0092] ① Let x represent the training dataset, where i is the i-th sample out of N samples; i y is the input data for the i-th sample; i y is the true class label (or annotation) of the i-th sample. i It belongs to {1,…,K} and each value in {1,…,K} represents a category.

[0093] ②f:x→P represents a classification model, the goal of which is to learn a function f so that the function f can predict a probability distribution P based on the input data x.

[0094] ③ The probability distribution P of sample x is located as This represents the probability distribution for predicting category y under fixed model parameters w and input sample x. This probability distribution includes the probability information of category y belonging to each category.

[0095] ④Given the known probability distribution P of sample x, the predicted class label for sample x is determined based on probability distribution P as follows: This represents the maximum probability information p in the probability distribution. kThe predicted category of sample x (i.e., the predicted class label, which is the prediction result obtained by the classification model for classifying and recognizing the sample).

[0096] When training a classification model f, the goal of the model training is: in, Let y represent the predicted class label of the i-th sample. i This represents the true class label of the i-th sample; It is an indicator function, representing the predicted class label of the i-th sample. and real-class tags y i When the values ​​are the same, the indicator function takes the value 1; otherwise, it takes the value 0. Therefore, This means training a classification model to ensure that the predicted class label of a sample x identified by the classification model is the same as the true class label, so as to ensure that the classification model has accurate prediction and recognition capabilities.

[0097] Based on the above symbol definitions, the prediction uncertainty information used to describe the prediction uncertainty of a classification model can be expressed as the following equation:

[0098] H p(y|x) (y)=E q(f) [H p(y|x,f) (y)]+I p(y,f|x) (y;f) (1)

[0099] Among them, H p(y|x) (y) represents the prediction uncertainty information, used to indicate or describe the prediction uncertainty of the classification model; E q(f) [H p(y|x,f) [y] represents data uncertainty information, used to indicate or describe data uncertainty; I p(y,f|x) (y;f) represents the model uncertainty information, used to represent or describe the feature uncertainty information.

[0100] In network modeling, the data uncertainty and model uncertainty that constitute prediction uncertainty are typically represented in different ways. For example, data uncertainty can be represented as the probability distribution of a sample x given a set of model parameters Θ of a classification model. Similarly, model uncertainty can be represented as the probability distribution of the model parameters of a classification model given a training dataset. In other words, the above prediction uncertainty can be expressed as the following equation:

[0101] P(y|x,D)=∫ Θ P(y|x,θ)p(θ|D)dθ (2)

[0102] Here, P(y|x,D) represents prediction uncertainty, describing the probability distribution of class y given a sample x and a training dataset D. P(y|x,θ) represents data uncertainty, describing the probability distribution of class y given model parameters θ and a sample x. p(θ|D) represents model uncertainty, describing the probability distribution of model parameters θ given a training dataset D. Θ is the set of model parameters for the classification model, which includes each model parameter θ in the classification model.

[0103] Based on the expression of data uncertainty information in formula (2), the parameters required for the computer device to calculate the data uncertainty information P(y|x,θ) for each sample include: the model parameters of the classification model during one round of model training, the sample x input to the classification model, and the true class label of the sample x. Therefore, after one round of model training, that is, after using the classification model to classify and identify the samples in the training dataset and obtaining the probability distribution of the classification model for each sample in the training dataset, the computer device will obtain the model parameters of the classification model. These model parameters are the parameters used by the classification model to classify and identify the samples during this round of model training; specifically, the obtained model parameter set includes one or more model parameters used in this round of model training. Then, based on the model parameters and the sample, the computer device predicts the probability distribution of the true class label corresponding to the sample; that is, given the sample x input to the classification model and the model parameters θ, the probability distribution of the true class label y is calculated or predicted. Among them, the probability distribution of the true class label y corresponding to the sample is the data uncertainty information corresponding to the sample x.

[0104] S202: Based on the source of data uncertainty, perform sample decoupling processing on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample.

[0105] Considering that the model uncertainty represented by the model uncertainty information p(θ|D) in formula (2) can be overcome through corresponding adjustment methods, while the data uncertainty represented by the data uncertainty information P(y|x,θ) is difficult to overcome, this application proposes a solution for data uncertainty to overcome data uncertainty and thus improve the model performance of the classification model. As mentioned above, this application mainly decouples data uncertainty into annotation uncertainty and feature uncertainty. Annotation uncertainty describes the label noise samples in the training dataset, and feature uncertainty describes the hard samples in the training dataset. The label noise samples and hard samples are distinguished, identified, and processed to help the classification model focus on valuable information in the training dataset, thereby improving the model training effect.

[0106] In specific implementation, based on the source of data uncertainty, sample decoupling processing is performed on the data uncertainty information corresponding to each sample to obtain annotation uncertainty information and feature uncertainty information corresponding to each sample. The annotation uncertainty information corresponds to annotation uncertainty and can be understood as a numerical value or expression representing or describing annotation uncertainty. Similarly, the feature uncertainty information corresponds to feature uncertainty and can be understood as a numerical value or expression representing or describing feature uncertainty. Furthermore, the sources of data uncertainty include labeled noisy samples and hard samples; therefore, the process of distinguishing and processing different sources of data uncertainty includes:

[0107] (1) When the source of data uncertainty is label noise samples, the computer device performs label prediction processing on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information corresponding to each sample.

[0108] The annotation uncertainty information corresponding to the sample is used to describe the probability that the sample is a labeled noise sample. In other words, the embodiments of this application support the estimation of the probability that each sample in the training dataset is a labeled noise sample; in this way, the impact of each sample on the training of the classification model is dynamically controlled according to the probability of the sample being a labeled noise sample, ensuring that the classification model can extract valuable information from the labeled noise samples, thereby improving the training effect of the classification model.

[0109] Considering that labeled noise samples refer to samples whose data has been contaminated, such as samples whose true class labels are incorrectly labeled, the annotation uncertainty caused by labeled noise samples arises because these samples are labeled with incorrect true class labels, leading to reduced model performance when training a classification model based on these incorrect true class labels. Therefore, embodiments of this application support predicting the true class label based on the predicted class label of a sample by the classification model, which serves as the annotation uncertainty for that sample, allowing for thorough analysis and identification of labeled noise samples.

[0110] Specifically, the computer device acquires the predicted class label corresponding to the sample. This predicted class label is obtained by the classification model through classification and recognition processing of the sample. The predicted class label represents the category to which the classification model predicts the sample belongs. For example, in an image recognition scenario, if the classification model predicts the predicted class label of an image as "cat," it means that the image content is most likely of the category "cat." Then, based on the sample and its corresponding predicted class label, the computer device predicts the probability distribution of the true class label corresponding to the sample. Specifically, given the sample input to the classification model and the predicted class label predicted by the model for that sample, the computer predicts the probability distribution of the true class label corresponding to the sample. The probability distribution of the true class label corresponding to the sample serves as the annotation uncertainty information for the sample. Based on this, the annotation uncertainty information corresponding to sample x can be expressed by the following formula:

[0111]

[0112] in, Let y be the predicted class label of sample x, and y be the true class label of sample x. This represents a sample x given an input classification model and the predicted class label that the classification model predicts for that sample. Under certain conditions, this is the probability distribution of the true class label y corresponding to sample x. It can be seen that when the predicted class label of sample x by the classification model is inconsistent with the true class label of sample x, the annotation uncertainty information corresponding to sample x will be high, indicating that sample x is more likely to be a label noise sample. Conversely, when the predicted class label of sample x by the classification model is consistent with the true class label of sample x, the annotation uncertainty information corresponding to sample x will be low, indicating that sample x is less likely to be a label noise sample. Therefore, the annotation uncertainty information corresponding to sample x can reflect the probability information that sample x is a label noise sample, thus effectively realizing the identification of label noise samples in the training dataset. This reduces the focus on these samples during subsequent optimization, preventing label noise samples from excessively affecting the prediction performance of the classification model. At the same time, retaining these samples for model training can enhance the classification model's ability to handle contaminated input data and improve the robustness of the classification model.

[0113] (2) When the source of data uncertainty is difficult samples, parameter prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample.

[0114] The feature uncertainty information corresponding to the sample is used to describe the probability that the sample is a difficult sample. In other words, the embodiments of this application support the estimation of the probability that each sample in the training dataset is a difficult sample; in this way, the impact of each sample on the training of the classification model is dynamically controlled according to the probability of the sample being a difficult sample, ensuring that the classification model can extract valuable information from difficult samples, thereby improving the training effect of the classification model.

[0115] Considering that difficult samples refer to samples that are difficult to classify or predict correctly during model training, meaning that the classification model struggles to predict these samples correctly, the annotation uncertainty caused by difficult samples arises because the model parameters in the classification model cannot predict these difficult samples. Based on this, embodiments of this application support predicting the predicted class label of a sample based on the model parameters of the classification model. This serves as the feature uncertainty corresponding to the sample, and can fully analyze and identify difficult samples in the training dataset.

[0116] Specifically, the computer device acquires the predicted class label corresponding to the sample, which is obtained by the classification model through the classification and recognition process of the sample. The computer device also acquires the model parameter set (including one or more model parameters), which are the parameters used by the classification model during the current training process to classify and recognize the sample. Then, based on the model parameters and the sample, the computer device predicts the probability distribution of the predicted class label corresponding to the sample; specifically, given the model parameters and the sample, it calculates the probability distribution of the predicted class label. Based on this, the feature uncertainty information corresponding to sample x can be expressed as the following formula:

[0117]

[0118] in, Let θ be the predicted class label of sample x, and θ be the model parameters used in this round of model training for the classification model. This represents the predicted class label given model parameters θ and sample x. The probability distribution of sample x. Therefore, the feature uncertainty information corresponding to sample x represents the probability distribution of the predicted class label of sample x. When the classification model can predict the predicted class label for sample x, the feature uncertainty information corresponding to sample x will be high, indicating that the model parameters of the classification model can predict the sample, and thus sample x is less likely to be a hard sample. Conversely, when the classification model cannot predict the predicted class label for sample x, the feature uncertainty information corresponding to sample x will be low, indicating that the model parameters of the classification model cannot predict the sample, and thus sample x is more likely to be a hard sample. Therefore, the feature uncertainty information corresponding to sample x can reflect the probability information that sample x is a hard sample, thus effectively realizing the identification of hard samples in the training dataset. This allows for increased attention to these samples during subsequent optimization, helping the classification model learn from hard samples. Retaining these samples for model training can enhance the robustness of the classification model.

[0119] Based on the above description, after decoupling the data uncertainty information of the samples into annotation uncertainty information and feature uncertainty information, the prediction uncertainty information of the classification model shown in the aforementioned formula (2) can be replaced by the following formula:

[0120]

[0121] The meanings of each part in formula (5) are described above and will not be repeated here.

[0122] In summary, steps S201-S202 mainly explain the reasoning process of decoupling prediction uncertainty information into annotation uncertainty information and feature uncertainty information; to facilitate understanding of this decoupling reasoning process, the following section, in conjunction with the appendix... Figure 3 Provide the specific decoupling process for sample decoupling. For example... Figure 3 As shown, the prediction uncertainty H of the known classification model p(y|x) (y) is derived from the uncertain information E in the data. q(f) [H p(y|x,f) (y)] and model uncertainty information I p(y,f|x) Composed of (y; f), i.e., H p(y|x) (y)=E q(f) [H p(y|x,f) (y)]+I p(y,f|x) (y; f). Regarding the data uncertainty E corresponding to the sample. q(f) [H p(y|x,f) This part (y) performs sample decoupling (or data decoupling), and the decoupling yields the uncertain annotation information corresponding to the sample. and feature uncertainty information

[0123] Therefore, on the one hand, by decoupling data uncertainty into annotation uncertainty and feature uncertainty, it is possible to more accurately identify labeled noisy samples and hard samples in the training dataset. This allows the classification model to more effectively address and handle these situations, thereby improving training effectiveness and model performance. On the other hand, retaining the use of labeled noisy samples and hard samples for model training enables the classification model to better adapt to and handle such situations, reducing misjudgments and misclassifications, and improving the robustness of the classification model. When enterprises understand the reliability and robustness of the classification model, they are more willing to apply it to their actual production environments, thereby driving industry innovation and development.

[0124] S203: Based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model, calculate the second loss information of the classification model.

[0125] After calculating the annotation uncertainty and feature uncertainty for each sample based on the aforementioned steps, the annotation uncertainty and feature uncertainty are designed as specific optimization objectives. This allows the traditional loss function of the classification model to be adjusted according to the optimization objectives, resulting in a target loss function. When adjusting the parameters of the classification model using the adjusted target loss function, the model performance of the classification model can be improved.

[0126] Specifically, given the known feature uncertainty information corresponding to the samples, the optimization objective for constructing the feature uncertainty (FU) of the classification model in this embodiment is as follows:

[0127]

[0128] The feature uncertainty of a classification model can be represented as the input sample x relative to the predicted class label under a set of model parameters Θ. The probability distribution. Therefore, the optimization objective of feature uncertainty is the feature uncertainty information.

[0129] Similarly, the optimization objective for establishing annotation uncertainty (AnnoU) in this application embodiment is as follows:

[0130]

[0131] Therefore, the optimization objective of annotation uncertainty is the annotation uncertainty information of the samples. By performing unsupervised clustering on the annotation uncertainty information of the samples, we can model the annotation uncertainty of the entire training dataset. The optimization objective of the annotation uncertainty of the entire training dataset is expressed as follows: This represents the set of predicted class labels corresponding to the training dataset, which includes the predicted class label for each sample in all samples. Unsupervised clustering is a machine learning method that aims to divide the training dataset into different groups or clusters for machine learning, based on data similarity or distance.

[0132] Based on the aforementioned optimization objectives for feature uncertainty and annotation uncertainty, a new loss function (i.e., the target loss function) is constructed as follows:

[0133]

[0134] in, This represents the loss weight corresponding to the labeled noisy sample; This represents the loss weights corresponding to the difficult samples; -ylog(p(x) i ,θ)) represents the standard loss function of the classification model.

[0135] In other words, the embodiments of this application use annotation uncertainty information and feature uncertainty information as the loss weights of the standard loss function of the classification model, indicating the weight influence of different samples (such as labeled noise samples and hard samples) on the standard loss function during model training; throughout the model training phase, the target loss function dynamically adjusts the loss weights to help the classification model focus on hard samples and reduce the focus on labeled noise samples, thereby training the classification model's learning ability for hard samples and minimizing the adverse effects of data uncertainty on the prediction performance of the classification model.

[0136] Based on this, according to the definition of the target loss function, after obtaining the annotation uncertainty information and feature uncertainty information corresponding to each sample, the computer device supports weighted calculation based on the annotation uncertainty information, feature uncertainty information, and the first loss information of the classification model to obtain the second loss information of the classification model. The first loss information of the classification model is calculated based on the prediction result (i.e., predicted class label) of the classification model for each sample and the true class label of each sample; specifically, it is the loss information calculated using the standard loss function of the classification model, such as the cross-entropy loss function. The second loss information of the classification model is the loss information calculated using the target loss function.

[0137] S204: Train the classification model based on the second loss information to obtain the trained classification model.

[0138] Based on the aforementioned steps, the computer obtains the second loss information of the classification model after this round of model training. The computer then performs backpropagation on the classification model based on this second loss information, aiming to adjust the model parameters according to the second loss information to obtain the trained classification model. Specifically, the computer adjusts the model parameters based on the second loss information, including parameter adjustments. This involves adjusting the model parameters in a progressively decreasing direction according to the second loss information, such as increasing the loss weights for feature uncertainty and decreasing the loss weights for annotation uncertainty. This results in the adjusted classification model paying more attention to difficult samples and less attention to samples with noisy labels. Adjusting the model parameters in a progressively decreasing direction according to the loss weights for feature uncertainty and annotation uncertainty during multiple iterations of optimization means adjusting the model parameters in this way.

[0139] Furthermore, if the classification model does not reach the stopping condition after this round of model training, the training dataset is used to continue iteratively training the parameter-adjusted classification model to obtain the trained classification model. The stopping condition may include, but is not limited to: reaching the total number of training rounds (e.g., 1000) for the classification model, or the second loss information gradually stabilizing (e.g., the difference between the second loss information after multiple consecutive training rounds tends to 0), or the second loss information equaling a preset value (e.g., 0), etc.

[0140] In summary, on the one hand, the embodiments of this application support decoupling the sources of data uncertainty in the prediction results output by the classification model into labeled noise samples and hard samples in the training dataset. During the model training phase, it achieves effective identification of labeled noise samples and hard samples in the training dataset, distinguishing and processing their impact on the classification model. This aims to reduce the classification model's focus on labeled noise samples, i.e., reduce attention to mislabeled samples, while strengthening the classification model's focus on hard samples, helping the classification model improve its prediction ability for hard samples, thereby improving the model's performance and robustness. On the other hand, the embodiments of this application replace the traditional loss function of the classification model with a target loss function. By distinguishing and processing the two sources of data uncertainty, it steadily improves the performance of the classification model, enhancing both its reliability and robustness, effectively promoting the development of uncertainty estimation research. Furthermore, the target loss function is a component or plug-in used to replace the traditional loss function; it can be easily integrated / deployed into the classification model as a component without changing its original model structure, reducing the learning cost for developers and providing a simple and effective model training method for the classification model. On the other hand, the embodiments of this application do not require the deletion of labeled noisy samples and difficult samples in the training dataset, thus retaining the influence of these two types of samples on the classification model, thereby improving the robustness of the classification model. This allows the classification model to better adapt to and handle labeled noisy samples and difficult samples, reducing misjudgments and misclassifications, and lowering risks. This has important practical significance for many key fields such as autonomous driving, medical diagnosis, and financial risk assessment.

[0141] Based on the foregoing Figure 2 As can be seen from the complete process of the classification model training method shown, during the model training stage, the embodiment of this application can use the annotation uncertainty information and feature uncertainty information corresponding to the sample as indicators of the first loss information during model training. This ensures that while reducing the classification model's focus on noisy labeled samples, it strengthens the classification model's focus on difficult samples, thereby improving the model performance and robustness. This classification model training method provided by the embodiment of this application can be applied to any category classification scenario and can improve the accuracy of the recognition results in the corresponding category classification scenario. The following uses image recognition as an example to introduce the specific implementation process of the classification model training method in an image recognition scenario; where, in the image recognition scenario, the samples in the training dataset are image samples, and the classification model is an image classification model.

[0142] Please see Figure 4 , Figure 4 A flowchart illustrating another classification model training method provided in an exemplary embodiment of this application is shown. Figure 4 The illustrated process mainly concerns the training method for classification models in image recognition scenarios. This process can be executed by a computer device, which can be... Figure 1 The terminal and / or server in the system shown. The classification model training method may include, but is not limited to, steps S401-S406:

[0143] S401: Obtain the data uncertainty information corresponding to each image sample in the training dataset.

[0144] As can be seen from the aforementioned formula (2), the data uncertainty information corresponding to the image sample is represented as P(y|x,θ); therefore, the data uncertainty information corresponding to the image sample can be calculated based on the image sample and the model parameters of the image classification model during the current model training process.

[0145] Specifically, after a round of model training, the computer device obtains the model parameters θ of the image classification model. These model parameters θ are the parameters used by the image classification model to classify and recognize image samples during this round of model training. Then, given an image sample x and model parameters θ, the computer device predicts the probability distribution of the true class label corresponding to the image sample x. This probability distribution is the data uncertainty information corresponding to the image sample.

[0146] S402: Based on the source of data uncertainty, perform sample decoupling processing on the data uncertainty information corresponding to each image sample to obtain the annotation uncertainty information and feature uncertainty information corresponding to each image sample.

[0147] After obtaining the data uncertainty information corresponding to the image sample, this embodiment of the application further decouples the data uncertainty information corresponding to the image sample into annotation uncertainty information and feature uncertainty information according to the source of data uncertainty. As mentioned above, the sources of data uncertainty include labeled noisy image samples and difficult image samples in the training dataset; when the source of data uncertainty is labeled noisy image samples, the error of the image classification model mainly comes from the incorrect labeling of the labeled noisy image samples, so the image sample corresponds to annotation uncertainty information; when the source of data uncertainty is difficult image samples, the error of the image classification model mainly comes from the image classification model, so the image sample corresponds to feature uncertainty information.

[0148] The calculation formula for the uncertain information in the annotation is shown in the aforementioned formula (3). It can be seen that after one round of model training, the computer device will obtain the predicted class labels corresponding to the image samples; then, given an image sample x and its corresponding predicted class labels, the computer device will... In the case of [condition], predict the probability distribution of the true class label y corresponding to the image sample x. This probability distribution is the annotation uncertainty information corresponding to the image sample.

[0149] Similarly, according to the formula (4) above, the calculation formula for the characteristic uncertainty information is... As can be seen, after one round of model training, the computer device obtains the model parameters θ of the image classification model; then, given an image sample x and the model parameters θ, the computer device predicts the predicted class label corresponding to that image sample x. The probability distribution is the feature uncertainty information corresponding to the image sample.

[0150] S403: Based on the annotation uncertainty information and feature uncertainty information corresponding to each image sample, as well as the first loss information of the image classification model, calculate the second loss information of the image classification model.

[0151] Based on the aforementioned steps, after decoupling the data uncertainty information of the image samples into annotation uncertainty information and feature uncertainty information according to the source of data uncertainty, this embodiment of the application also uses the loss function traditionally used in image classification models to calculate the first loss information of the image classification model; if the traditionally used loss function is the cross-entropy loss function, then the first loss information of the image classification model is calculated based on the cross-entropy loss function, the predicted class label and the true class label corresponding to the image sample.

[0152] Furthermore, in this embodiment, the annotation uncertainty information and feature uncertainty information corresponding to the image sample are used as the loss weights of the first loss information to calculate the second loss information of the image classification model. Specifically, the annotation uncertainty information corresponding to each image sample, the feature uncertainty information corresponding to each image sample, and the first loss information of the image classification model are weighted and calculated to obtain the second loss information of the image classification model.

[0153] The advantage of using the first and second loss information corresponding to the image samples as the loss weights of the first loss information to calculate the new loss information of the image classification model is that: using the labeled noisy image samples and hard images in the training dataset as indicators of the first loss information, and training the model in the direction of reducing the influence of labeled noisy image samples on the image classification model and increasing the influence of hard samples on the image classification model during the backpropagation process, can effectively reduce the focus of the image classification model on labeled noisy image samples and strengthen the focus of the image classification model on hard image samples, thereby improving the model performance of the image classification model.

[0154] S404: Train the image classification model based on the second loss information to obtain the trained classification model.

[0155] After calculating the second loss information of the image classification model based on the aforementioned steps S401-S403, the computer device can perform backpropagation (i.e., the aforementioned backward propagation) on the image classification model based on this second loss information. The aim is to adjust the model parameters of the image classification model according to the second loss information to obtain the trained image classification model. Specifically, the computer device adjusts the model parameters of the image classification model based on the second loss information. This includes adjusting the model parameters in a way that progressively reduces the second loss information. For example, it adjusts the model parameters by increasing the loss weights for feature uncertainty (i.e., the feature uncertainty information corresponding to the image sample) and decreasing the loss weights for annotation uncertainty (i.e., the annotation uncertainty information corresponding to the image sample). This results in the adjusted image classification model paying more attention to difficult image samples and less attention to image samples with label noise. Increasing the focus of image classification models on difficult image samples can help them have better predictive capabilities for such samples. Similarly, reducing the focus of image classification models on image samples with noisy labels can help them be less affected by mislabeled image samples during training, thereby improving their predictive capabilities.

[0156] Furthermore, in the actual model training process, the image classification model will be trained through multiple rounds of iterative training using the training dataset, such as 1000 rounds of iterative training, in order to optimize the model prediction ability of the image classification model round by round through multiple rounds of iterative model training.

[0157] As can be seen from the specific implementation process shown in steps S401-S404 above, in the image recognition scenario, when this application embodiment targets the image classification model, it identifies labeled noisy image samples and difficult image samples in the training dataset, and distinguishes the impact of labeled noisy image samples and difficult image samples on the image classification model, which greatly improves the model prediction ability of the image classification model, thereby improving the accuracy of image content recognition in the image recognition scenario.

[0158] It should be noted that the specific implementation process shown in steps S401-S404 is the same as that described above. Figure 2 The specific implementation processes shown in steps S201-S204 of the illustrated embodiment are similar; the difference lies in: Figure 4The illustrated embodiment demonstrates model training for an image classification model within an image recognition scenario. In practical applications, the model training and inference processes may differ depending on the specific category classification scenario. Due to space limitations, this embodiment does not elaborate on the specific process of training the classification model for each category classification scenario; however, it is explained here.

[0159] S405: Acquire the image to be recognized.

[0160] S406: Call the image classification model trained by the model to perform classification and recognition processing on the image and obtain the recognition result of the image.

[0161] After training the image classification model for the image recognition scenario based on the aforementioned steps S401-S404, if the computer device receives an image to be recognized, such as an image to be published in a video application or an image to be sent in a social application, the computer device will call the trained image classification model to perform classification and recognition processing on the image, obtaining the image recognition result. The image recognition result indicates whether the image content matches the category corresponding to the image classification model. For example, if the image classification model is used to determine whether the image content is a cat, then the recognition result obtained by the image classification model can indicate that the image matches the label "cat," or indicate that the image does not match the label "cat."

[0162] It should be noted that the embodiments of this application do not limit the model structure, model type, and labels of the image classification model. For example, the image classification model can be a binary classifier that identifies the label "cat" as mentioned above, or it can be a multi-classifier that can identify multiple labels (such as cat, dog, and bird).

[0163] In summary, the embodiments of this application support the effective identification and differentiation of the impact of label-noisy image samples and difficult image samples on the image classification model when training the image classification model according to the source of data uncertainty. This aims to increase the image classification model's attention to difficult image samples and reduce its attention to label-noisy image samples, thereby avoiding the reduction in model robustness caused by discarding label-noisy samples and expanding the application scope of the image classification model in image recognition scenarios.

[0164] The methods of the embodiments of this application have been described in detail above. To facilitate better implementation of the above-described solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below. In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0165] Figure 5 This illustration shows a schematic diagram of a classification model training apparatus provided in an exemplary embodiment of this application; the classification model training apparatus can be used to perform... Figure 2 and Figure 4 Some or all of the steps in the method embodiments shown. Please refer to [link / reference]. Figure 5 The uncertainty sources of the prediction results output by this classification model include: labeled noisy samples and hard samples in the training dataset. The device includes the following units:

[0166] The acquisition unit 501 is used to acquire the data uncertainty information corresponding to each sample in the training dataset. The data uncertainty information is used to characterize: the degree of influence of the sample on the prediction uncertainty of the prediction result; the prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model.

[0167] The processing unit 502 is used to perform sample decoupling processing on the data uncertainty information corresponding to each sample according to the source of data uncertainty, so as to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample; the annotation uncertainty information is used to describe the probability information of the sample being a labeled noise sample, and the feature uncertainty information is used to describe the probability information of the sample being a hard sample;

[0168] The processing unit 502 is also used to calculate the second loss information of the classification model based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model;

[0169] The processing unit 502 is also used to train the classification model based on the second loss information to obtain the trained classification model; the trained classification model is applied to category classification scenarios, which include at least one of the following: image recognition scenario, autonomous driving scenario, risk assessment scenario, medical diagnosis scenario, and email classification scenario.

[0170] In one implementation, the training dataset includes samples and their corresponding true class labels; the processing unit 502, when acquiring data uncertainty information corresponding to each sample in the training dataset, is specifically used for:

[0171] Obtain the model parameters of the classification model. The model parameters are the parameters used by the classification model to classify and identify samples in this round of model training.

[0172] Based on model parameters and samples, predict the probability distribution of the true class label corresponding to the sample;

[0173] The probability distribution of the true class label corresponding to the sample serves as the data uncertainty information corresponding to the sample.

[0174] In one implementation, the processing unit 502 is used to perform sample decoupling processing on the data uncertainty information corresponding to each sample according to the source of data uncertainty, and to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample. Specifically, it is used to:

[0175] When the source of data uncertainty is labeled noisy samples, label prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information corresponding to each sample; and,

[0176] When the source of data uncertainty is difficult samples, parameter prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample.

[0177] In one implementation, the training dataset includes samples and their corresponding ground truth class labels; the processing unit 502 is used to perform label prediction processing on the data uncertainty information corresponding to each sample, and to obtain the annotation uncertainty information corresponding to each sample, specifically for:

[0178] Obtain the predicted class label corresponding to the sample. The predicted class label is obtained by the classification model through the classification and recognition process of the sample.

[0179] Based on the sample and the predicted class label, predict the probability distribution of the true class label corresponding to the sample;

[0180] The probability distribution of the true class labels corresponding to the samples serves as the annotation uncertainty information for the samples.

[0181] In one implementation, the processing unit 502 is used to perform parameter prediction processing on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample, specifically for:

[0182] Obtain the predicted class label corresponding to the sample. The predicted class label is obtained by the classification model through the classification and recognition process of the sample.

[0183] Obtain the model parameters of the classification model. The model parameters are the parameters used by the classification model to classify and identify samples in this round of model training.

[0184] Based on model parameters and samples, predict the probability distribution of the predicted class label corresponding to the sample;

[0185] The probability distribution of the predicted class label corresponding to the sample serves as the feature uncertainty information corresponding to the sample.

[0186] In one implementation, the processing unit 502, when calculating the second loss information of the classification model based on the annotation uncertainty information corresponding to each sample, the feature uncertainty information corresponding to each sample, and the first loss information of the classification model, specifically performs the following:

[0187] The second loss information of the classification model is obtained by weighting the annotation uncertainty information, the feature uncertainty information, and the first loss information of the classification model for each sample.

[0188] In one implementation, the processing unit 502, when training the classification model based on the second loss information to obtain the trained classification model, specifically performs the following:

[0189] Based on the second loss information, the model parameters of the classification model are adjusted so that the adjusted classification model pays more attention to difficult samples and less attention to samples with label noise.

[0190] The parameter-adjusted classification model is iteratively trained using the training dataset to obtain the trained classification model.

[0191] In one implementation, when the category classification scenario is an image recognition scenario, the training dataset includes image samples, and the classification model is an image classification model; the sources of data uncertainty in the prediction results output by the image classification model include: labeled noisy image samples and difficult image samples in the training dataset;

[0192] When the category classification scenario is email classification, the training dataset includes email text samples, and the classification model is an email classification model. The sources of data uncertainty in the prediction results output by the email classification model include: labeled noisy email text and difficult email text in the training dataset.

[0193] In one implementation, where the category classification scenario is the image recognition scenario, the processing unit 502 is further configured to:

[0194] Acquire the image to be identified;

[0195] The image classification model trained by the model is invoked to perform classification and recognition processing on the image, and the recognition result is obtained. The recognition result is used to describe the category to which the image content belongs.

[0196] According to one embodiment of this application, Figure 5 The various units in the classification model training device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the classification model training device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units. According to another embodiment of this application, the device can be run on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), RAM, and ROM, capable of performing functions such as... Figure 2 and Figure 4 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 5 The diagram illustrates a classification model training apparatus and a classification model training method for implementing embodiments of this application. The computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the same medium, and run therein.

[0197] In this embodiment, the data uncertainty source of the prediction result output by the classification model is decoupled into labeled noise samples and hard samples in the training dataset. After obtaining the prediction result of the sample output by the classification model in any round of model iteration training, the computer device obtains the data uncertainty information corresponding to each sample in the training dataset. This data uncertainty information is used to characterize the degree of influence of the sample on the prediction uncertainty of the prediction result output in this round of iteration training. The prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model, such as 80% unreliability. In order to distinguish, identify and process the impact of labeled noise samples and hard samples in the training dataset on the classification model during the model training phase, this embodiment supports the computer device to split the data uncertainty information corresponding to each sample into annotation uncertainty information and feature uncertainty information according to the source of data uncertainty after obtaining the data uncertainty information corresponding to each sample. Among them, the annotation uncertainty information characterizes the probability information that the sample is a labeled noise sample, while the feature uncertainty information characterizes the probability information that the sample is a hard sample, thereby realizing the effective identification of labeled noise samples and hard samples in the training dataset. Then, based on the annotation uncertainty and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model, the second loss information of the classification model is calculated. That is, the annotation uncertainty and feature uncertainty information corresponding to the sample are used as indicators of the first loss information during model training. Specifically, the annotation uncertainty information is used as the loss weight of the label noise samples relative to the first loss information, and the feature uncertainty information is used as the loss weight of the hard samples relative to the first loss information. In this way, when training the classification model based on the second loss information, the model training can be carried out in the direction of reducing the loss weight of the label noise samples relative to the first loss information and increasing the loss weight of the hard samples relative to the first loss information. The aim is to reduce the classification model's attention to the label noise samples, that is, to reduce the attention to the mislabeled samples, and to strengthen the classification model's attention to the hard samples, so as to help the classification model improve its prediction ability for hard samples, thereby improving the model performance and robustness of the classification model.

[0198] Figure 6 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. Please refer to... Figure 6The computer device includes a processor 601, a communication interface 602, and a computer-readable storage medium 603. The processor 601, communication interface 602, and computer-readable storage medium 603 can be connected via a bus or other means. The communication interface 602 is used to receive and send data. The computer-readable storage medium 603 can be stored in the computer device's memory and is used to store computer programs, including program instructions. The processor 601 is used to execute the program instructions stored in the computer-readable storage medium 603. The processor 601 (or CPU (Central Processing Unit)) is the computing and control core of the computer device, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve corresponding method flows or corresponding functions.

[0199] This application embodiment also provides a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the processing system of the computer device. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by the processor 601, which may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0200] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor 601 loads and executes one or more instructions stored in the computer-readable storage medium to implement the corresponding steps in the above-described classification model training method embodiment; the sources of data uncertainty in the prediction results output by the classification model include: label noise samples and hard samples in the training dataset; specifically, the processor 601 loads and executes one or more instructions in the computer-readable storage medium for the following steps:

[0201] Obtain the data uncertainty information for each sample in the training dataset. The data uncertainty information is used to characterize: the degree of influence of the sample on the prediction uncertainty of the prediction result; the prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model.

[0202] Based on the source of data uncertainty, sample decoupling processing is performed on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample; the annotation uncertainty information is used to describe the probability information of the sample being a labeled noise sample, and the feature uncertainty information is used to describe the probability information of the sample being a hard sample;

[0203] Based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model, the second loss information of the classification model is calculated.

[0204] The classification model is trained based on the second loss information to obtain the trained classification model. The trained classification model is applied to category classification scenarios, which include at least one of the following: image recognition scenario, autonomous driving scenario, risk assessment scenario, medical diagnosis scenario, and email classification scenario.

[0205] In one implementation, the training dataset includes samples and their corresponding true class labels; one or more instructions in the computer-readable storage medium are loaded by the processor 601 and, when executing to obtain uncertain information about the data corresponding to each sample in the training dataset, specifically perform the following steps:

[0206] Obtain the model parameters of the classification model. The model parameters are the parameters used by the classification model to classify and identify samples in this round of model training.

[0207] Based on model parameters and samples, predict the probability distribution of the true class label corresponding to the sample;

[0208] The probability distribution of the true class label corresponding to the sample serves as the data uncertainty information corresponding to the sample.

[0209] In one implementation, when one or more instructions in the computer-readable storage medium are loaded and executed by the processor 601 to perform sample decoupling processing on the data uncertainty information corresponding to each sample according to the source of data uncertainty, and to obtain the annotation uncertainty information and feature uncertainty information corresponding to each sample, the following steps are specifically performed:

[0210] When the source of data uncertainty is labeled noisy samples, label prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information corresponding to each sample; and,

[0211] When the source of data uncertainty is difficult samples, parameter prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample.

[0212] In one implementation, the training dataset includes samples and their corresponding ground truth class labels. When one or more instructions in a computer-readable storage medium are loaded by the processor 601 and executed to perform label prediction processing on the uncertain data information corresponding to each sample, resulting in uncertain annotation information for each sample, the following steps are specifically performed:

[0213] Obtain the predicted class label corresponding to the sample. The predicted class label is obtained by the classification model through the classification and recognition process of the sample.

[0214] Based on the sample and the predicted class label, predict the probability distribution of the true class label corresponding to the sample;

[0215] The probability distribution of the true class labels corresponding to the samples serves as the annotation uncertainty information for the samples.

[0216] In one implementation, when one or more instructions in the computer-readable storage medium are loaded by the processor 601 and executed to perform parameter prediction processing on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample, the following steps are specifically performed:

[0217] Obtain the predicted class label corresponding to the sample. The predicted class label is obtained by the classification model through the classification and recognition process of the sample.

[0218] Obtain the model parameters of the classification model. The model parameters are the parameters used by the classification model to classify and identify samples in this round of model training.

[0219] Based on model parameters and samples, predict the probability distribution of the predicted class label corresponding to the sample;

[0220] The probability distribution of the predicted class label corresponding to the sample serves as the feature uncertainty information corresponding to the sample.

[0221] In one implementation, when one or more instructions in the computer-readable storage medium are loaded by the processor 601 and executed to calculate the second loss information of the classification model based on the annotation uncertainty information corresponding to each sample, the feature uncertainty information corresponding to each sample, and the first loss information of the classification model, the following steps are specifically performed:

[0222] The second loss information of the classification model is obtained by weighting the annotation uncertainty information, the feature uncertainty information, and the first loss information of the classification model for each sample.

[0223] In one implementation, when one or more instructions in the computer-readable storage medium are loaded by the processor 601 and executed to train the classification model based on the second loss information to obtain the trained classification model, the following steps are specifically performed:

[0224] Based on the second loss information, the model parameters of the classification model are adjusted so that the adjusted classification model pays more attention to difficult samples and less attention to samples with label noise.

[0225] The parameter-adjusted classification model is iteratively trained using the training dataset to obtain the trained classification model.

[0226] In one implementation, when the category classification scenario is an image recognition scenario, the training dataset includes image samples, and the classification model is an image classification model; the sources of data uncertainty in the prediction results output by the image classification model include: labeled noisy image samples and difficult image samples in the training dataset;

[0227] When the category classification scenario is email classification, the training dataset includes email text samples, and the classification model is an email classification model. The sources of data uncertainty in the prediction results output by the email classification model include: labeled noisy email text and difficult email text in the training dataset.

[0228] In one implementation, the category classification scenario is the image recognition scenario, and one or more instructions in the computer-readable storage medium are loaded by the processor 601 and further executed as follows:

[0229] Acquire the image to be identified;

[0230] The image classification model trained by the model is invoked to perform classification and recognition processing on the image, and the recognition result is obtained. The recognition result is used to describe the category to which the image content belongs.

[0231] Based on the same inventive concept, the principle and beneficial effects of the computer device provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the classification model training method in the embodiments of this application in solving the problem. Please refer to the principle and beneficial effects of the method implementation. For the sake of brevity, they will not be repeated here.

[0232] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned classification model training method.

[0233] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0234] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0235] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A classification model training method, characterized in that, The sources of data uncertainty in the prediction results output by the classification model include: labeled noisy samples and hard samples in the training dataset, and the method includes: Obtain the data uncertainty information corresponding to each sample in the training dataset. The data uncertainty information is used to characterize: the degree of influence of the sample on the prediction uncertainty of the prediction result; the prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model. According to the sources of data uncertainty, sample decoupling processing is performed on the data uncertainty information corresponding to each sample to obtain annotation uncertainty information and feature uncertainty information corresponding to each sample; the annotation uncertainty information is used to describe the probability information that the sample is the labeled noise sample, and the feature uncertainty information is used to describe the probability information that the sample is the hard sample; Based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, and the first loss information of the classification model, the second loss information of the classification model is calculated; the first loss information is calculated based on the prediction result of the classification model for each sample and the true class label of each sample. The classification model is trained based on the second loss information to obtain the trained classification model; the trained classification model is applied to category classification scenarios, which include at least one of the following: image recognition scenario, customer classification scenario, risk assessment scenario, medical diagnosis scenario, and email classification scenario.

2. The method as described in claim 1, characterized in that, The training dataset includes the samples and the corresponding true class labels of the samples; The step of obtaining the data uncertainty information corresponding to each sample in the training dataset includes: Obtain the model parameters of the classification model, which are the parameters used by the classification model to classify and identify the samples in this round of model training; Based on the model parameters and the sample, predict the probability distribution of the true class label corresponding to the sample; The probability distribution of the true class label corresponding to the sample serves as the data uncertainty information corresponding to the sample.

3. The method as described in claim 1 or 2, characterized in that, The step involves decoupling the data uncertainty information for each sample according to the source of the data uncertainty, to obtain the annotation uncertainty information and feature uncertainty information for each sample, including: When the source of data uncertainty is the label noise samples, label prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information corresponding to each sample; and... When the source of data uncertainty is the difficult sample, parameter prediction processing is performed on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample.

4. The method as described in claim 3, characterized in that, The training dataset includes the samples and their corresponding ground truth class labels; the label prediction processing of the data uncertainty information corresponding to each sample to obtain the annotation uncertainty information corresponding to each sample includes: Obtain the predicted class label corresponding to the sample, wherein the predicted class label is obtained by the classification model performing classification and recognition processing on the sample; Based on the sample and the predicted class label, predict the probability distribution of the true class label corresponding to the sample; The probability distribution of the real class label corresponding to the sample serves as the annotation uncertainty information corresponding to the sample.

5. The method as described in claim 3, characterized in that, The step of performing parameter prediction processing on the data uncertainty information corresponding to each sample to obtain the feature uncertainty information corresponding to each sample includes: Obtain the predicted class label corresponding to the sample, wherein the predicted class label is obtained by the classification model performing classification and recognition processing on the sample; Obtain the model parameters of the classification model, which are the parameters used by the classification model to classify and identify the samples in this round of model training; Based on the model parameters and the sample, predict the probability distribution of the predicted class label corresponding to the sample; The probability distribution of the predicted class label corresponding to the sample serves as the feature uncertainty information corresponding to the sample.

6. The method as described in claim 1 or 2, characterized in that, The step of calculating the second loss information of the classification model based on the annotation uncertainty information corresponding to each sample, the feature uncertainty information corresponding to each sample, and the first loss information of the classification model includes: The second loss information of the classification model is obtained by weighting the annotation uncertainty information, the feature uncertainty information, and the first loss information of the classification model for each sample.

7. The method as described in claim 1 or 2, characterized in that, The step of training the classification model based on the second loss information to obtain the trained classification model includes: Based on the second loss information, the model parameters of the classification model are adjusted so that the adjusted classification model pays more attention to the difficult samples and less attention to the labeled noisy samples. The training dataset is used to iteratively train the classification model with adjusted parameters to obtain the trained classification model.

8. The method as described in claim 1, characterized in that, When the category classification scenario is the image recognition scenario, the training dataset includes image samples, and the classification model is an image classification model; the sources of data uncertainty in the prediction results output by the image classification model include: labeled noisy image samples and difficult image samples in the training dataset; When the category classification scenario is the email classification scenario, the samples included in the training dataset are email texts, and the classification model is an email classification model; the sources of data uncertainty in the prediction results output by the email classification model include: labeled noisy email texts and difficult email texts in the training dataset.

9. The method as described in claim 8, characterized in that, The category classification scenario is the image recognition scenario, and the method further includes: Acquire the image to be identified; The image classification model trained by the model is invoked to perform classification and recognition processing on the image, and the recognition result of the image is obtained; the recognition result is used to describe the category to which the image content included in the image belongs.

10. A classification model training device, characterized in that, The sources of data uncertainty in the prediction results output by the classification model include: labeled noisy samples and hard samples in the training dataset, and the device includes: The acquisition unit is used to acquire data uncertainty information corresponding to each sample in the training dataset. The data uncertainty information is used to characterize: the degree of influence of the sample on the prediction uncertainty of the prediction result; the prediction uncertainty is used to characterize the reliability of the prediction result output by the classification model. The processing unit is configured to perform sample decoupling processing on the data uncertainty information corresponding to each sample according to the source of data uncertainty, to obtain annotation uncertainty information and feature uncertainty information corresponding to each sample; the annotation uncertainty information is used to describe the probability information that the sample is the label noise sample, and the feature uncertainty information is used to describe the probability information that the sample is the hard sample; The processing unit is further configured to calculate the second loss information of the classification model based on the annotation uncertainty information and feature uncertainty information corresponding to each sample, as well as the first loss information of the classification model; the first loss information is calculated based on the prediction result of the classification model for each sample and the true class label of each sample. The processing unit is further configured to train the classification model based on the second loss information to obtain the trained classification model; the trained classification model is applied to category classification scenarios, which include at least one of the following: image recognition scenario, autonomous driving scenario, risk assessment scenario, medical diagnosis scenario, and email classification scenario.

11. A computer device, characterized in that, include: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the classification model training method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the classification model training method as described in any one of claims 1-9.

Images