Classification model and vehicle image classification model training method, device and equipment

Abstract

The present disclosure provides a classification model and a training method, device and equipment of a vehicle image classification model, and relates to the technical field of artificial intelligence such as deep learning, automatic driving and intelligent transportation. The specific implementation scheme is: obtaining a classification model and a test set; performing category prediction on the test set by the classification model to obtain a first predicted category corresponding to each test sample; determining a first labeling proportion corresponding to each labeled category based on the first predicted category and a labeled category, and determining first training samples corresponding to each labeled category from a data set to be labeled based on the first labeling proportion of each labeled category; and training the classification model based on the first training samples corresponding to each labeled category. By predicting the test set by the classification model, determining the first labeling proportion corresponding to each labeled category, and optimizing the number proportion of the training samples corresponding to each labeled category based on the first labeling proportion, the training effect of the classification model can be improved.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence technology such as deep learning, autonomous driving, and intelligent transportation, and in particular to a classification model and a training method, apparatus, and equipment for a vehicle image classification model. Background Technology

[0002] In machine learning, the quality of data determines the upper limit of the technology, and algorithms can only infinitely approach this limit. To improve the performance of the algorithm model, data is continuously mined, and then the mined data is labeled and used for model training.

[0003] In current technology, a common approach is to send all the mined data for labeling. This is reasonable in the early stages of model training. However, as the algorithm model is optimized, this data mining method has a clear problem. In addition to the high cost of labeling, it may lead to an imbalance in the number of categories in the training sample set, resulting in a long-tail distribution. As a result, the model performs particularly well in some categories and poorly in others. Summary of the Invention

[0004] This disclosure provides a training method, apparatus, and device for classification models and vehicle image classification models.

[0005] According to a first aspect of this disclosure, a method for training a classification model is provided, comprising: acquiring a classification model and a test set, wherein the test set contains test samples of multiple labeled categories, and each labeled category has at least one test sample; performing category prediction on the test set using the classification model to obtain a first predicted category corresponding to each test sample in the test set; determining a first submission ratio corresponding to each labeled category based on the first predicted category and the labeled category corresponding to each test sample, and determining a first training sample corresponding to each labeled category from a dataset to be submitted for bidding based on the first submission ratio of each labeled category, wherein the number of first training samples corresponding to each labeled category is determined according to the first submission ratio of the corresponding labeled category; and training the classification model based on the first training samples corresponding to each labeled category.

[0006] According to a first aspect of this disclosure, a method for training a vehicle image classification model is provided, comprising: acquiring a vehicle image classification model and a vehicle image test set, wherein the vehicle image test set contains image test samples of multiple image annotation categories, and each image annotation category has at least one image test sample; performing category prediction on the vehicle image test set using the vehicle image classification model to obtain the image prediction category corresponding to each image test sample in the vehicle image test set; determining the submission ratio of image training samples corresponding to each image annotation category based on the image prediction category and image annotation category corresponding to each image test sample, and determining the image training samples corresponding to each image annotation category from the dataset to be submitted for bidding based on the submission ratio of image training samples of each image annotation category, wherein the number of image training samples corresponding to each image annotation category is determined according to the submission ratio of image training samples of the corresponding image annotation category; and training the vehicle image classification model based on the image training samples corresponding to each image annotation category.

[0007] According to a third aspect of this disclosure, a training apparatus for a classification model is provided, comprising: an acquisition module for acquiring a classification model and a test set, the test set containing test samples of multiple labeled categories, each labeled category having at least one test sample; a prediction module for performing category prediction on the test set using the classification model to obtain a first predicted category corresponding to each test sample in the test set; a determination module for determining a first submission ratio corresponding to each labeled category based on the first predicted category and the labeled category corresponding to each test sample, and determining a first training sample corresponding to each labeled category from a dataset to be submitted for labeling based on the first submission ratio of each labeled category, wherein the number of first training samples corresponding to each labeled category is determined according to the first submission ratio of the corresponding labeled category; and a training module for training the classification model based on the first training samples corresponding to each labeled category.

[0008] According to a fourth aspect of this disclosure, a training apparatus for a vehicle image classification model is provided, comprising: a receiving module for acquiring a vehicle image classification model and a vehicle image test set, wherein the vehicle image test set contains image test samples of multiple image annotation categories, and at least one image test sample for each image annotation category; a testing module for performing category prediction on the vehicle image test set using the vehicle image classification model to obtain the image prediction category corresponding to each image test sample in the vehicle image test set; a calculation module for determining the image training sample submission ratio corresponding to each image annotation category based on the image prediction category and image annotation category corresponding to each image test sample, and determining the image training samples corresponding to each image annotation category from the dataset to be submitted for submission based on the image training sample submission ratio of each image annotation category, wherein the number of image training samples corresponding to each image annotation category is determined according to the image training sample submission ratio of the corresponding image annotation category; and an updating module for training the vehicle image classification model based on the image training samples corresponding to each image annotation category.

[0009] According to a fifth aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a training method for a classification model as described in one aspect of the above-described embodiment.

[0010] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein a computer program / instruction is stored thereon, the computer instructions being used to cause the computer to execute the training method of the classification model described in the above-mentioned embodiment.

[0011] According to a seventh aspect of this disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the training method for the classification model described in one aspect of the embodiments above.

[0012] By using a classification model to predict the test set, determining the first submission ratio for each labeled category, and then optimizing the proportion of training samples for each labeled category based on the first submission ratio, the training effect of the classification model can be improved.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0015] Figure 1 A flowchart illustrating a training method for a classification model provided in an embodiment of this disclosure;

[0016] Figure 2 A flowchart illustrating a training method for another classification model provided in an embodiment of this disclosure;

[0017] Figure 3 A flowchart illustrating a training method for another classification model provided in an embodiment of this disclosure;

[0018] Figure 4 A flowchart illustrating a training method for another classification model provided in an embodiment of this disclosure;

[0019] Figure 5 A flowchart illustrating a training method for another classification model provided in an embodiment of this disclosure;

[0020] Figure 6 A flowchart illustrating a training method for another classification model provided in an embodiment of this disclosure;

[0021] Figure 7 A schematic flowchart illustrating a training method for a vehicle image classification model provided in an embodiment of this disclosure;

[0022] Figure 8 A schematic diagram of the structure of a training device for a classification model provided in an embodiment of this disclosure;

[0023] Figure 9 A schematic diagram of the structure of a training device for a vehicle image classification model provided in an embodiment of this disclosure;

[0024] Figure 10 This is a block diagram of an electronic device for a training method of a classification model according to an embodiment of the present disclosure. Detailed Implementation

[0025] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0026] The following description, with reference to the accompanying drawings, outlines the classification model and the training method, apparatus, and device for the vehicle image classification model according to embodiments of this disclosure.

[0027] Deep learning (DL) is a new research direction in the field of machine learning (ML). It was introduced into machine learning to bring it closer to its original goal—artificial intelligence. Deep learning learns the inherent laws and hierarchical representations of sample data. The information gained during this learning process greatly aids in the interpretation of data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition far exceeding previous related technologies.

[0028] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It involves both hardware and software technologies. AI hardware technologies generally include computer vision, speech recognition, natural language processing, and related technologies such as deep learning, big data processing, and knowledge graphs.

[0029] Intelligent driving essentially involves cognitive engineering of attention attraction and distraction, mainly comprising three aspects: network navigation, autonomous driving, and human intervention. The prerequisites for intelligent driving are that the selected vehicle meets the dynamic requirements of driving, the onboard sensors can acquire relevant visual and auditory signals and information, and the corresponding servo systems are controlled through cognitive computing.

[0030] Figure 1 This is a flowchart illustrating a training method for a classification model provided in an embodiment of this disclosure.

[0031] like Figure 1 As shown, the training method for this classification model may include:

[0032] S101, Obtain the classification model and test set. The test set contains test samples of multiple labeled categories, with at least one test sample for each labeled category.

[0033] It should be noted that the execution entity of the classification model training method in this embodiment can be a hardware device with data processing capabilities and / or the necessary software to drive the hardware device. Optionally, the execution entity may include a workstation, server, computer, user terminal, and other intelligent devices. The user terminal includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, and vehicle terminals.

[0034] It should be noted that the classification model in this embodiment can process various objects, such as speech classification models, text classification models, and image classification models, etc., without any limitation. It is understood that the type of test samples in the test set will also differ depending on the classification model. For example, the test samples corresponding to the speech classification model are test samples containing sound data, the test samples corresponding to the text classification model are natural language test samples, and the test samples corresponding to the image classification model are test samples containing image data, etc. Optionally, the classification model may also include other forms of test samples; for example, the text classification model may include non-natural language test samples and natural language test samples, etc.

[0035] It should be noted that the test set in this embodiment is pre-defined. This test set includes multiple label categories, each corresponding to at least one test sample. The label categories are pre-defined and can be manually set or obtained through data mining; no limitations are imposed here. For example, the label categories can be red, blue, and purple, and can also include people, animals, and buildings, etc.

[0036] Understandably, in order to better test the performance of classification models, the labeled categories include all the categories that the classification model can output. This allows training to be performed based on the output categories of each classification model, preventing misjudgments of model performance due to some output categories not appearing in the test set.

[0037] S102, use a classification model to predict the category of the test set to obtain the first predicted category corresponding to each test sample in the test set.

[0038] It should be noted that the first test category is the category output by the classification model after the test samples in the test set are input. The first test category can be the same as or different from the labeled category. When the first test category is the same as the labeled category, the classification model is considered to have made a successful prediction. When the first test category is different from the labeled category, the classification model is considered to have failed to make a prediction.

[0039] In this embodiment of the disclosure, a test set is used to test the functionality of the classification model. The classification model predicts the performance of test samples in the test set, and the performance of the current classification model is determined by the labeled category and the first predicted category of the test sample.

[0040] S103. Based on the first predicted category and the annotation category corresponding to each test sample, determine the first submission ratio corresponding to each annotation category, and based on the first submission ratio of each annotation category, determine the first training samples corresponding to each annotation category from the dataset to be submitted for annotation. The number of first training samples corresponding to each annotation category is determined according to the first submission ratio of the corresponding annotation category.

[0041] It should be noted that the first submission ratio refers to the proportion of each labeled category in the dataset to be submitted. After obtaining the first predicted category, the model performance can be evaluated by comparing and processing the first predicted category with the labeled categories of the test samples. At the same time, the first submission ratio of each labeled category can be adjusted based on the test results of each labeled category.

[0042] By adjusting the first labeling ratio, the training sample set can be optimized, preventing imbalance of labeled categories in the training sample set during the training process and improving the training effect of the classification model.

[0043] It should be noted that the training set in this embodiment is pre-set and can be manually created or obtained through other means; no limitation is made here. It should also be noted that the methods for obtaining the training set may differ depending on the type of classification model. For example, when the classification model is a speech classification model, training samples containing sound data can be obtained by collecting sound samples and performing sound analysis and processing on the sound samples; when the classification model is a text classification model, text training samples can be obtained by collecting text and processing the text; and when the classification model is an image classification model, training samples containing image data can be obtained by collecting image samples and performing image processing on the image samples.

[0044] In this embodiment of the disclosure, according to the design requirements of model training, a dataset size to be submitted for bidding can be predetermined. Based on a first bidding ratio, the number of first training samples corresponding to each label category is determined. Then, training samples for the corresponding label category are randomly selected from the training set based on the number of first training samples for each label category. It should be noted that the number of datasets to be submitted for bidding can be changed according to specific design requirements.

[0045] S104, The classification model is trained based on the training samples corresponding to each labeled category.

[0046] Training samples corresponding to each labeled category are input into the model to obtain the predicted category. Then, the parameters of the classification model are adjusted based on the predicted category and the labeled category of the training samples. In this embodiment, the loss function of the classification model can be determined first, and then the loss value can be obtained by inputting the predicted category and the labeled category of the training samples into the loss function. The parameters of the classification model are then adjusted based on the loss value.

[0047] In this embodiment, a classification model and a test set are first obtained. The test set contains multiple labeled categories, each corresponding to at least one test sample. Then, the classification model predicts the categories of the test set to obtain the first predicted category for each test sample. Next, based on the first predicted category and the labeled category for each test sample, a first submission ratio for each labeled category is determined. Based on the first submission ratio for each labeled category, first training samples for each labeled category are determined from the dataset to be submitted for bidding. The number of first training samples for each labeled category is determined according to the first submission ratio for that category. Finally, the classification model is trained based on the training samples for each labeled category. In this way, by using the classification model to predict the test set, determining the first submission ratio for each labeled category, and optimizing the proportion of training samples for each labeled category based on the first submission ratio, the training effect of the classification model can be improved.

[0048] In this embodiment of the disclosure, different classification models may correspond to different model types, and their corresponding inputs and outputs may also be different.

[0049] Optionally, the classification model can be a text classification model, such as a multiple Bernoulli model or a multinomial model. By inputting training text, the text classification model can predict the labeled category of the training text. It should be noted that the labeled category of the text can be multiple, including the type of text, such as poetry or prose, and can also predict the type that the text represents, such as animal, human, or building.

[0050] Optionally, the classification model can be an image classification model such as LeNet, AlexNet, or GoogLeNet. By inputting training image data, the image classification model can predict the labeled category of the training image data. It should be noted that the labeled category of the training image data can be various, including the category of the training image data, such as oil painting or photograph, and also the content of the training image data, such as animal, human, or building.

[0051] Optionally, the classification model can be a Hidden Markov Model, a Deep Neural Network, etc. By inputting training speech data, the speech classification model can predict the labeled category of the training speech data. It should be noted that the labeled category of the training speech data can be various, including categories such as songs, recordings, and dialogues, as well as content recognition such as meetings, conversations, and arguments.

[0052] In the above embodiments, based on the first predicted category and the labeled category corresponding to each test sample, the first labeling ratio corresponding to each labeled category is determined respectively, and it can also be achieved through... Figure 2 To further explain, the method includes:

[0053] S201, for any labeled category, determine the positive or negative sample identifier corresponding to the test sample based on the first predicted category and the labeled category of the test sample.

[0054] It should be noted that since the test samples often contain multiple label categories, for any given label category, the test samples under that label category are positioned as positive samples, while the test samples under other label categories are defined as negative samples.

[0055] In this embodiment of the disclosure, there can be multiple positive and negative sample identifiers. Since the model has errors in prediction, there is a possibility that a positive sample is predicted as a negative sample, a positive sample is predicted as a positive sample, a negative sample is predicted as a positive sample, and a negative sample is predicted as a negative sample. Therefore, different situations can be defined as different positive and negative sample identifiers.

[0056] For example, taking a text classification model, when the labeled category to be evaluated is "building", if the test sample is a house, and the prediction result is "building", then the positive or negative sample label can be a positive sample predicted as positive; if the prediction result is an animal, then the positive or negative sample label can be a positive sample predicted as negative. If the test sample is a dog, and the prediction result is "building", then the positive or negative sample label can be a negative sample predicted as negative; if the prediction result is an animal, then the positive or negative sample label can be a negative sample predicted as positive.

[0057] S202, determine the sample score corresponding to the labeled category based on positive and negative sample identifiers.

[0058] It should be noted that the sample score corresponding to the labeled category is the prediction performance score of the test sample corresponding to the labeled category in the model. The higher the sample score, the better the prediction performance of the test sample corresponding to that labeled category in the model.

[0059] In this embodiment of the disclosure, after obtaining the positive and negative sample identifiers, the sample score corresponding to the labeled category can be determined based on the quantity ratio of each positive and negative sample identifier.

[0060] Optionally, the ratio of positive to negative sample labels can be calculated using a sample scoring function to determine the sample score corresponding to the labeled category. It should be noted that this sample scoring function can be pre-defined and can be changed according to actual design needs; no limitations are imposed here.

[0061] It should be noted that different types of classification models may have different sample scoring functions. For example, speech classification models, text classification models, and image classification models may have different scoring functions.

[0062] S203, determine the first submission ratio corresponding to the labeled category based on the sample score.

[0063] In this embodiment of the disclosure, after obtaining the sample scores corresponding to the labeled categories, the first submission ratio corresponding to the labeled category can be determined by the magnitude of the sample scores. For labeled categories with higher sample scores, it can be considered that the model's prediction effect for samples of that labeled category is better, and the first submission ratio corresponding to that labeled category can be appropriately reduced. For labeled categories with lower sample scores, it can be considered that the model's prediction effect for samples of that labeled category is worse, and the first submission ratio corresponding to that labeled category can be appropriately increased.

[0064] In this embodiment of the disclosure, the sample score can be compared with the sample score threshold. When the sample score is greater than the sample score threshold, the model can be considered to have a good prediction effect on the sample of the labeled category. When the sample score is less than the sample score threshold, the model can be considered to have a poor prediction effect on the sample of the labeled category.

[0065] Optionally, the sample scores corresponding to each labeling category can be input into the first submission ratio determination function to obtain the first submission ratio corresponding to each labeling category.

[0066] In this embodiment, firstly, for any labeled category, based on the first predicted category and the labeled category of the test sample, the positive and negative sample identifiers corresponding to the test sample are determined. Then, based on the positive and negative sample identifiers, the sample score corresponding to the labeled category is determined. Finally, based on the sample score, the first submission ratio corresponding to the labeled category is determined. Thus, by calling the current classification model to predict the test sample, classifying the test samples corresponding to the labeled categories based on the prediction results, and calculating the sample score corresponding to each labeled category based on the classification results, the first submission ratio corresponding to each labeled category is accurately calculated. This provides a basis for subsequent optimization of the quantity ratio of each labeled category in the training samples, while also reducing the training sample collection cycle and improving collection efficiency.

[0067] In this embodiment of the disclosure, the positive and negative sample identifiers corresponding to the test samples can be determined by the following method: In response to a test sample being a positive sample and the first predicted category of the test sample being the same as the labeled category to be evaluated, the positive and negative sample identifier corresponding to the test sample is determined as a first positive and negative sample identifier; in response to a test sample being a negative sample and the first predicted category of the test sample being the same as the labeled category to be evaluated, the positive and negative sample identifier corresponding to the test sample is determined as a second positive and negative sample identifier; in response to a test sample being a positive sample and the first predicted category of the test sample being different from the labeled category to be evaluated, the positive and negative sample identifier corresponding to the test sample is determined as a third positive and negative sample identifier. Thus, by determining the positive and negative sample identifiers of each test sample based on the test results, the test samples can be accurately classified, and a data foundation can be provided for subsequently determining the sample scores corresponding to each labeled category.

[0068] In the above embodiments, the sample score corresponding to the labeled category is determined based on the positive and negative sample identifiers, and can also be determined by... Figure 3 To further explain, the method includes:

[0069] S301, determine the first number of test samples in the labeled test samples where the positive and negative sample identifiers are the first positive and negative sample identifiers, the second number of test samples in the labeled test samples where the positive and negative sample identifiers are the second positive and negative sample identifiers, and the third number of test samples in the labeled test samples where the positive and negative sample identifiers are the third positive and negative sample identifiers.

[0070] It should be noted that the first number of test samples with the first positive and negative sample identifier, the second number of test samples with the second positive and negative sample identifier in the test samples of the labeled category, and the third number of test samples with the third positive and negative sample identifier in the test samples of the labeled category in this embodiment are specific to one of the labeled categories. The first number of test samples with the first positive and negative sample identifier, the second number of test samples with the second positive and negative sample identifier in the test samples of the labeled category, and the third number of test samples with the third positive and negative sample identifier in the test samples of the labeled category are all different for different labeled categories.

[0071] Obtaining the first number of test samples with the first positive and negative sample identifier, the second number of test samples with the second positive and negative sample identifier in the test samples of the labeled category, and the third number of test samples with the third positive and negative sample identifier in the test samples of the labeled category is to statistically analyze the proportion and number of each positive and negative sample identifier under the labeled category, so as to facilitate the analysis of the prediction effect of the labeled category.

[0072] S302, determine the precision of the labeled category based on the first quantity and the second quantity, and determine the recall of the labeled category based on the first quantity and the third quantity.

[0073] In this embodiment of the disclosure, precision represents the probability that a sample is actually positive among all samples predicted as positive, and can be used to measure the accuracy of the classification model in predicting the labeled category. Recall represents the probability that a sample is predicted as positive among samples that are actually positive, and can be used to measure this. It can be calculated using the following formula:

[0074]

[0075] TP represents the first number of test samples identified by the first positive and negative sample identifier, FP represents the second number of test samples identified by the second positive and negative sample identifier, and FN represents the third number of test samples identified by the third positive and negative sample identifier.

[0076] S303, determines the sample score for the labeled category based on precision and recall.

[0077] The sample score for the labeled category is determined based on precision and recall. This can be achieved by determining a first coefficient based on the product of precision and recall, a second coefficient based on the sum of precision and recall, and then determining the sample score for the labeled category based on the ratio of the first coefficient to the second coefficient.

[0078] In this embodiment of the disclosure, by calculating the sample score of the labeled category, the prediction effect of the current labeled category in the classification model can be determined, which can serve as an important basis for subsequent adjustment of the training sample ratio.

[0079] In this embodiment of the disclosure, the formula for determining the sample score of the labeled category is:

[0080]

[0081] Where TP is the first number of test samples identified by the first positive and negative sample identifiers, FP is the second number of test samples identified by the second positive and negative sample identifiers, FN is the third number of test samples identified by the third positive and negative sample identifiers, P is precision, R is recall, and F1 score is the sample score. Here, P represents the probability that a sample predicted as positive is actually positive among all samples predicted as positive, R represents the probability that a sample actually positive is predicted as positive among samples actually positive, and F1 score considers both precision and recall. By analyzing F1, a balance can be achieved by maximizing both simultaneously.

[0082] The first submission ratio corresponding to the labeling category is determined based on the sample score, and can also be achieved through... Figure 4 To further explain, the method includes:

[0083] S401, determine the initial submission ratio for each labeled category based on sample scores.

[0084] In this embodiment of the disclosure, the formula for calculating the initial bid submission ratio is:

[0085]

[0086] in, Let P be the score of the sample corresponding to the i-th labeled category, where e is the natural constant. i This represents the initial submission ratio corresponding to the i-th labeling category.

[0087] S402, determine the third coefficient based on the sum of the initial submission ratios for each labeling category.

[0088] In this embodiment of the disclosure, the formula for calculating the third coefficient is as follows:

[0089]

[0090] Among them, P j is the initial submission ratio corresponding to the j-th annotation category, N is the total number of annotation categories, and b is the third coefficient.

[0091] S403, determine the first submission ratio for each annotation category based on the ratio of the initial submission ratio to the third coefficient.

[0092] It should be noted that the initial submission ratio for each annotation category may be greater than 1 and cannot be used as the final allocation ratio. In this embodiment, the initial submission ratio can be normalized to generate a first submission ratio that can intuitively represent the first submission ratio for each annotation category of the training samples. It should be noted that the first submission ratio is a constant less than 1, and the sum of the first submission ratios for all annotation categories is 1.

[0093] In this embodiment of the disclosure, the formula for determining the first submission ratio for each labeling category is:

[0094]

[0095] in, P represents the first submission ratio corresponding to the i-th annotation category. i Let be the initial submission ratio for the i-th labeled category, and b be the third coefficient. The formula shows that the higher the sample score for the i-th labeled category, the smaller the initial submission ratio, and consequently, the smaller the first submission ratio.

[0096] After obtaining the sample scores corresponding to the labeled categories, the initial submission ratio for each labeled category can be determined based on the sample scores. Then, a third coefficient is determined based on the sum of the initial submission ratios for each labeled category. Finally, the first submission ratio for each labeled category is determined based on the ratio of the initial submission ratio to the third coefficient. By normalizing the initial submission ratios and converting them into the first submission ratio, the proportion of each labeled category can be more intuitively represented, facilitating subsequent proportional division and mining of training samples.

[0097] In the above embodiments, the classification model is trained based on the first training samples corresponding to each labeled category, and can also be trained by... Figure 5 To further explain, the method includes:

[0098] S501, Input the first training sample into the classification model to obtain the second predicted category corresponding to the first training sample.

[0099] For specific steps, please refer to the content in the above embodiments, which will not be repeated here.

[0100] S502, generate a loss value based on the labeled category and the second predicted category corresponding to the first training sample, and adjust the classification model based on the loss value.

[0101] In this embodiment, the loss function of the classification model is first determined, and then the loss value is calculated based on the second predicted category and the labeled category corresponding to the first training sample. It should be noted that the calculated loss value can be compared with a loss threshold. When the loss value is greater than the loss threshold, the classification model is considered to have a good prediction effect on the samples of the corresponding labeled category. If the loss value is less than the loss threshold, the classification model is considered to have a poor prediction effect on the samples of the corresponding labeled category, and the parameters of the classification model need to be adjusted.

[0102] It should be noted that different classification models may have different loss functions, and no restrictions are imposed here. The specific loss function should be determined based on the actual design requirements.

[0103] The classification model is trained using the first training samples corresponding to each labeled category. The first training samples are input into the model to obtain the second predicted category. A loss value is generated based on the labeled category and the second predicted category, and the classification model is adjusted based on this loss value. By adjusting the model using the loss value and then evaluating the performance of the adjusted model on a test set, dynamic model adjustment can be achieved, increasing the efficiency of model training and improving the final result.

[0104] It should be noted that the adjusted model is still not necessarily the optimal model, and performance testing of the adjusted classification model is required. In this embodiment of the disclosure, the optimal classification model obtained through training can be achieved through... Figure 6 To further explain, the method includes:

[0105] S601 uses the adjusted classification model to predict the category of the test set in order to obtain the third predicted category corresponding to each test sample in the test set.

[0106] In this embodiment of the disclosure, the performance of the adjusted classification model can be tested using a test set. By inputting the test set into the classification model, the third predicted category corresponding to each test sample can be obtained.

[0107] S602, determine the sample score corresponding to each annotation category based on the third prediction category and the annotation category.

[0108] The specific steps for determining the sample scores corresponding to each label category based on the third prediction category and the label category can be found in the above embodiments, and will not be repeated here.

[0109] S606, in response to the fact that the sample score of any annotation category is less than the corresponding preset threshold, based on each third prediction category and annotation category, the second submission ratio corresponding to each annotation category is determined, and the second training samples corresponding to each annotation category are determined from the dataset to be submitted for annotation, wherein the number of second training samples corresponding to each annotation category is determined according to the target submission ratio of the corresponding annotation category.

[0110] It should be noted that the prediction threshold is preset and can be changed according to actual design needs. In this embodiment, if the sample score of the labeled category is greater than or equal to the corresponding preset threshold, the classification model can be considered to have a good prediction effect on that labeled category.

[0111] Similarly, if the sample score of a labeled category is less than the corresponding preset threshold, it can be considered that the classification model's prediction effect for that labeled category is poor, and the proportion of training samples for that labeled category needs to be increased. The second submission ratio for each labeled category can be determined separately, and the second training samples for each labeled category can be determined from the dataset to be submitted for labeling. The number of second training samples for each labeled category is determined based on the target submission ratio for that labeled category. The specific steps for determining the second training samples can be referred to in the above embodiments, and will not be repeated here.

[0112] S604. Based on the second training samples corresponding to each labeled category, train the classification model and repeat the above steps until the sample scores of each labeled category are greater than or equal to the corresponding preset threshold.

[0113] In this embodiment, the classification model first predicts the category of the test set using an adjusted classification model to obtain the third predicted category for each test sample in the test set. Then, based on the third predicted category and the labeled category, the sample score corresponding to each labeled category is determined. Subsequently, in response to the situation where the sample score of any labeled category is less than a corresponding preset threshold, a second submission ratio corresponding to each labeled category is determined based on each third predicted category and labeled category. Second training samples corresponding to each labeled category are then determined from the dataset to be submitted for submission. The number of second training samples corresponding to each labeled category is determined according to the target submission ratio for that labeled category. Finally, the classification model is trained based on the second training samples corresponding to each labeled category until the sample score of each labeled category is greater than the corresponding preset threshold. The classification model's performance is tested using a test set, and the ratio of the number of training samples corresponding to each labeled category is continuously adjusted based on the classification prediction results until training is complete. This dynamic adjustment continuously optimizes the ratio of the number of training samples corresponding to each labeled category, achieving better model training results.

[0114] In the scenario of autonomous driving, it is necessary to identify and classify objects in front of the vehicle to distinguish them. At this time, image classification models are used to predict and classify the images collected in front to achieve various purposes. For example, it can determine whether the object in front is an obstacle, and it can also classify the traffic lights in front to determine the indication of the traffic lights.

[0115] In this embodiment of the disclosure, the vehicle image classification model needs to be trained first. Figure 7 This is a flowchart illustrating a training method for a vehicle image classification model provided in an embodiment of the present disclosure.

[0116] like Figure 7 As shown, the training method for this vehicle image classification model includes:

[0117] S701, Obtain the vehicle image classification model and the vehicle image test set. The vehicle image test set contains image test samples of multiple image annotation categories, with at least one image test sample for each image annotation category.

[0118] In this embodiment, the vehicle image classification model is pre-defined. The vehicle image classification model can be a LeNet model, an AlexNet model, or a GoogLeNet model, etc. There are no limitations here, and the specific model needs to be set according to the actual design requirements.

[0119] It should be noted that the vehicle image test set includes multiple image test samples containing image data.

[0120] It should be noted that the image test samples can be collected manually or during actual vehicle driving. Optionally, they can also be selected from an image test sample library. This image test sample library is a database of image test samples that can be stored in the storage space of an electronic device for easy retrieval when needed.

[0121] The labeling categories for image test samples can be various, and there are no restrictions here. For example, the labeling categories can be obstacles, pedestrians, traffic lights, etc.

[0122] S702 uses a vehicle image classification model to predict the category of a vehicle image test set, thereby obtaining the predicted image category for each image test sample in the vehicle image test set.

[0123] For specific instructions, please refer to the content in the above embodiments, which will not be repeated here.

[0124] S703, based on the image prediction category and image annotation category corresponding to each image test sample, determine the proportion of image training samples to be submitted for each image annotation category, and based on the proportion of image training samples to be submitted for each image annotation category, determine the image training samples corresponding to each image annotation category from the dataset to be submitted for submission. The number of image training samples corresponding to each image annotation category is determined according to the proportion of image training samples to be submitted for submission of the corresponding image annotation category.

[0125] For specific instructions, please refer to the content in the above embodiments, which will not be repeated here.

[0126] S704 trains a vehicle image classification model based on image training samples corresponding to each image annotation category.

[0127] For specific instructions, please refer to the content in the above embodiments, which will not be repeated here.

[0128] In this embodiment, a vehicle image classification model and a vehicle image test set are first obtained. The vehicle image test set contains image test samples of multiple image annotation categories, with at least one image test sample for each image annotation category. Then, the vehicle image classification model is used to predict the category of the vehicle image test set to obtain the predicted image category corresponding to each image test sample in the vehicle image test set. Then, based on the predicted image category and the image annotation category corresponding to each image test sample, the proportion of image training samples sent for each image annotation category is determined. Based on the proportion of image training samples sent for each image annotation category, image training samples corresponding to each image annotation category are determined from the dataset to be sent for bidding. The number of image training samples corresponding to each image annotation category is determined according to the proportion of image training samples sent for the corresponding image annotation category. Finally, the vehicle image classification model is trained based on the image training samples corresponding to each image annotation category. Therefore, by using a vehicle image classification model to predict the test set of vehicle images, determining the proportion of images submitted for each labeled category, and optimizing the proportion of training samples for each labeled category based on the proportion of images submitted for each labeled category, the training effect of the vehicle image classification model can be improved, thereby enhancing the performance of the vehicle image classification model in practical applications and improving the user experience.

[0129] In practical applications, vehicles can collect image data that needs to be classified. For example, they can collect images of the front, rear, and sides of the vehicle, process the image data, and then call a trained vehicle image classification model. The processed image data is input into the vehicle image classification model for category prediction, thus achieving various uses. For example, by classifying image data, it is possible to determine whether an object in the image is an obstacle, identify road signs ahead, and determine the indication of traffic lights ahead.

[0130] In this embodiment of the disclosure, the vehicle can acquire image data through an image acquisition device. The image acquisition device may include various types, such as an in-vehicle camera or an in-vehicle camera, etc., without any limitation.

[0131] After obtaining the image data, it is necessary to process the image data. For example, filtering out poor quality image data, adjusting the image data to a uniform format and size, etc. There are no restrictions here, and the specifics need to be determined according to the actual design requirements.

[0132] Corresponding to the classification model training methods provided in the above embodiments, an embodiment of this disclosure also provides a classification model training device. Since the classification model training device provided in this disclosure corresponds to the classification model training methods provided in the above embodiments, the implementation methods of the above classification model training methods are also applicable to the classification model training device provided in this disclosure, and will not be described in detail in the following embodiments.

[0133] Figure 8 This is a schematic diagram of the structure of a training device for a classification model provided in an embodiment of this disclosure. Figure 8 As shown, the training device 800 for the classification model includes: an acquisition module 810, a prediction module 820, a determination module 830, and a training module 840.

[0134] The acquisition module 810 is used to acquire a classification model and a test set, wherein the test set contains test samples of multiple labeled categories, and there is at least one test sample for each labeled category.

[0135] The prediction module 820 is used to predict the category of the test set through the classification model to obtain the first predicted category corresponding to each test sample in the test set.

[0136] The determination module 830 is used to determine the first submission ratio corresponding to each annotation category based on the first prediction category and annotation category corresponding to each test sample, and to determine the first training samples corresponding to each annotation category from the dataset to be submitted based on the first submission ratio of each annotation category, wherein the number of first training samples corresponding to each annotation category is determined according to the first submission ratio of the corresponding annotation category.

[0137] The training module 840 is used to train the classification model based on the first training samples corresponding to each of the labeled categories.

[0138] In one embodiment of this disclosure, the determining module 830 is further configured to: for any labeling category, determine the positive and negative sample identifiers corresponding to the test sample based on the first predicted category and the labeling category of the test sample; determine the sample score corresponding to the labeling category based on the positive and negative sample identifiers; and determine the first submission ratio corresponding to the labeling category based on the sample score.

[0139] In one embodiment of this disclosure, the test samples include positive samples and negative samples. Positive samples are those whose sample category is the same as the label category to be evaluated, and negative samples are those whose sample category is different from the label category to be evaluated. The determining module 830 is further configured to: in response to a test sample being a positive sample and the first predicted category of the test sample being the same as the label category to be evaluated, determine the positive or negative sample identifier corresponding to the test sample as a first positive or negative sample identifier; in response to a test sample being a negative sample and the first predicted category of the test sample being the same as the label category to be evaluated, determine the positive or negative sample identifier corresponding to the test sample as a second positive or negative sample identifier; and in response to a test sample being a positive sample and the first predicted category of the test sample being different from the label category to be evaluated, determine the positive or negative sample identifier corresponding to the test sample as a third positive or negative sample identifier.

[0140] In one embodiment of this disclosure, the determining module 830 is further configured to: determine a first number of test samples in the test samples of the labeled category whose positive and negative sample identifiers are first positive and negative sample identifiers, a second number of test samples in the test samples of the labeled category whose positive and negative sample identifiers are second positive and negative sample identifiers, and a third number of test samples in the test samples of the labeled category whose positive and negative sample identifiers are third positive and negative sample identifiers; determine the precision of the labeled category based on the first number and the second number, and determine the recall of the labeled category based on the first number and the third number; and determine the sample score of the labeled category based on the precision and the recall.

[0141] In one embodiment of this disclosure, the determining module 830 is further configured to: determine a first coefficient based on the product of precision and recall; determine a second coefficient based on the sum of precision and recall; and determine a sample score for the labeled category based on the ratio of the first coefficient and the second coefficient.

[0142] In one embodiment of this disclosure, the determining module 830 is further configured to: determine the initial submission ratio of the labeling category based on the sample score; determine a third coefficient based on the sum of the initial submission ratios of each labeling category; and determine a first submission ratio of each labeling category based on the ratio of the initial submission ratio of each labeling category to the third coefficient.

[0143] In one embodiment of this disclosure, the training module 840 is further configured to: input a first training sample into a classification model to obtain a second predicted category corresponding to the first training sample; generate a loss value based on the labeled category and the second predicted category corresponding to the first training sample; and adjust the classification model based on the loss value.

[0144] In one embodiment of this disclosure, the training module 840 is further configured to: predict the category of the test set using the adjusted classification model to obtain the third predicted category corresponding to each test sample in the test set; determine the sample score corresponding to each labeled category based on the third predicted category and the labeled category; in response to the fact that the sample score of any labeled category is less than the corresponding preset threshold, determine the second submission ratio corresponding to each labeled category based on each third predicted category and the labeled category, and determine the second training samples corresponding to each labeled category from the dataset to be submitted for submission, wherein the number of second training samples corresponding to each labeled category is determined according to the target submission ratio of the corresponding labeled category; train the classification model based on the second training samples corresponding to each labeled category, and repeat the above steps until the sample scores of each labeled category are all greater than or equal to the corresponding preset threshold.

[0145] Figure 9 This is a schematic diagram of the structure of a training device for a vehicle image classification model provided in an embodiment of this disclosure. Figure 9 As shown, the training device 900 for the vehicle image classification model includes: a receiving module 910, a testing module 920, a calculation module 930, and an updating module 940.

[0146] The receiving module 910 is used to acquire a vehicle image classification model and a vehicle image test set. The vehicle image test set contains image test samples of multiple image labeling categories, with at least one image test sample for each image labeling category.

[0147] The test module 920 is used to predict the category of the vehicle image test set through the vehicle image classification model, so as to obtain the image prediction category corresponding to each image test sample in the vehicle image test set.

[0148] The calculation module 930 is used to determine the proportion of image training samples sent for each image annotation category based on the image prediction category and image annotation category corresponding to each image test sample, and to determine the image training samples corresponding to each image annotation category from the dataset to be sent for annotation based on the proportion of image training samples sent for each image annotation category. The number of image training samples corresponding to each image annotation category is determined according to the proportion of image training samples sent for the corresponding image annotation category.

[0149] The update module 940 is used to train the vehicle image classification model based on the image training samples corresponding to each image annotation category.

[0150] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0151] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0152] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0153] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to computer programs / instructions stored in read-only memory (ROM) 1002 or loaded from storage unit 1006 into random access memory (RAM) 1003. RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0154] Multiple components in device 1000 are connected to I / O interface 1005, including: input units 1006 such as keyboard, mouse, etc.; output units 1007 such as various types of displays, speakers, etc.; storage units 1008 such as disks, optical discs, etc.; and communication units 1009 such as network cards, modems, wireless transceivers, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0155] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as methods for training classification models. For example, in some embodiments, the methods for training classification models can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1006. In some embodiments, part or all of the computer program / instructions can be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program / instructions are loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the methods for training classification models described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform a training method for the classification model by any other suitable means (e.g., by means of firmware).

[0156] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implementations in one or more computer programs / instructions that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0158] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0160] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0161] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. The client-server relationship is created by computer programs / instructions running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0162] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in the disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this document does not impose any restrictions.

[0163] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training an image classification model, comprising: obtaining an image classification model and an image test set, the image test set containing a plurality of image test samples of labeled categories, each of the image test samples of the labeled categories being at least one; performing category prediction on the image test set by the classification model to obtain a first predicted category corresponding to each of the image test samples in the image test set; for any of the image labeled categories, determining a first number, a second number and a third number of image test samples in which the positive and negative sample identifiers of the image labeled category are respectively a first positive and negative sample identifier, a second positive and negative sample identifier and a third positive and negative sample identifier; determining a precision of the image labeled category based on the first number and the second number, and determining a recall of the image labeled category based on the first number and the third number; determining a sample score of the image labeled category based on the precision and the recall; determining a first submission ratio corresponding to the image labeled category based on the sample score; determining a first training sample corresponding to each of the labeled categories from an image data set to be submitted based on the first submission ratio of each of the labeled categories, wherein the number of the first training sample corresponding to each of the labeled categories is determined according to the first submission ratio of the corresponding labeled category; training the image classification model based on the first training sample corresponding to each of the labeled categories; wherein the image test sample is a positive sample, if the first predicted category of the image test sample is the same as the labeled category to be evaluated, the positive and negative sample identifier corresponding to the image test sample is determined as the first positive and negative sample identifier, if the first predicted category of the image test sample is not the same as the labeled category to be evaluated, the positive and negative sample identifier corresponding to the image test sample is determined as the third positive and negative sample identifier; the image test sample is a negative sample, and the first predicted category of the image test sample is the same as the labeled category to be evaluated, the positive and negative sample identifier corresponding to the image test sample is determined as the second positive and negative sample identifier.

2. The method of claim 1, wherein, The determination of the sample score of the labeled category based on the precision and the recall comprises: determining a first coefficient according to the product of the precision and the recall; determining a second coefficient according to the sum of the precision and the recall; determining the sample score of the labeled category according to the ratio of the first coefficient and the second coefficient.

3. The method of claim 2, wherein, The determination of the first submission ratio corresponding to the labeled category based on the sample score comprises: determining an initial submission ratio of the labeled category based on the sample score; determining a third coefficient according to the sum of the initial submission ratios of each of the labeled categories; determining the first submission ratio of each of the labeled categories according to the ratio of the initial submission ratio of each of the labeled categories and the third coefficient.

4. The method of any one of claims 1-3, wherein, The training of the image classification model based on the first training sample corresponding to each of the labeled categories comprises: inputting the first training sample into the image classification model to obtain a second predicted category corresponding to the first training sample; generate a loss value based on the first training sample corresponding label category and the second prediction category, and adjust the image classification model based on the loss value.

5. The method of claim 4, wherein, The method further comprises: performing category prediction on the image test set by the adjusted image classification model to obtain a third prediction category corresponding to each test sample in the image test set; determining a sample score corresponding to each image label category based on the third prediction category and the image label category; in response to the sample score of any image label category being less than a corresponding preset threshold, determining a second labeling proportion corresponding to each image label category based on each of the third prediction category and the label category, and determining a second training sample corresponding to each image label category from the image data set to be labeled, wherein the number of the second training sample corresponding to each image label category is determined according to the target labeling proportion of the corresponding image label category; training the image classification model based on the second training sample corresponding to each image label category, and repeating the above steps until the sample score of each image label category is greater than or equal to the corresponding preset threshold.

6. An image classification model training device, comprising: an acquisition module configured to acquire an image classification model and an image test set, wherein the image test set comprises image test samples of multiple label categories, and each image test sample of each label category is at least one; a prediction module configured to perform category prediction on the image test set by the image classification model to obtain a first prediction category corresponding to each image test sample in the image test set; a determination module configured to determine, for any image label category, a first number, a second number and a third number of image test samples whose positive and negative sample identifiers are a first positive and negative sample identifier, a second positive and negative sample identifier and a third positive and negative sample identifier, respectively, among the test samples of the image label category; determine a precision of the image label category based on the first number and the second number, and determine a recall of the image label category based on the first number and the third number; and determine a sample score of the image label category based on the precision and the recall; determine a first labeling proportion corresponding to each image label category based on the sample score, and determine a first training sample corresponding to each image label category from an image data set to be labeled based on the first labeling proportion of each image label category, wherein the number of the first training sample corresponding to each image label category is determined according to the first labeling proportion of the corresponding image label category; a training module configured to train the image classification model based on the first training sample corresponding to each image label category; wherein the image test sample is a positive sample, if the first prediction category of the image test sample is the same as the label category to be evaluated, the positive and negative sample identifier corresponding to the image test sample is determined as the first positive and negative sample identifier, and if the first prediction category of the image test sample is not the same as the label category to be evaluated, the positive and negative sample identifier corresponding to the image test sample is determined as the third positive and negative sample identifier. The image test sample is a negative sample, and the first predicted category of the image test sample is the same as the label category to be evaluated, and the positive and negative sample identifier corresponding to the image test sample is determined as a second positive and negative sample identifier.

7. The apparatus of claim 6, wherein, The determination module is further configured to: determine a first coefficient according to the product of the precision and the recall; determine a second coefficient according to the sum of the precision and the recall; determine the sample score of the label category according to the ratio of the first coefficient and the second coefficient.

8. The apparatus of claim 7, wherein, The determination module is further configured to: determine an initial label sending ratio of the label category based on the sample score; determine a third coefficient according to the sum of the initial label sending ratios of the label categories; determine a first label sending ratio of each label category according to the ratio of the initial label sending ratio of the label category and the third coefficient.

9. The apparatus of any one of claims 6-8, wherein, The training module is further configured to: input the first training sample into the image classification model to obtain a second predicted category corresponding to the first training sample; generate a loss value based on the label category and the second predicted category corresponding to the first training sample, and adjust the image classification model based on the loss value.

10. The apparatus of claim 9, wherein, The training module is further configured to: perform category prediction on the image test set through the adjusted image classification model to obtain a third predicted category corresponding to each test sample in the image test set; determine a sample score corresponding to each label category based on the third predicted category and the label category; in response to the sample score of any label category being less than a corresponding preset threshold, determine a second label sending ratio corresponding to each label category based on the third predicted category and the label category, and determine a second training sample corresponding to each label category from the image data set to be labeled, wherein the number of the second training sample corresponding to each label category is determined according to the target label sending ratio of the corresponding label category; train the image classification model based on the second training sample corresponding to each label category, and repeat the above steps until the sample score of each label category is greater than or equal to the corresponding preset threshold.

11. An electronic device comprising: at least one processor, and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the training method of the image classification model of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the training method of the image classification model of any one of claims 1-5.

13. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions enable the processor to perform the training method of the image classification model of any one of claims 1-5 when executed by the processor.

Images