A model training method, a category prediction method and apparatus

Abstract

The application discloses a model training method, a category prediction method, a device, a computing device and a computer readable storage medium, to solve the problem of low category prediction accuracy of the model. The training method comprises: training a category prediction first model by using an original sample object set; determining the matching degree between the label of the sample object and the predicted category of the sample object by using the trained first model; selecting sample objects meeting a preset condition from the original sample object set; and training a category prediction second model by using the selected sample objects. Since the sample objects used for training the category prediction second model are sample objects with relatively high label annotation accuracy, compared with the category prediction model trained directly based on the dirty data set, the category prediction model trained by the method provided in the application embodiment has higher accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method, a category prediction method, an apparatus, a computing device, and a computer-readable storage medium. Background Technology

[0002] Video understanding refers to the identification of content within videos. Currently, artificial intelligence (AI) models can be used for video understanding. Compared to purely manual video understanding, this method has significant advantages in processing efficiency and has thus been widely adopted in the industry.

[0003] Using AI models for video understanding requires training the model on labeled sample videos. However, when staff label the samples, subjective factors can easily influence the labeling and classification paradigm, leading to different labels for the same video. Directly using such sample videos for AI model training will not yield a highly accurate model. Summary of the Invention

[0004] This application provides a model training method to solve the problem that the accuracy of class prediction of the trained model is not high when using sample objects with inaccurate labeling.

[0005] This application also provides a model training device, a category prediction method, a category prediction device, a computing device, and a computer storage medium.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] A model training method, comprising:

[0008] The first model for category prediction is trained using the sample objects in the original sample object set to obtain the trained first model; the sample objects are the labeled objects used as training samples.

[0009] Using the trained first model, the class of the sample object is predicted to determine the degree of matching between the label of the sample object and the predicted class of the sample object.

[0010] Based on the matching degree, sample objects that meet the preset conditions are selected from the original sample object set as the first sample object;

[0011] The first sample object is used to train the second model for category prediction.

[0012] A model training method, comprising:

[0013] Obtain a recombined set of sample objects; wherein the sample objects in the recombined set of sample objects include: a first sample object that meets a preset condition, and a second sample object that does not meet the preset condition, wherein the second sample object is labeled with a preset tag;

[0014] Using the classification model trained on the first sample object as a teacher model, the class of the sample objects in the recombined sample object set is predicted to obtain the class distribution predicted by the teacher model.

[0015] The student model is used to predict the category of the sample objects in the recombined sample object set to obtain the category distribution predicted by the student model; the category distribution of the second object in the category distribution predicted by the student model is discarded to obtain the category distribution of the remaining first sample objects, which is used as the target category distribution predicted by the student model.

[0016] The loss function of the student model is determined based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution.

[0017] The student model is updated until convergence based on the loss function to obtain a trained student model.

[0018] A category prediction method, comprising:

[0019] Obtain the object to be classified for prediction;

[0020] The object is input into the category prediction model to obtain the category prediction result of the object output by the category prediction model;

[0021] The category prediction model is trained using any of the model training methods described above.

[0022] A model training device, comprising:

[0023] The first training unit is used to train the first category prediction model using sample objects from the original sample object set to obtain the trained first model; the sample objects are labeled objects used as training samples.

[0024] A prediction unit is used to predict the category of the sample object using the trained first model obtained by the first training unit, so as to determine the degree of matching between the label of the sample object and the predicted category of the sample object.

[0025] The selection unit is used to select sample objects that meet preset conditions from the original sample object set as the first sample object based on the matching degree determined by the prediction unit.

[0026] The second training unit is used to train the second category prediction model using the first sample object selected by the selection unit.

[0027] A model training device, comprising:

[0028] A sample acquisition unit is used to acquire a recombined set of sample objects; wherein the sample objects in the recombined set of sample objects include: a first sample object that meets a preset condition, and a second sample object that does not meet the preset condition, wherein the second sample object is labeled with a preset tag;

[0029] The category distribution prediction unit is used to use a category prediction model trained based on the first sample object as a teacher model to predict the category of the sample objects in the recombined sample object set, so as to obtain the category distribution predicted by the teacher model; use a student model to predict the category of the sample objects in the recombined sample object set, so as to obtain the category distribution predicted by the student model; discard the category distribution of the second object in the category distribution predicted by the student model, so as to obtain the category distribution of the remaining first sample objects, which is used as the target category distribution predicted by the student model.

[0030] The loss function determination unit is used to determine the loss function of the student model based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution.

[0031] The model training unit is used to update the student model until convergence based on the loss function determined by the loss function determination unit, so as to obtain a trained student model.

[0032] A category prediction device, comprising:

[0033] The object acquisition unit is used to acquire the object to be predicted in terms of category.

[0034] The category prediction unit is used to input the object acquired by the object acquisition unit into the category prediction model to obtain the category prediction result of the object output by the category prediction model;

[0035] The category prediction model is trained using any of the model training methods described above.

[0036] A computing device includes: a memory and a processor, wherein,

[0037] The memory is used to store programs;

[0038] The processor, coupled to the memory, is used to execute the program stored in the memory to perform any of the above-described model training methods, or to perform the above-described category prediction methods.

[0039] A computer-readable storage medium storing a computer program that, when executed by a computer, can implement any of the above-described model training methods, or can implement the above-described category prediction methods.

[0040] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0041] For the first model training method provided in this application embodiment, since the sample videos used to train the second class prediction model are selected sample videos with relatively high labeling accuracy, the class prediction model trained by the method provided in this application embodiment will have higher class prediction accuracy than the class prediction model trained directly on the dirty dataset.

[0042] Regarding the second model training method provided in this application embodiment, since the sample videos used to train the teacher model are sample objects that meet preset conditions (relatively high labeling accuracy), the teacher model trained using this method will have higher category prediction accuracy compared to the category prediction model trained directly on dirty datasets. Furthermore, by employing knowledge distillation to train the student model using this teacher model, the trained student model can inherit the high accuracy of the teacher model. Simultaneously, because the student model has higher computational efficiency than the teacher model, it is more suitable for deployment on computing devices with limited computing power, such as user terminals.

[0043] Specifically, in the first model training method, when determining the loss function, the class distribution of the second object that does not meet the preset conditions in the class distribution predicted by the student model is discarded, so that the class distribution of the remaining first sample object is obtained. This class distribution is then used as the target class distribution predicted by the student model. Furthermore, a cross-entropy function is constructed between the target class distribution and the corresponding label of the first sample object, serving as the first loss function for knowledge distillation. This ensures that the calculation basis of the first loss function is the class distribution result obtained when the student model achieves correct classification. This approach allows the first loss function to better achieve the "hard objective" of knowledge distillation, guaranteeing a better training effect for the student model and resulting in a better classification performance from the trained student model. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1a This is a flowchart illustrating the specific implementation of a model training method provided in Embodiment 1 of this application.

[0046] Figure 1b This is a schematic diagram illustrating the implementation process of knowledge distillation in Embodiment 1 of this application;

[0047] Figure 2 This is a flowchart illustrating the specific implementation of a category prediction method provided in Embodiment 2 of this application.

[0048] Figure 3 This is a schematic diagram illustrating an implementation process of applying the model training method provided in Example 1 to an actual application, as shown in Example 3 of this application.

[0049] Figure 4a This is a schematic diagram of the specific structure of a model training device provided in Embodiment 4 of this application;

[0050] Figure 4b This is a schematic diagram of the specific structure of another model training device provided in Embodiment 4 of this application;

[0051] Figure 5 This is a schematic diagram of the specific structure of a category prediction device provided in Embodiment 5 of this application;

[0052] Figure 6 This is a schematic diagram of the specific structure of a computing device provided in Embodiment 6 of the application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0055] Example 1

[0056] To address the problem that training a model using sample objects with inaccurate labels results in insufficient accuracy in class prediction, Embodiment 1 of this application provides a model training method.

[0057] It should be noted that the "object" mentioned in the embodiments of this application may include, but is not limited to, multimedia files, such as, but not limited to, one or more of the following: video, audio, images, and text.

[0058] The model described in this application embodiment can be used to predict the category of an object, and can be called a category prediction model. In particular, if the model is used to understand a video based on its content and then to predict its category, such a category prediction model can also be called a video understanding model.

[0059] Since the model training method provided in this application is universal regardless of the type of object, this application mainly uses video as the object and video understanding model as the category prediction model to introduce the method provided in this application for ease of text description and reader understanding.

[0060] Furthermore, the executing entity of this method can be a server or a server cluster, which can be local or cloud-based. Of course, the executing entity of this method can be any computing device capable of implementing the method; this application does not limit the type of computing device as the executing entity.

[0061] The following example uses the computing device used for model training as the execution subject of this model training method, combined with, for example... Figure 1a The schematic diagram shown illustrates the implementation process of the method, and provides a detailed description of the method provided in the embodiments of this application.

[0062] Step 11: Use the sample videos in the original sample video set to train the video understanding model to obtain a trained video understanding model;

[0063] The trained video understanding model mentioned here is not the final desired video understanding model, but only a model trained to assist in training the final desired video understanding model. For ease of distinction, the video understanding model trained in step 11 will be referred to as the first video understanding model, and the trained video understanding model described in step 11 will be referred to as the trained first model.

[0064] The following provides a further explanation of step 11:

[0065] The original sample video set mentioned in step 11 consists of sample videos; a sample video refers to a video that has been labeled and used as a training sample. The sample videos in the original sample video set include two categories: positive sample videos and negative sample videos. In this embodiment, it is assumed that the label of positive sample videos is 1, while the label of negative sample videos is 0.

[0066] As described in the background section, the labels can be manually labeled by staff. However, this application does not limit the labeling to manual labeling.

[0067] To facilitate the introduction of this method, in the embodiments of this application, it is assumed that some sample videos in the original sample video set have inaccurate labeling, so the original sample video set can also be called a "dirty dataset".

[0068] In step 11, the construction of the first video understanding model can be based on the Temporal Shift Module (TSM) framework, and models with relatively large parameters and relatively strong learning capabilities, such as ResNet-101, can be used as the backbone model.

[0069] The following is an explanation of some of the terms used in the above description:

[0070] TSM is a module that simulates 3D modeling through time displacement, thereby achieving a balance between effect and performance and improving video understanding capabilities. It can maintain the complexity of 2D Convolutional Neural Networks (CNN) but achieve the effect of 3D CNN.

[0071] ResNet, short for Residual Neural Network, has a structure that can greatly accelerate the training of neural networks and significantly improve the accuracy of the trained models.

[0072] ResNet-101 refers to a ResNet network with 101 layers. Compared to networks like ResNet-18 with only 18 layers, ResNet-101 is a deep residual neural network. While maintaining high accuracy, ResNet-101's computational complexity is also within an acceptable range compared to networks like ResNet-200. Therefore, in this embodiment, ResNet-101 is chosen as the backbone model of the framework.

[0073] In this embodiment, ResNet-101 is used as the backbone model to extract image features from each frame of the sample video; TSM, as the framework, is used to fuse image features and inter-frame information to achieve understanding of the sample video.

[0074] Step 12: Using the trained first model, perform video category prediction on the sample videos in the original sample video set to determine the degree of matching between the label of the sample video and the predicted category of the sample video;

[0075] Regarding step 12, in one specific implementation, the trained first model is used to predict the video category of the sample videos in the original sample video set. This can yield the confidence value that the "predicted category of the sample video" belongs to the "category represented by the label of the sample video", or the confidence value that the "predicted category of the sample video" matches the "category represented by the label of the sample video". This is called the category confidence value.

[0076] The category confidence score is a probability value that falls within the interval [0, 1]. The larger the value, the greater the probability that the sample video belongs to the positive sample video.

[0077] The class confidence score of any sample video can characterize the degree of matching between the label of the sample video and the predicted class of the sample video. Specifically, if the label of the sample video is 1, and assuming that the class confidence score of the sample video is relatively high, such as 0.99, this value indicates that the sample video is very likely to be a positive sample video. Thus, this value characterizes the degree of matching between the label "1" of the sample video and the predicted class of the sample video, which means that the labeling accuracy of the sample video with the label "1" is relatively high.

[0078] If the label of the sample video is 0, and assuming that the confidence value of the corresponding category of the sample video is relatively high, such as 0.99, this indicates that the sample video is more likely to be a positive sample video. This means that the matching degree between the label "0" of the sample video and the predicted category of the sample video is relatively low, which means that the labeling accuracy of the sample video currently labeled with "0" is low.

[0079] It should be noted that, in the embodiments of this application, the determination of the matching degree between the label and the predicted category of the sample video is not limited to the method of determining the value of the category confidence. Those skilled in the art can use any method that can determine the matching degree to implement step 12.

[0080] Step 13: Based on the matching degree determined by performing Step 12, select sample videos (referred to as the first sample videos) that meet the preset conditions from the original sample video set;

[0081] The condition mentioned here, which meets the preset conditions, may include: the degree of matching meets the preset matching degree requirements.

[0082] After determining the degree of matching between the labels of the sample videos and the predicted categories, it becomes clear which sample videos are likely to have inaccurate labels. Therefore, inaccurately labeled sample videos can be removed from the original sample video set, while relatively accurately labeled sample videos are retained / selected as sample videos for training the second video understanding model in subsequent step 14.

[0083] This selection method ensures that the sample videos used to train the second video understanding model are relatively accurately labeled, thereby making the video understanding model obtained by training the second video understanding model more accurate.

[0084] Following the previous example of determining the matching degree by determining the value of the category confidence score, in one implementation, step 13 can be achieved as follows:

[0085] On the one hand, based on the category confidence values ​​of the positive sample videos in the original sample video set, positive sample videos with a quantity that meets the first proportion are selected in descending order of value.

[0086] As explained above, a higher class confidence value indicates a greater likelihood that the sample video is a positive sample. Therefore, in step 13, the positive sample videos can be sorted in descending order of their class confidence values. Positive sample videos ranked higher have more accurate labeling, allowing for the selection of, for example, the top 80% (i.e., the first percentage) of positive sample videos as part of the subsequent training video understanding of the second model.

[0087] Similarly, for negative sample videos in the original video set, they can be sorted in ascending order of their category confidence values. The higher the ranking of a negative sample video, the lower its probability of being a positive sample video; that is, the higher its current labeling accuracy. Therefore, negative sample videos, for example, those ranking in the top 80% (which can be called the second percentage, and the second percentage can be the same as or different from the first percentage), can be selected as part of the sample videos for subsequent training to understand the second model.

[0088] The selected positive and negative sample videos together constitute the sample videos used to train the second video understanding model.

[0089] Alternatively, in one implementation, a first class confidence threshold and a second class confidence threshold can be preset, with the first class confidence threshold > the second class confidence threshold. Thus, for positive sample videos in the original video set, positive sample videos with a class confidence value greater than the first class confidence threshold can be selected as part of the sample videos for subsequent training of the second understanding model; for negative sample videos in the original video set, negative sample videos with a class confidence value smaller than the second class confidence threshold can be selected as another part of the sample videos for subsequent training of the second understanding model. The selected positive and negative sample videos together constitute the sample videos used to train the second video understanding model.

[0090] Step 14: Use the selected sample video (i.e. the first sample video) to train the second video understanding model to obtain a video understanding model for video understanding.

[0091] The second video understanding model can have the same structure as the first video understanding model described above. Of course, the structures of the second and first video understanding models can also differ.

[0092] In one embodiment of this application, step 14 includes: training a second video understanding model using selected sample videos, and using the trained second video understanding model as the final video understanding model. The final video understanding model can be deployed to, for example, a user terminal device to perform video understanding tasks.

[0093] As analyzed above, since the sample videos used to train the second video understanding model are selected sample videos with relatively high labeling accuracy, the video understanding model trained by the method provided in this application will have higher video understanding accuracy compared to the video understanding model trained directly on the dirty dataset.

[0094] By performing steps 11 to 14 above, the method provided in Embodiment 1 of this application yields a highly accurate video understanding model, which can be referred to as a "large model".

[0095] Considering the computational efficiency requirements for deployable video understanding models, Embodiment 1 of this application further proposes a "small model" that is more computationally efficient and more suitable for deployment, based on a large model and using knowledge distillation.

[0096] In one implementation, this small model can be obtained by improving step 14. Specifically, the improved step 14 may include the following sub-steps:

[0097] Sub-step 141: Using the first sample video selected by executing step 13, train the second video understanding model to obtain the trained second model;

[0098] Sub-step 142: Using the sample videos in the recombined sample video set, the trained second model is used as the teacher model, and the student model is trained using knowledge distillation technology to obtain the trained student model, which is the final video understanding model for video understanding.

[0099] Combination such as Figure 1b The diagram shown illustrates the implementation process of knowledge distillation. The following provides some explanation of sub-step 142:

[0100] The recombined sample video set mentioned in sub-step 142 consists of the first sample video selected by executing step 13 and the second sample video labeled with preset tags. Figure 1b In this context, `dadaset_mix` represents the recombined sample video set, where `dataset_clean` contains labeled data and represents the sample videos selected by executing step 13; `dataset_unlabel` contains unlabeled data and represents the second sample video.

[0101] The second sample video refers to a sample video in the original sample video set whose matching degree does not meet the preset matching degree requirement, that is, a sample video that does not meet the preset conditions. Specifically, the remaining sample videos in the original sample video set, excluding the sample videos selected by performing step 13, can be used as the second sample video mentioned here.

[0102] In this embodiment of the application, the preset label annotating the second sample video is a label used to distinguish the target video sample from the selected sample video. Taking the label of the positive sample video in the selected sample video as "1" and the label of the negative sample video as "0" as an example, the preset label mentioned here can be, for example, "-100".

[0103] Based on the recombined sample video set consisting of the first sample video and the second sample video labeled with preset tags, in sub-step 142, the trained second model can be used as the teacher model, and the student model can be trained using knowledge distillation technology. The explanation of the knowledge distillation technology is as follows:

[0104] In chemistry, distillation is an effective method for separating components with different boiling points. The general steps involve first heating the mixture to vaporize the lower-boiling-point component, then cooling and condensing it to separate the target substance. In artificial intelligence, knowledge distillation refers to first training a large, powerful model (often called a teacher model) – essentially raising the temperature; then transferring the knowledge contained within it to smaller models (often called student models) – equivalent to restoring the temperature during the testing phase, thus extracting the knowledge from the large, powerful model and transferring it to the smaller model. This process is thus figuratively called knowledge distillation.

[0105] The goal of knowledge distillation is to make the distribution of the softmax layer output of the student model sufficiently close to the distribution of the softmax layer output of the teacher model. From a technical implementation perspective, the specific approach of knowledge distillation is as follows: increase the temperature parameter (a hyperparameter in the softmax function) of the teacher model's softmax layer to obtain a suitable soft target set (i.e., the prediction results output by the teacher model). Then, for the student model to be trained, use the same temperature parameter value to match the soft target set of the teacher model, and use it as part of the overall objective function of the student model to induce the training of the student model and achieve knowledge transfer.

[0106] Generally, neural networks use a softmax layer to calculate the probability value for each class. The softmax function is:

[0107]

[0108] Where α represents a category; q α This is the value of the softmax function, which is the probability value of classifying the identified object into category α, output by the softmax layer.

[0109] z α The prediction result with the number α is actually obtained after the neural network makes a prediction for a classification problem. α belongs to a value in the range [1, M]; M is the total number of categories corresponding to the classification problem.

[0110] z β The prediction result that can be obtained by the neural network after predicting the classification problem is given, and the value of β is in the range of [1, M].

[0111] T is the temperature parameter, and it is usually set to 1. Setting T to a larger number, such as 20, will result in a softer probability distribution.

[0112] In this embodiment, to obtain a more computationally efficient and deployable "small model," a small model can be obtained by training the second video understanding model using the sample videos selected in step 13. This trained second model can then be used as a teacher model, and knowledge distillation can be employed to obtain a small model, which serves as the final video understanding model for video understanding. The specific implementation process is as follows:

[0113] On the one hand, the student model is used to predict the video category of the recombined sample video set (specifically, the selected sample videos in the recombined sample video set) to obtain the category distribution predicted by the student model, denoted as prob_student.

[0114] The student model can be a model with fewer parameters and higher computational efficiency than the teacher model, such as ResNet18, EfficientNetB0, or MobileNet (e.g., MobileNetV2), and embedded into the TSM framework as the student model. Figure 1b As shown, MobileNetV2 can be selected as the backbone model and embedded into the TSM model framework to obtain the student model model_student.

[0115] When predicting video categories, based on the recombined sample video set, some sample videos can be randomly selected, and the student model can be used to predict the category distribution of the videos.

[0116] After the student model `model_student` has predicted the category distribution `prob_student` for some randomly selected sample videos, if any of these randomly selected sample videos have a label of "-100" when using the student model `model_student`—meaning the label of the sample video has a relatively low match with the video content—then, when calculating the first loss function (described later), the category distribution of the sample videos with the label "-100" can be discarded, and the category distribution of the sample videos with the label "-100" will not be used in the calculation of the first loss function. The category distribution of the remaining sample videos after discarding them all belong to the category distribution of the first sample video that meets the preset conditions.

[0117] The purpose of this is to take into account that in knowledge distillation, the first loss function corresponds to a "hard target," which is that the student model (model_student) should strive to achieve the correct classification. In this embodiment, the category distribution results of sample videos with a label of -100 are discarded in order to ensure that the calculation of the first loss function is based on the category distribution results obtained from "the student model achieving the correct classification."

[0118] On the other hand, the trained second model is used as the teacher model to predict the video categories of randomly selected sample videos from the recombined sample video set, so as to obtain the category distribution predicted by the teacher model. Let this category distribution be denoted as prob_teacher. Figure 1b As shown in the embodiment of this application, ResNet101 can be selected as the backbone model and embedded in the TSM model framework. This model serves as the second model for video understanding. It is trained through the steps shown in Figure 1 to obtain the trained second model for video understanding, which serves as the teacher model for knowledge distillation and is denoted as model_clean.

[0119] It should be noted that the above prob_student and prob_teacher are obtained by using the student model model_student and the teacher model model_clean respectively to predict the video category of the same batch of sample videos randomly selected from the recombined sample video set.

[0120] When using the teacher model (model_clean) to predict the video category of the reconstructed sample video set, it is not necessary to focus on the labels of the sample videos in the set. Instead, the teacher model (model_clean) can be used to predict the category distribution of the randomly selected sample videos from the reconstructed sample video set.

[0121] like Figure 1b As shown, after determining the class distribution prob_teacher predicted by the teacher model model_clean and the class distribution prob_student predicted by the student model model_student, the loss function of the student model model_student can be determined based on prob_teacher and prob_student; and the student model model_student is trained based on the loss function.

[0122] For the student model `model_student`, knowledge distillation involves two loss functions. The first loss function is the cross-entropy function between the predicted class distribution of the student model `model_student` and the predicted labels of the sample videos, calculated after predicting the class distribution of the student model `model_student`. The second loss function is the KL divergence (Kullback-Leibler divergence, a metric for measuring the similarity between two probability distributions) loss function calculated between the class distribution predicted by the student model `model_student` and the class distribution predicted by the teacher model `model_clean`.

[0123] In one embodiment of this application, the calculation of the loss function and the training of the student model model_student based on the loss function can be implemented in the following way:

[0124] As mentioned earlier, the student model `model_student` is used to predict the class distribution of some randomly selected sample videos from the recombined sample video set. After obtaining the class distribution `prob_student` predicted by the student model `model_student`, the class distribution of the sample videos with a label of -100 in `prob_student` is discarded to obtain the class distribution of the remaining sample videos (i.e., the first sample video), which is used as the target class distribution predicted by the student model `model_student`. The cross-entropy loss function between the target class distribution and the corresponding label of the first sample video is calculated and used as the first loss function.

[0125] The first loss function corresponds to the "hard target," which means that the student model, model_student, should try its best to achieve the correct classification.

[0126] The cross-entropy loss function can be expressed as follows [2]:

[0127]

[0128] in:

[0129] C represents the cross-entropy loss function;

[0130] m represents the total number of the first sample videos predicted using the student model model_student;

[0131] i represents the ID of the first sample video, with a value range of [1, m];

[0132] y represents the actual label (target value) of the first sample video;

[0133] 'a' represents the value (predicted value) output by the student model_student when predicting the first sample video. The values ​​output by the student model_student when predicting each of the first sample videos contained in some randomly selected sample videos constitute the target category distribution.

[0134] The second loss function is to use the student model (model_student) to predict the class distribution prob_student of some randomly selected sample videos from the reconstructed sample video set (the sample videos are not discarded regardless of whether the label is -100), and the teacher model (model_clean) to predict the class distribution prob_teacher of the same randomly selected sample videos. Then, the KL divergence (Kullback-Leibler divergence, a metric used to measure the similarity between two probability distributions) loss of prob_student and prob_teacher is calculated, and the KL divergence loss is used as the second loss function.

[0135] The second loss function corresponds to the "soft target". The soft target is to make the class distribution predicted by the student model model_student as close as possible to the class distribution predicted by the teacher model model_clean after adding T.

[0136] Specifically, the KL divergence loss can be calculated as shown in the following formula [3]:

[0137]

[0138] Where j is the sample video number, with a value range of [1, N], and N is the maximum number of sample videos to be predicted, which is the total number of sample videos in the recombined sample video combination.

[0139] p(x j The probability that the sample video with ID j belongs to a certain category is obtained by the student model model_student performing category prediction; the probability that the student model model_student performs category prediction on each sample video randomly selected from the recombined sample video set constitutes the category distribution prob_student predicted by the student model model_student.

[0140] q(x j The probability that the sample video with ID j belongs to a certain category is obtained by the teacher model model_clean in predicting the category of the sample video; the probability that the teacher model model_teacher obtains in predicting the category of each sample video randomly selected from the recombined sample video set constitutes the category distribution prob_teacher predicted by the teacher model model_teacher.

[0141] After obtaining the first and second loss functions, the sum of the first and second loss functions is calculated as the final loss function of the student model model_student; then, based on this final loss function, the parameters of the student model model_student are updated by gradient descent.

[0142] The process of repeatedly calculating the first and second loss functions, and then updating the parameters of the student model (model_student) using gradient descent based on the sum of the first and second loss functions, continues until the student model (model_student) converges, resulting in a smaller model (the trained student model) that is more efficient than the teacher model (model_clean).

[0143] Compared to the teacher model model_clean, the trained student model model_student has higher accuracy, higher efficiency, and lower complexity, making it more suitable for deployment, especially on devices with limited computing power, such as user terminals.

[0144] Specifically, in the first model training method, when determining the loss function, the category distribution of sample videos with a label of -100 in the category distribution predicted by the student model is discarded to obtain the category distribution of the first sample video that meets the preset conditions, which is used as the target category distribution predicted by the student model. Then, the cross-entropy function between the target category distribution and the label of the corresponding first sample object is constructed as the first loss function of knowledge distillation. This allows the calculation basis of the first loss function to be the category distribution result obtained by "the student model completing the correct classification".

[0145] This approach allows the first loss function to better achieve the "hard objective" of knowledge distillation, ensuring that the knowledge distillation process produces a better training effect on the student model, resulting in a better classification performance for the trained student model.

[0146] It should be noted that, as Figure 1bThe knowledge distillation process illustrated, including the specific training methods for the teacher model involved, may not be limited to, for example... Figure 1a As shown. That is to say, Figure 1b The process shown is the same as Figure 1a The process shown may not be strongly coupled; the two can be two relatively independent methods, or they can be combined.

[0147] Example 2

[0148] To address the problem that training a model using sample videos with inaccurate labeling results in insufficient accuracy in class prediction, Embodiment 2 of this application provides a class prediction method. This method uses a model trained using the model training method provided in Embodiment 1 to predict the class of an object to be classified, thereby improving the accuracy of class prediction.

[0149] The flowchart of the specific implementation of this method is as follows: Figure 2 As shown, it includes the following steps:

[0150] Step 21: Obtain the object to be predicted by category;

[0151] As described in Embodiment 1 of this application, the term "object" as used herein may include, but is not limited to, multimedia files, such as, but not limited to, one or more of video, audio, images, and text. Regarding video, this may include, but is not limited to, short videos and live videos produced and uploaded by users through a client to a video publishing platform, etc.

[0152] Step 22: Input the acquired object into the category prediction model to obtain the category prediction result of the object output by the category prediction model.

[0153] The aforementioned category prediction model was trained using the model training method described in Example 1.

[0154] Since the objects used for training using the model training method described in Example 1 are sample objects that meet the preset conditions (the labeling accuracy is relatively high), the category prediction model trained using the method provided in Example 1 of this application—whether it is the trained video understanding second model or the trained student model—will have higher classification accuracy than the category prediction model trained directly based on the dirty dataset. Therefore, using such a category prediction model for category prediction can correspondingly obtain category prediction results with higher accuracy.

[0155] Specifically, since the sample videos used to train the teacher model are selected from those that meet preset conditions (relatively high labeling accuracy), the teacher model trained using the method provided in this application will have higher category prediction accuracy compared to a category prediction model trained directly on a dirty dataset. Furthermore, by employing knowledge distillation to train the student model using this teacher model, the trained student model can inherit the high accuracy of the teacher model. Simultaneously, because the student model has higher computational efficiency than the teacher model, it is more suitable for deployment on computing devices with limited computing power, such as user terminals. Using such a student model as a category prediction model can efficiently obtain higher accuracy category prediction results.

[0156] Example 3

[0157] Embodiment 3 of this application aims to provide an implementation process for applying the model training method provided in Embodiment 1 to a practical application scenario.

[0158] Please refer to the attached document. Figure 3 The following is a flowchart of the specific implementation of the model training method provided in the embodiments when applied in practice. The process includes two steps: 1. Active learning; 2. Model distillation (i.e. knowledge distillation) and semi-supervised learning.

[0159] The specific implementation process of these two stages will be described in detail below.

[0160] I. Active Learning

[0161] Step 1: Train the model model_dirty based on the dirty dataset dataset_dirty. model_dirty can be referred to as the large model.

[0162] The model here is built using TSM as the framework and ResNet101 as the backbone.

[0163] ResNet101 serves as the backbone model, used to extract image features from each frame of the video, while the TSM model framework is used to fuse image features and inter-frame information, and then perform video understanding and classification based on the fusion result.

[0164] The "dirty" dataset mentioned here consists of labeled sample videos. Because some of the labeled data types in these sample videos don't quite match the specific content, datasets containing such videos are called "dirty" datasets.

[0165] Step 2: Use the large model model_dirty to predict all sample videos in the dirty dataset dataset_dirty, and obtain the class confidence value of each sample video dataset_dirty_conf;

[0166] The category confidence score is a value between 0 and 1. A higher confidence score means that the sample video is more likely to be a positive sample video and less likely to be a negative sample video; conversely, a lower confidence score means that the sample video is less likely to be a positive sample video and more likely to be a negative sample video.

[0167] Step 3: Group the sample videos according to their labels. For positive sample videos, retain the top 80% of positive sample videos after sorting them by confidence value from highest to lowest. For negative sample videos, retain the top 80% of negative sample videos after sorting them by confidence value from lowest to highest.

[0168] Specifically, based on the labels of each sample video in dataset_dirty (assuming a label of 1 represents a positive sample video and a label of 0 represents a negative sample video), the sample videos in dataset_dirty can be divided into two groups: a group with a label of 1 (i.e., the positive sample video group) and a group with a label of 0 (i.e., the negative sample video group).

[0169] For the group with label 0 (i.e., the negative sample video group): Sort the sample videos in the negative sample video group in ascending order according to the value of dataset_dirty_conf, and keep the first 80% of the sample videos, denoted as dataset_neg.

[0170] For the group with label 1 (i.e., the positive sample video group): sort the sample videos in the positive sample video group in descending order according to the value of dataset_dirty_conf, and keep the first 80% of the sample videos, denoted as dataset_pos.

[0171] Merge dataset_pos and dataset_neg into a new training dataset dataset_clean; the discarded data is retained and denoted as dataset_unlabel.

[0172] Step 4: Retrain a new model using the dataset_clean. The structure can be the same as model_dirty, but the weights are different. Denote this as model_clean. Figure 3 The "clean large model" described above completes active learning at this step.

[0173] After completing active learning, a high-precision large model, model_clean, is obtained. However, it is computationally inefficient and consumes a lot of computing resources when deployed. Therefore, model distillation / knowledge distillation methods can be used to obtain a computationally efficient small model, while semi-supervised methods can be used to make full use of dataset_unlabeled data.

[0174] II. Model Distillation and Semi-Supervised Learning

[0175] Following step four above, the following section, starting from step five, describes the specific implementation process of the second stage:

[0176] Step 5: Add a -100 label to the dataset_unlabel data and mix it with the dataset_clean data, denoted as dataset_mix;

[0177] The -100 label can be used to distinguish whether the data comes from dataset_unlabel or dataset_clean.

[0178] Select a model with fewer parameters, higher computational efficiency, and relatively weaker learning ability (such as MobileNetV2) as the backbone model, embed it into the TSM model framework as the target small model, denoted as model_student, and at the same time, use model_clean obtained through active learning as the teacher model model_clean to form a student / teacher model model_clean pair, and then execute step six.

[0179] Step Six: Randomly select a batch of sample videos from dataset_mix;

[0180] Step 7: Use model_clean (large model) to predict the class of this batch of randomly selected sample videos and obtain the class distribution prob_teacher for each sample video;

[0181] Here, there is no need to distinguish whether the label of the sample video is -100 or not; model_clean can be used directly to predict the sample class distribution prob_teacher.

[0182] Step 8: For the randomly selected batch of sample videos mentioned above, obtain the class distribution prob_student of the batch of sample videos through model_student (target small model);

[0183] Here, there is no need to distinguish whether the label of the sample video is -100 or not; model_student can be used directly to predict the sample class distribution prob_student.

[0184] Step 9: Calculate the KL divergence loss function for prob_student and prob_teacher, denoted as loss_distill;

[0185] The purpose of this loss function is to make the predictions of the student model as close as possible to the predictions of the high-precision teacher model.

[0186] Step 10: Based on the category distribution of sample videos whose labels are not -100 in prob_student, and the labels of these sample videos whose labels are not -100, calculate the cross-entropy function, denoted as loss_hard;

[0187] The purpose of this loss function is to ensure that the category of sample videos whose label is not -100 is predicted correctly by model_student as much as possible.

[0188] Step 11: Calculate the sum of loss_distill and loss_hard as the final loss function, and then update the weights of model_student through gradient calculation, while keeping the weights of model_clean unchanged;

[0189] Step 12: Determine if model_student has converged. If it has converged, end the process and obtain a high-precision, high-efficiency small model model_final. If it has not converged, proceed to step 6.

[0190] The method provided in Embodiment 3 of this application can solve the noise problem caused by dirty data, make full use of all data, and obtain a high-precision and high-efficiency small video understanding model. Using this small video understanding model to perform video understanding can replace manual video understanding, which can free up manpower, improve operational efficiency, and reduce operating costs.

[0191] Example 4

[0192] To address the issue of insufficient class prediction accuracy in models trained using sample objects with inaccurate labels, Embodiment 4 of this application provides a model training device, the specific structure of which is shown in the schematic diagram below. Figure 4a As shown, it includes the following functional units:

[0193] The first training unit 41 is used to train the first category prediction model using sample objects in the original sample object set to obtain the trained first model; the sample objects are labeled objects used as training samples.

[0194] Prediction unit 42 is used to predict the category of the sample object using the trained first model obtained by the first training unit 41, so as to determine the degree of matching between the label of the sample object and the predicted category of the sample object;

[0195] The selection unit 43 is used to select a sample object that meets the preset conditions from the original sample object set as the first sample object based on the matching degree determined by the prediction unit 42.

[0196] The sample objects that meet the preset conditions mentioned here refer to sample objects with relatively high labeling accuracy. For specific selection methods, please refer to the method embodiments described above; they will not be repeated here.

[0197] The second training unit 44 is used to train the second category prediction model using the first sample object selected by the selection unit 43.

[0198] Optionally, to obtain a more efficient and easily deployable category prediction model, the second training unit 44 may specifically include:

[0199] The second model training module is used to train the category prediction second model using the first sample object selected by the selection unit 43, so as to obtain the trained second model.

[0200] The knowledge distillation module is used to train the student model using the sample objects in the recombined sample object set, with the trained second model obtained from the second model training module as the teacher model, and to obtain the trained student model.

[0201] The recombined sample object set consists of the first sample object and a second sample object that does not meet the preset conditions; the second sample object is labeled with a preset tag.

[0202] The preset label is used to distinguish between the first sample object and the second sample object.

[0203] Optionally, the knowledge distillation module may specifically include the following sub-modules:

[0204] The category distribution determination submodule is used to use the trained second model as the teacher model to predict the category of the sample objects in the recombined sample object set, so as to obtain the category distribution predicted by the teacher model; use the student model to predict the category of the first sample object, so as to obtain the category distribution predicted by the student model; discard the category distribution of the second object in the category distribution predicted by the student model, so as to obtain the category distribution of the remaining first sample objects, which is used as the target category distribution predicted by the student model.

[0205] The loss function determination submodule is used to determine the loss function of the student model based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution.

[0206] The training submodule is used to update the student model until convergence based on the loss function determined by the submodule.

[0207] Optionally, the loss function determination submodule can be used specifically for:

[0208] Based on the target category distribution, a first loss function is obtained; based on the category distribution predicted by the teacher model and the category distribution predicted by the student model, a second loss function is obtained; based on the first loss function and the second loss function, the loss function of the student model is obtained.

[0209] Optionally, the loss function determination submodule can be used specifically for:

[0210] Based on the target category distribution, a cross-entropy function between the target category distribution and the label of the corresponding first sample object is constructed as a first loss function; based on the category distribution predicted by the teacher model and the category distribution predicted by the student model, a KL divergence loss function is constructed as a second loss function for the category distribution predicted by the teacher model and the category distribution predicted by the student model; the sum of the first loss function and the second loss function is determined as the loss function of the student model.

[0211] Optionally, prediction unit 42 may specifically include:

[0212] The confidence value determination module is used to predict the category of the sample object using the trained first model to obtain the category confidence value of the sample object; the category confidence value represents the matching degree and is positively correlated with the matching degree.

[0213] Correspondingly, selecting unit 43 may specifically include:

[0214] The positive sample object selection module is used to select positive sample objects in descending order of their category confidence values ​​from the original sample object set, with the number of objects meeting the first proportion.

[0215] The negative sample object selection module is used to select negative sample objects in the original sample object set in ascending order of the category confidence value of the negative sample objects, and the number of these negative sample objects meets the second proportion.

[0216] The device provided in this application provides a higher category prediction accuracy than the category prediction model trained directly on a dirty dataset, since the sample objects used to train the second category prediction model are sample objects that meet preset conditions.

[0217] Furthermore, Embodiment 4 of this application also provides a model training device for obtaining a category prediction model with both high accuracy and efficiency. Specifically, a schematic diagram of the structure of the model training device is shown below. Figure 4b As shown, it includes the following functional units:

[0218] Sample acquisition unit 45 is used to acquire a set of recombinant sample objects;

[0219] The recombined sample object set includes: a first sample object selected from the original sample object set that meets the preset conditions, and a second sample object that does not meet the preset conditions, wherein the second sample object is labeled with a preset tag;

[0220] The category distribution prediction unit 46 is used to use a category prediction model trained based on the selected sample objects as a teacher model to predict the category of the sample objects in the recombined sample object set, so as to obtain the category distribution predicted by the teacher model; to use a student model to predict the category of the first sample object, so as to obtain the category distribution predicted by the student model; to discard the category distribution of the second object in the category distribution predicted by the student model, so as to obtain the category distribution of the remaining first sample object, which is used as the target category distribution predicted by the student model.

[0221] The specific training methods for the teacher model can be as follows: Figure 1a As shown, however, other training methods can also be used to train the teacher model in this embodiment.

[0222] The student model mentioned here is generally a model with higher computational efficiency compared to the teacher model.

[0223] The loss function determination unit 47 is used to determine the loss function of the student model based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution determined by the class distribution prediction unit 46.

[0224] The model training unit 48 is used to update the student model until convergence based on the loss function determined by the loss function determination unit 47, so as to obtain a trained student model.

[0225] Optionally, the loss function determination unit 47 can be used for:

[0226] Based on the target category distribution, a first loss function is obtained; based on the category distribution predicted by the teacher model and the category distribution predicted by the student model, a second loss function is obtained; based on the first loss function and the second loss function, the loss function of the student model is obtained.

[0227] Optionally, the loss function determination unit 47 can be used for:

[0228] Based on the target category distribution, a cross-entropy function is constructed between the target category distribution and the corresponding label of the first sample object, serving as the first loss function; based on the category distribution predicted by the teacher model and the category distribution predicted by the student model, a KL divergence loss function is constructed between the category distribution predicted by the teacher model and the category distribution predicted by the student model, serving as the second loss function; the sum of the first loss function and the second loss function is determined as the loss function of the student model.

[0229] Adopting such Figure 4b The device shown uses samples with relatively high labeling accuracy to train the teacher model. Therefore, compared to a category prediction model trained directly on a dirty dataset, the teacher model trained using this device has higher category prediction accuracy. Furthermore, by employing knowledge distillation to train the student model using the teacher model, the trained student model can inherit the high accuracy of the teacher model. Simultaneously, because the student model has higher computational efficiency than the teacher model, it is more suitable for deployment on computing devices with limited computing power, such as user terminals.

[0230] Specifically, by discarding the class distribution of second objects that do not meet the preset conditions from the class distribution predicted by the student model when determining the loss function, and obtaining the class distribution of the remaining first sample objects as the target class distribution predicted by the student model, and then constructing the cross-entropy function between the target class distribution and the corresponding label of the first sample object as the first loss function of knowledge distillation, the calculation basis of the first loss function of knowledge distillation is the class distribution result obtained by "the student model completing the correct classification". This approach allows the first loss function to better achieve the "hard objective" of knowledge distillation, ensuring that the knowledge distillation process produces a better training effect on the student model, resulting in a better classification performance for the trained student model.

[0231] Example 5

[0232] Embodiment 5 of this application provides a category prediction device for achieving accurate understanding of objects. A schematic diagram of the category prediction device is shown below. Figure 5 As shown, it includes the following functional units:

[0233] The object acquisition unit 51 is used to acquire the object to be predicted in terms of category.

[0234] The category prediction unit 52 is used to input the object obtained by the object acquisition unit 51 into the category prediction model to obtain the category prediction result of the object output by the category prediction model.

[0235] The category prediction model is trained using the model training method described in Embodiment 1 of this application.

[0236] Since the sample objects used to train the second category prediction model using the model training method described in Example 1 are sample objects that meet the preset conditions (the labeling accuracy is relatively high), the category prediction model trained using the method provided in Example 1 of this application will have higher category prediction accuracy than the category prediction model trained directly on the dirty dataset. Therefore, using such a category prediction model for category prediction can correspondingly obtain category prediction results with higher accuracy.

[0237] Specifically, by discarding the class distribution of second objects that do not meet the preset conditions from the class distribution predicted by the student model when determining the loss function, and obtaining the class distribution of the remaining first sample objects as the target class distribution predicted by the student model, and then constructing the cross-entropy function between the target class distribution and the corresponding label of the first sample object as the first loss function of knowledge distillation, the calculation basis of the first loss function of knowledge distillation is the class distribution result obtained by "the student model completing the correct classification". This approach allows the first loss function to better achieve the "hard objective" of knowledge distillation, ensuring that the knowledge distillation process produces a better training effect on the student model, resulting in a better classification performance for the trained student model.

[0238] Example 6

[0239] Based on the same inventive concept as Embodiment 1 of this application, Embodiment 6 of this application provides a computing device to solve the problem that the accuracy of category prediction of the trained model is not high due to the use of sample objects with inaccurate labeling.

[0240] like Figure 6As shown, the computing device includes a memory 61 and a processor 62. The memory 61 can be configured to store various other data to support operation on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device. The memory 61 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0241] The processor 62, coupled to the memory 61, is used to execute the program stored in the memory 61 to perform a model training method as described in Embodiment 1 of this application, or to perform a category prediction method as described in Embodiment 1 of this application.

[0242] When the processor 62 executes the program in the memory 61, in addition to the functions described above, it can also perform other functions, as detailed in the descriptions of the preceding embodiments.

[0243] Furthermore, such as Figure 6 As shown, the computing device also includes other components such as a display 64, a communication component 63, a power supply component 65, and an audio component 66. Figure 6 The diagram only shows some components and does not mean that the computing device includes only these components. Figure 6 The components shown.

[0244] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the steps or functions of the methods provided in the above embodiments.

[0245] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0246] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0247] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A model training method, characterized in that, include: Using the sample objects in the original sample object set, the first model for category prediction is trained to obtain the trained first model; The sample objects include multimedia files that are labeled and used as training samples; the labels represent the categories of the sample objects. Using the trained first model, the class of the sample object is predicted to determine the degree of matching between the label of the sample object and the predicted class of the sample object. Based on the matching degree, sample objects that meet the preset conditions are selected from the original sample object set as the first sample object; Using the first sample object, the second model for category prediction is trained to obtain the trained second model; Using the sample objects in the recombined sample object set, and with the trained second model as the teacher model, the student model is trained using knowledge distillation techniques to obtain a trained student model; during the training process of the student model, the teacher model and the student model perform category prediction on the same batch of sample objects selected from the recombined sample object set; The recombined sample object set consists of the first sample object and a second sample object that does not meet the preset conditions; the second sample object is labeled with a preset tag.

2. The method as described in claim 1, characterized in that, Using sample objects from the recombined sample object set, and with the trained second model as the teacher model, the student model is trained using knowledge distillation techniques, including: Using the trained second model as the teacher model, class prediction is performed on the sample objects selected from the recombined sample object set to obtain the class distribution predicted by the teacher model; Using a student model, class prediction is performed on the selected sample objects to obtain the class distribution predicted by the student model; the class distribution of the second sample objects in the class distribution predicted by the student model is discarded to obtain the class distribution of the remaining first sample objects, which is used as the target class distribution predicted by the student model. Based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution, the loss function of the student model is determined. The student model is updated until convergence based on the loss function.

3. The method as described in claim 2, characterized in that, Based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution, the loss function of the student model is determined, including: Based on the target category distribution, obtain the first loss function; Based on the category distribution predicted by the teacher model and the category distribution predicted by the student model, a second loss function is obtained; Based on the first loss function and the second loss function, the loss function of the student model is obtained.

4. The method as described in claim 3, characterized in that, The first loss function is obtained based on the target category distribution; the second loss function is obtained based on the category distribution predicted by the teacher model and the category distribution predicted by the student model. Based on the first loss function and the second loss function, the loss function of the student model is obtained, including: Based on the target category distribution, a cross-entropy function is constructed between the target category distribution and the corresponding label of the first sample object, which serves as the first loss function; Based on the class distributions predicted by the teacher model and the class distributions predicted by the student model, a KL divergence loss function is constructed for the class distributions predicted by the teacher model and the student model, which serves as the second loss function. The sum of the first loss function and the second loss function is determined as the loss function of the student model.

5. The method as described in claim 1, characterized in that, Using the trained first model, class prediction is performed on the sample object to determine the degree of matching between the label of the sample object and the predicted class of the sample object, including: Using the trained first model, the category of the sample object is predicted to obtain the category confidence value of the sample object; the category confidence value represents the matching degree and is positively correlated with the matching degree.

6. The method as described in claim 5, characterized in that, Based on the matching degree, sample objects whose matching degree meets the preset matching degree requirements are selected from the original sample object set, including: Based on the category confidence values ​​of the positive sample objects in the original sample object set, positive sample objects with quantities matching the first proportion are selected in descending order of their values. Based on the category confidence values ​​of the negative sample objects in the original sample object set, negative sample objects with quantities matching the second proportion are selected sequentially in ascending order of value.

7. A model training method, characterized in that, include: Obtain a recombined set of sample objects; wherein the sample objects in the recombined set of sample objects include: a first sample object that meets preset conditions, and a second sample object that does not meet the preset conditions, the second sample object being labeled with a preset tag; the sample objects in the recombined set of sample objects include multimedia files that are used as training samples after being labeled with tags, the tags of the sample objects indicating the category of the sample objects; Using the classification model trained on the first sample object as a teacher model, the class of the sample objects in the recombined sample object set is predicted to obtain the class distribution predicted by the teacher model. The student model is used to predict the category of the sample objects in the recombined sample object set to obtain the category distribution predicted by the student model; the category distribution of the second sample objects in the category distribution predicted by the student model is discarded to obtain the category distribution of the remaining first sample objects, which is used as the target category distribution predicted by the student model. The loss function of the student model is determined based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution. The student model is updated until convergence based on the loss function to obtain a trained student model.

8. The method as described in claim 7, characterized in that, Based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution, the loss function of the student model is determined, including: Based on the target category distribution, obtain the first loss function; Based on the category distribution predicted by the teacher model and the category distribution predicted by the student model, a second loss function is obtained; Based on the first loss function and the second loss function, the loss function of the student model is obtained.

9. The method as described in claim 8, characterized in that, The first loss function is obtained based on the target category distribution; the second loss function is obtained based on the category distribution predicted by the teacher model and the category distribution predicted by the student model. Based on the first loss function and the second loss function, the loss function of the student model is obtained, including: Based on the target category distribution, a cross-entropy function is constructed between the target category distribution and the corresponding label of the first sample object, which serves as the first loss function; Based on the class distributions predicted by the teacher model and the class distributions predicted by the student model, a KL divergence loss function is constructed for the class distributions predicted by the teacher model and the student model, which serves as the second loss function. The sum of the first loss function and the second loss function is determined as the loss function of the student model.

10. A category prediction method, characterized in that, include: Obtain the object to be classified for prediction; The object is input into the category prediction model to obtain the category prediction result of the object output by the category prediction model; The category prediction model is trained using the model training method described in any one of claims 1 to 9.

11. A model training device, characterized in that, include: The first training unit is used to train the first class prediction model using sample objects from the original sample object set, so as to obtain the trained first model. The sample objects include multimedia files that are labeled and used as training samples; the labels of the sample objects indicate the categories of the sample objects. A prediction unit is used to predict the category of the sample object using the trained first model obtained by the first training unit, so as to determine the degree of matching between the label of the sample object and the predicted category of the sample object. The selection unit is used to select sample objects that meet preset conditions from the original sample object set as the first sample object based on the matching degree determined by the prediction unit. The second training unit is used to train the category prediction second model using the first sample object to obtain a trained second model; using the sample objects in the recombined sample object set, and with the trained second model as the teacher model, the student model is trained using knowledge distillation technology to obtain a trained student model; during the training process of the student model, the teacher model and the student model perform category prediction on the same batch of sample objects selected from the recombined sample object set; The recombined sample object set consists of the first sample object and a second sample object that does not meet the preset conditions; the second sample object is labeled with a preset tag.

12. A model training device, characterized in that, include: A sample acquisition unit is used to acquire a recombined set of sample objects; wherein the sample objects in the recombined set of sample objects include: a first sample object that meets preset conditions, and a second sample object that does not meet the preset conditions, the second sample object being labeled with a preset tag; the sample objects in the recombined set of sample objects include multimedia files that are labeled and used as training samples, the tags of the sample objects indicating the categories of the sample objects; The category distribution prediction unit is used to use a category prediction model trained based on the first sample object as a teacher model to predict the category of the sample objects in the recombined sample object set, so as to obtain the category distribution predicted by the teacher model; to use a student model to predict the category of the sample objects in the recombined sample object set, so as to obtain the category distribution predicted by the student model; and to discard the category distribution of the second sample object in the category distribution predicted by the student model, so as to obtain the category distribution of the remaining first sample objects, which is used as the target category distribution predicted by the student model. The loss function determination unit is used to determine the loss function of the student model based on the class distribution predicted by the teacher model, the class distribution predicted by the student model, and the target class distribution. The model training unit is used to update the student model until convergence based on the loss function determined by the loss function determination unit, so as to obtain a trained student model.

13. A category prediction device, characterized in that, include: The object acquisition unit is used to acquire the object to be predicted in terms of category. The category prediction unit is used to input the object acquired by the object acquisition unit into the category prediction model to obtain the category prediction result of the object output by the category prediction model; The category prediction model is trained using the model training method described in any one of claims 1 to 9.

14. A computing device, characterized in that, include: Memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program stored in the memory for performing the method of any one of claims 1 to 9, or for performing the method of claim 10.

15. A computer-readable storage medium storing a computer program, which, when executed by a computer, enables the implementation of the method according to any one of claims 1 to 9, or enables the implementation of the method according to claim 10.

Images