A picture classification method, a picture classification model training method and related devices

Abstract

Embodiments of the present disclosure provide a picture classification method, comprising: obtaining a picture to be identified; inputting the picture into a pre-trained picture classification model; and outputting a result of picture classification based on the trained picture classification model. In the above process, on the one hand, the lack of labeled samples can be made up, the labeling cost can be reduced, and the accuracy and generalization ability of the model can be improved. On the other hand, the accuracy of the picture classification model can be improved, and the problem of poor generalization ability of the picture classification model can be solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of artificial intelligence, and more particularly, embodiments of the present disclosure relate to a picture classification method, a picture classification model training method and related devices. BACKGROUND

[0002] This section is intended to provide background information to facilitate an understanding of embodiments of the disclosure set forth in the claims. The description herein does not constitute admission of prior art.

[0003] With the continuous development of artificial intelligence technology, it has become a research hotspot to classify and identify pictures based on computer vision technology. At present, different identification models can be designed for different picture classification tasks, and then the picture classification model is supervised trained based on the labeled picture training set.

[0004] However, since the quantity and quality of data directly determine the quality of the prediction model, in order to obtain a high-quality picture classification model, not only a large amount of sample data is needed, but also the accuracy of labeling is required, which leads to the problem of high labeling cost.

[0005] In addition, since the picture data distribution in different picture training sets is different, and there are a large amount of picture data on the Internet, when actually classifying, there may be a problem of insufficient model accuracy due to different picture data distribution in the test set. SUMMARY

[0006] In this context, embodiments of the present disclosure aim to provide a picture classification method, a picture classification model training method and related devices.

[0007] In a first aspect of the embodiments of the present disclosure, a picture classification method is provided, comprising:

[0008] obtaining a picture to be identified;

[0009] inputting the picture into a pre-trained picture classification model;

[0010] outputting a picture classification result based on the trained picture classification model.

[0011] In a second aspect of the embodiments of the present disclosure, a picture classification model training method is provided, comprising:

[0012] obtaining a training sample set, the training sample set comprising unlabeled samples and labeled samples;

[0013] inputting the unlabeled samples into N picture classification models respectively to determine corresponding first output results;

[0014] input the N first output results to a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a corresponding second output result for each picture classification model;

[0015] determine, based on a preset algorithm, an output result in the N second output results that meets a preset condition, and label a pseudo label for the unlabeled sample corresponding to the output result that meets the preset condition, to determine a pseudo labeled sample; wherein the pseudo label is determined based on the second output result;

[0016] train the N picture classification models based on a semi-supervised learning method respectively according to the labeled sample and the pseudo labeled sample, to obtain N trained picture classification models.

[0017] In a third aspect of the embodiments of the present disclosure, a picture classification device is provided, comprising:

[0018] a picture acquisition module, configured to acquire a picture to be recognized;

[0019] an input module, configured to input the picture to a pre-trained picture classification model;

[0020] an output module, configured to output a picture classification result based on the trained picture classification model.

[0021] In a fourth aspect of the embodiments of the present disclosure, a picture classification model training device is provided, comprising:

[0022] a second acquisition module, configured to acquire a training sample set, the training sample set comprising unlabeled samples and labeled samples;

[0023] a second determination module, configured to input the unlabeled samples to N picture classification models respectively, to determine corresponding first output results;

[0024] a second fusion module, configured to input the N first output results to a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a corresponding second output result for each picture classification model;

[0025] a second screening module, configured to determine, based on a preset algorithm, an output result in the N second output results that meets a preset condition;

[0026] a second labeling module, configured to label a pseudo label for the unlabeled sample corresponding to the output result that meets the preset condition, to determine a pseudo labeled sample; wherein the pseudo label is determined based on the second output result;

[0027] A second training module is configured to train the N picture classification models based on a semi-supervised learning method according to the labeled samples and the pseudo-labeled samples, and obtain the N trained picture classification models.

[0028] In a fifth aspect of the embodiments of the present disclosure, a storage medium is provided, and the storage medium has stored thereon computer instructions which, when executed by a processor, implement the steps of the method described as follows:

[0029] Obtaining a picture to be recognized;

[0030] Inputting the picture into a pre-trained picture classification model;

[0031] Outputting a result of picture classification based on the trained picture classification model.

[0032] Or, the storage medium has stored thereon computer instructions which, when executed by a processor, implement the steps of the method described as follows:

[0033] Obtaining a training sample set, the training sample set including unlabeled samples and labeled samples;

[0034] Inputting the unlabeled samples into N picture classification models respectively, and determining corresponding first output results;

[0035] Inputting the N first output results into a preset knowledge transfer model for fusion, so as to determine, for each picture classification model, a corresponding second output result by the knowledge transfer model;

[0036] Determining, based on a preset algorithm, an output result in the N second output results that satisfies a preset condition, and labeling a pseudo-label for the unlabeled sample corresponding to the output result that satisfies the preset condition, to determine a pseudo-labeled sample; wherein the pseudo-label is determined based on the second output result;

[0037] Training the N picture classification models based on a semi-supervised learning method according to the labeled samples and the pseudo-labeled samples, and obtaining the N trained picture classification models.

[0038] In a sixth aspect of the embodiments of the present disclosure, a computing device is provided, and the computing device includes:

[0039] A processor; and a memory for storing processor-executable instructions;

[0040] The processor implements the steps of the method described as follows by running the executable instructions.

[0041] Obtaining a picture to be recognized;

[0042] inputting the picture into a pre-trained picture classification model;

[0043] outputting a result of picture classification based on the pre-trained picture classification model.

[0044] Alternatively, the processor implements the steps of the following method by running the executable instructions:

[0045] obtaining a training sample set, the training sample set including unlabeled samples and labeled samples;

[0046] inputting the unlabeled samples into N picture classification models respectively to determine corresponding first output results;

[0047] inputting the N first output results into a preset knowledge transfer model for fusion, so as to determine, for each picture classification model, a corresponding second output result by the knowledge transfer model;

[0048] determining, based on a preset algorithm, an output result in the N second output results that satisfies a preset condition, and labeling a pseudo label for the unlabeled sample corresponding to the output result that satisfies the preset condition, to determine a pseudo-labeled sample; wherein the pseudo label is determined based on the second output result;

[0049] training the N picture classification models based on a semi-supervised learning method according to the labeled samples and the pseudo-labeled samples, to obtain N trained picture classification models.

[0050] The above embodiments of the present disclosure have at least the following beneficial effects:

[0051] Through the above technical solutions, on the one hand, since the unlabeled samples are not only easy to obtain but also have more data and can obtain more useful information compared to the labeled samples, based on the semi-supervised learning method, the defects of insufficient labeled samples can be made up, the labeling cost can be reduced, and the accuracy and generalization ability of the model can be improved. On the other hand, by transferring the results of the unlabeled samples output by each picture classification model, noise data in the training can be filtered, and pseudo-labeled samples with higher quality can be selected, thereby improving the precision of the picture classification model and solving the problem of poor generalization ability of the picture classification model. In addition, the accuracy of the model can be improved through end-to-end training, and the finally trained picture classification model can be used to complete the picture classification task. BRIEF DESCRIPTION OF DRAWINGS

[0052] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: A few exemplary embodiments of the present disclosure are illustrated in the accompanying drawings, which are shown by way of illustration and not by way of limitation.

[0053] FIG. 1 A flowchart schematically illustrating a picture classification method according to an embodiment of the present disclosure is shown;

[0054] FIG. 2 A schematic diagram schematically illustrating a picture classification model training method according to an embodiment of the present disclosure is shown;

[0055] FIG. 3 A flowchart schematically illustrating a picture classification model training method according to an embodiment of the present disclosure is shown;

[0056] FIG. 4 A schematic diagram schematically illustrating another picture classification model training method according to an embodiment of the present disclosure is shown;

[0057] FIG. 5 A block diagram schematically illustrating a picture classification apparatus according to an embodiment of the present disclosure is shown;

[0058] FIG. 6 A block diagram schematically illustrating a picture classification model training apparatus according to an embodiment of the present disclosure is shown;

[0059] FIG. 7 A schematic diagram schematically illustrating a computer readable storage medium according to an embodiment of the present disclosure is shown;

[0060] FIG. 8 A schematic diagram schematically illustrating an electronic device capable of implementing the above method according to an embodiment of the present disclosure is shown.

[0061] In the drawings, identical or corresponding labels represent identical or corresponding parts. DETAILED DESCRIPTION

[0062] The principles and spirits of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, and in no way limit the scope of the present disclosure. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0063] Those skilled in the art know that the embodiments of the present disclosure can be implemented as a system, apparatus, device, method or computer readable storage medium. Therefore, the present disclosure can be embodied in the form of a complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0064] According to an embodiment of the present disclosure, a picture classification method, a picture classification model training method and related devices are provided.

[0065] In this document, it should be understood that the number of any elements in the figures is intended to be illustrative only and not limiting; and any references to "one embodiment", "an embodiment", "example embodiment", etc. should be understood only to mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the present disclosure.

[0066] The principles and spirits of the present disclosure will be explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY

[0068] The present inventors have found that, in solving a picture classification task, although a picture classification model can be trained based on supervised learning, the quality of the model trained by supervised learning depends on the quantity and quality of picture samples. Therefore, in order to improve the accuracy of the picture classification model and enhance the accuracy of the picture classification result, a large number of picture samples need to be accurately labeled, which results in a problem of greatly increased labeling cost.

[0069] Therefore, the present specification provides a technical solution for reducing labeling cost and improving model accuracy and generalization ability by constructing N different picture classification models, training each picture classification model based on semi-supervised learning, and using the pseudo label corresponding to the unlabeled data obtained by fusing the knowledge transfer model during training.

[0070] The core technical concept of the present specification is that:

[0071] Between the input end of the picture and the output end of the classification result, N different picture classification models and a knowledge transfer model are constructed, the unlabeled samples are input into each picture classification model, the prediction results of the picture classification model on the unlabeled samples are input into the knowledge transfer model for fusion to obtain the pseudo label corresponding to the unlabeled samples, and then each picture classification model is trained based on semi-supervised learning according to the labeled samples and the pseudo labeled pseudo labeled samples, and the iteration is continuously performed until the picture classification model that can be used to complete the picture classification task is finally obtained.

[0072] By the above technical solution, on the one hand, since the unlabeled samples are not only easy to obtain but also have more data and can obtain more useful information than the labeled samples, based on the semi-supervised learning method, the defects of insufficient labeled samples can be made up, the labeling cost can be reduced, and the accuracy and generalization ability of the model can be improved. On the other hand, by performing knowledge transfer on the results of the unlabeled samples output by each picture classification model, noise data in the training can be filtered, and higher-quality pseudo-labeled samples can be screened out, thereby improving the precision of the picture classification model and solving the problem of poor generalization ability of the picture classification model. In addition, the accuracy of the model can be improved through end-to-end training, and the finally trained picture classification model can be used to complete the picture classification task.

[0073] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure will be specifically introduced below.

[0074] Overview of Application Scenarios

[0075] When performing a picture classification task, a picture classification model can usually be supervised trained based on a labeled picture training set.

[0076] For example, when the picture classification task is to classify and identify cats in pictures, training samples need to be prepared, a large number of pictures containing cats can be obtained from the Internet, and artificial labeling is performed. The pictures containing cats are labeled with "cat" tags, and the labeled pictures are used as training samples to train the picture classification model.

[0077] When the picture classification model trained based on the above is used to classify the input pictures, the predicted score is compared with the threshold value. If the predicted score is greater than the threshold value, it is judged that the picture contains a cat, and if the predicted score is less than the threshold value, the picture does not contain a cat.

[0078] Then, in the above supervised training process, if a good model classification effect is desired, a large amount of training sample data is required, which in turn leads to high labeling cost.

[0079] Moreover, the precision of the model also depends on the quality of the labeled samples. If the noise data is too large, it will increase the difficulty of distinguishing relevant features from irrelevant features for the model, resulting in a decrease in the precision of the model. High-quality labeled samples also increase the labeling cost.

[0080] It is worth noting that in some picture classification tasks, there may also be a problem of insufficient sample data. For example, in order to purify the online environment, pictures containing uncivilized information, violent and terrorist information, and other garbage information can be identified from the Internet.

[0081] But compared with the massive picture data in the Internet, the number of illegal pictures is extremely low, and it is difficult to obtain a large number of pictures suitable for illegal picture identification tasks. Therefore, when the training sample is insufficient, the trained model not only has low accuracy, but also often has poor generalization ability.

[0082] Among them, the generalization ability in a broad sense refers to the prediction ability of the learned model to unknown data, that is, the trained network can also give appropriate output to the data outside the learning set with the same rule, and this ability is called generalization ability.

[0083] It should be noted that the above application scenarios are only shown for the purpose of facilitating the understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0084] Exemplary Method

[0085] The technical concept of the present specification will be described in detail below through specific embodiments.

[0086] The present disclosure aims to provide a technical solution for reducing labeling cost and improving model accuracy and generalization ability by constructing N different picture classification models, training each picture classification model based on semi-supervised learning, and using the pseudo label corresponding to the unlabeled data obtained by fusing the knowledge transfer model during training, so as to use the trained picture classification model for picture classification task.

[0087] In implementation, a picture to be identified can be obtained.

[0088] Then, the picture to be identified is input into the pre-trained picture classification model.

[0089] Among them, the training of the picture classification model can be completed by utilizing knowledge transfer between multiple models from the perspective of semi-supervised learning.

[0090] Next, based on the trained picture classification model, the result of picture classification is output.

[0091] By the above technical scheme, on the one hand, since the unlabeled samples are not only easy to obtain but also have more data and can obtain more useful information compared with the labeled samples, the semi-supervised learning method can not only make up for the defects of insufficient labeled samples, but also reduce the labeling cost, and improve the accuracy and generalization ability of the model. On the other hand, by performing knowledge transfer on the results of the unlabeled samples output by each picture classification model, the noise data in the training is filtered, and the pseudo-labeled samples with higher quality are screened out, so as to improve the precision of the picture classification model and solve the problem of poor generalization ability of the picture classification model. In addition, the accuracy of the model can be improved through end-to-end training, and the finally trained picture classification model can be used to complete the picture classification task.

[0092] Please refer to FIG. 1 , FIG. 1 is a flowchart of a picture classification method provided by an example embodiment, and the method comprises the following steps:

[0093] Step 101, obtaining a picture to be identified.

[0094] Among them, the picture mentioned above can be a picture to be identified obtained according to a preset method when performing a picture classification task.

[0095] For example, when performing a violation picture identification task, pictures posted by users can be obtained from each post of a forum for violation picture identification.

[0096] Step 102, inputting the picture into a pre-trained picture classification model.

[0097] Among them, the pre-trained picture classification model mentioned above can be a single picture classification model, or a combination of multiple picture classification models.

[0098] For example, the picture can be input into each picture classification model respectively, and the final classification result of the picture can be determined according to the corresponding results of these picture classification models.

[0099] Step 103, outputting the result of picture classification based on the trained picture classification model.

[0100] For example, for the violation picture identification task, after inputting the picture into the pre-trained picture classification model, the prediction score of the picture can be obtained, and by judging the prediction score, when the prediction score is greater than the set threshold, the picture can be considered as a prohibited picture, otherwise as a non-prohibited picture, and the final classification result is output.

[0101] Among them, before inputting the picture into the model, data preprocessing can be performed.

[0102] For example, in order to ensure the balance between speed and accuracy, a picture scaling operation can be performed to scale the resolution size of the picture. The specific scaling size can be determined by a person skilled in the art according to actual task requirements, and the present disclosure does not limit this.

[0103] For example, in order to ensure that the data input by the picture is consistent with the data distribution in the model training process, a normalization operation can also be performed on the picture.

[0104] The above trained picture classification model, since during training, N different picture classification models and a knowledge transfer model can be constructed between the input end of the picture and the output end of the classification result, by inputting the unlabeled sample into each picture classification model, and then inputting the prediction result of the picture classification model on the unlabeled sample into the knowledge transfer model for fusion, the pseudo label corresponding to the unlabeled sample is obtained, and then based on semi-supervised learning, each picture classification model is trained based on the labeled sample and the pseudo labeled pseudo labeled sample, and through continuous iteration, the picture classification model that can be used to complete the picture classification task is finally obtained.

[0105] During the training of the above picture classification model, on the one hand, since the unlabeled sample is not only easy to obtain, but also has more data, it can obtain more useful information, so based on the semi-supervised learning method, it can not only make up for the defects of insufficient labeled samples, but also reduce the labeling cost, and also improve the accuracy and generalization ability of the model. On the other hand, by transferring the knowledge of the unlabeled sample results output by each picture classification model, the noise data in the training is filtered, and the pseudo labeled sample with higher quality is selected, thereby improving the precision of the picture classification model and solving the problem of poor generalization ability of the picture classification model.

[0106] Based on this, since the training method of the above picture classification model can improve the generalization ability and accuracy of the model, when the picture classification is performed based on the trained picture classification model, the picture can be classified more easily, and the accuracy of the picture classification can be improved.

[0107] Next, the training process of the model will be described in detail. Before describing, it should be noted that the above picture classification model for picture classification can be any model or combination of multiple models selected from the following N trained picture classification models.

[0108] Please refer to FIG. 2 , FIG. 2 is a schematic diagram of a picture classification model training method provided by an example embodiment. The method comprises the following steps:

[0109] Step 201, obtaining a training sample set, the training sample set comprising unlabeled samples and labeled samples.

[0110] In the embodiment, the training sample set described above can be composed of unlabeled samples and labeled samples.

[0111] Among them, the unlabeled samples are not only easy to obtain, but also do not need to be labeled, which can greatly reduce the labeling cost. Moreover, by adding unlabeled samples, the accuracy of the classification boundary can be improved, and the robustness of the model can be improved.

[0112] It should be noted that before inputting the above training sample into the model, the sample can also be preprocessed, for example, the sample can be scaled or normalized.

[0113] Among them, the preprocessing operation taken on the picture to be identified before prediction is consistent with the operation of preprocessing the sample.

[0114] For example, if the sample is scaled to a specified 224 pixel x 224 pixel when preprocessed, the picture also needs to be scaled to a specified 224 pixel x 224 pixel when preprocessed before using the trained model to predict the picture to be identified.

[0115] For example, the method used for normalizing the picture to be identified needs to be consistent with the method used for normalizing the sample.

[0116] As described above, since the number of samples constructed in some picture classification tasks is limited, the number of training samples can be increased by data augmentation operation.

[0117] Data augmentation refers to processing existing data to generate more value from limited data, thereby improving the effect of the model. Common data augmentation methods include cropping, flipping, rotating, scaling, shifting, and Gaussian noise.

[0118] For example, by cropping, a part of the original image can be randomly cropped, and the cropped image can be used as a new training sample; by flipping, the original image can be horizontally or vertically flipped, and the flipped image can be used as a new training sample; by shifting, the original image can be translated in a specified direction, and the translated image can be used as a new training sample. For specific data augmentation operations, those skilled in the art can choose as needed.

[0119] Among them, it should be noted that according to the different degrees of data change, the data augmented samples can be divided into weak data augmented samples and strong data augmented samples.

[0120] In an embodiment shown, the obtained unlabeled samples can be subjected to data augmentation to obtain first data augmented samples and second data augmented samples.

[0121] In an example, the unlabeled samples in the training sample set can be subjected to data augmentation operation, and weak data augmented samples can be obtained by performing weak data augmentation operation on the unlabeled samples according to different degrees of data variation, and strong data augmented samples can be obtained by performing strong data augmentation operation on the strong data augmented samples.

[0122] In another example, since the number of prohibited pictures is extremely small relative to the massive picture data on the Internet, the number of samples that can be obtained is limited, and the obtained prohibited pictures can be subjected to data processing by using multiple data augmentation methods to obtain weak data augmented samples and strong data augmented samples according to different degrees of data variation.

[0123] In the above process, by subjecting the samples to data augmentation, the problem of insufficient training samples can be solved, thereby improving the generalization ability of the model.

[0124] Step 202, inputting the unlabeled samples into N picture classification models respectively to determine corresponding first output results.

[0125] In constructing the above N picture classification models, a person skilled in the art can select any known picture classification model, such as ResNet series, ResNeXt series, Transformer series, etc., and the present disclosure does not limit this.

[0126] In an example, the unlabeled samples can be inputted into the constructed multiple picture classification models respectively, and each picture classification model outputs a predicted score corresponding to the unlabeled sample.

[0127] Similarly, the unlabeled samples subjected to data augmentation can also be inputted into the above N picture classification models.

[0128] In an embodiment shown, the first data augmented samples can be inputted into N picture classification models respectively to determine corresponding third output results, and the second data augmented samples can also be inputted into N picture classification models respectively to determine corresponding fourth output results.

[0129] In an example, the weak data augmented samples can be inputted into N picture classification models respectively to determine predicted results of the weak data augmented samples, and the strong data augmented samples can also be inputted into N picture classification models respectively to determine predicted results of the strong data augmented samples.

[0130] It should be noted that the threshold used to determine the classification result according to the prediction score may be different for different picture classification models.

[0131] For example, assuming that the classification threshold corresponding to the A model is 0.85, and the classification threshold given by the B model is 0.75, for the same picture, even if the prediction scores given are both 0.8, the classification results are not the same, which means that the prediction accuracy of different models is different.

[0132] However, even so, through the technical solutions provided by the present disclosure, the plurality of picture classification models can be fully trained, and finally the requirements for accuracy and generalization ability can be met.

[0133] As described above, before determining the pseudo label corresponding to the unlabeled sample according to the knowledge transfer model, the output results of each picture classification model need to be input into the knowledge transfer model.

[0134] Step 203: input the N first output results into a preset knowledge transfer model for fusion, so as to determine a corresponding second output result for each picture classification model by the knowledge transfer model.

[0135] In this embodiment, after obtaining the first output result of each model, the N first output results can be input into a preset knowledge transfer model for fusion to obtain the second output result determined for each picture classification model.

[0136] Knowledge transfer refers to transferring knowledge between multiple fields or tasks, which simply means transferring the knowledge of field B to field A to improve the classification effect of field A.

[0137] For example, assuming that N is 4, when the knowledge transfer model is fused, the first output results of models B, C and D can be used to improve or optimize the learning effect of model A, and the fused second output result is output. The second output result of model B can be obtained by knowledge transfer of models A, C and D.

[0138] By using knowledge transfer, existing knowledge data can be reused, and there is no need to spend cost to re-collect and label new data sets, which can enhance the generalization ability of the model.

[0139] In addition, since the data variation degree of the strong data augmentation sample is large, the classification result accuracy given by the model is low in the initial stage of model training, so the weak data augmentation sample can be classified first.

[0140] Therefore, for the sample processed by data augmentation, the output result input into the knowledge transfer model can be the prediction result corresponding to the weak data augmentation sample.

[0141] In an embodiment shown, the N third output results can be input into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, the fifth output result corresponding to the first data enhancement sample respectively for each picture classification model.

[0142] For example, the prediction result corresponding to the weak data enhancement sample output by each picture classification model can be input into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, the prediction result after knowledge transfer corresponding to the weak data enhancement sample respectively for each picture classification model.

[0143] In step 204, based on a preset algorithm, the output result meeting the preset condition in the N second output results is determined, and the pseudo label is marked for the unlabeled sample corresponding to the output result meeting the preset condition, so as to determine the pseudo labeled sample; wherein the pseudo label is determined based on the second output result.

[0144] In this embodiment, based on a preset algorithm, the output result meeting the preset condition can be screened from the above-mentioned N second output results, and the pseudo label determined based on the output result meeting the preset condition is marked for the unlabeled sample corresponding to the output result meeting the preset condition, so as to determine the pseudo labeled sample.

[0145] Wherein, the pseudo label refers to the label marked for the unlabeled sample.

[0146] For example, assuming that the output results B1, B3 and B5 meeting the preset condition in the second output results B1-B6 predicted by the models M1-M6 for the unlabeled sample UD1, and the pseudo labels P1, P3 and P5 are determined based on the above-mentioned output results B1, B3 and B5 meeting the preset condition, then the pseudo labeled sample input into M1, M3 and M5 can be shown in Table 1 as follows:

[0147] Table 1

[0148]

[0149] Wherein, the above-mentioned P1, P3 and P5 are only an example for distinguishing, and in fact, they can also be the same label.

[0150] It can be understood that for multiple sample data in the unlabeled sample, the marking process of the above-mentioned unlabeled sample UD1 can be referred to, and finally the corresponding pseudo labeled sample can be determined for different models.

[0151] In the above-mentioned step 204, when determining the output result meeting the preset condition in the N second output results based on a preset algorithm, a method based on a fixed threshold can be used.

[0152] In an embodiment shown, it can be determined whether the second output result corresponding to each of the N picture classification models reaches the first threshold value set for each of the N picture classification models; the number of times the second output result reaches the first threshold value is determined; and in response to the number of times reaching a second threshold value, the second output result reaching the first threshold value is determined as the output result satisfying the preset condition.

[0153] The first threshold value set for each of the N picture classification models can be a threshold score used by the picture classification model when judging according to the predicted score, or a re-set threshold score, which can be set by the person skilled in the art, and the disclosure does not limit it.

[0154] For example, for each weak data augmentation sample, it can be determined whether the predicted score output by each picture classification model for the weak data augmentation sample reaches the classification threshold score set by the picture classification model, and the number of times the classification threshold score is reached is counted.

[0155] Further, it can be determined whether the number of times the predicted score reaches the classification threshold score reaches a threshold number of times, and if so, the predicted score reaching the classification threshold score is determined as the output result satisfying the preset condition.

[0156] In short, the results of classifying the same sample by multiple picture classification models are counted, and if the number of consistent classifications reaches a threshold value, the results reaching the threshold value are selected.

[0157] In the above process, for single-label tasks, a threshold number of times corresponding to the label can be set, and for multi-label classification tasks, a threshold number of times can be set for each label.

[0158] In addition to the above method based on a fixed threshold value, a method based on a dynamic threshold value can also be used.

[0159] In an embodiment shown, a first loss value corresponding to each picture classification model can be determined; wherein the first loss value includes a loss value determined when the picture classification model is iteratively trained based on the labeled sample until the first loss function corresponding to the picture classification model converges; the fourth output result and the fifth output result are input into a second loss function to calculate a second loss value corresponding to each picture classification model, respectively; and in response to the second loss value being less than the first loss value, the fourth output result and the fifth output result are determined as the output result satisfying the preset condition.

[0160] The loss value when the model converges in supervised training can be used as a judgment basis for unlabeled labeling of the unlabeled data.

[0161] For example, each picture classification model can be iteratively trained based on the labeled samples until the first loss function corresponding to the picture classification model converges, and the loss value when the convergence is determined as the first loss value corresponding to the picture classification model.

[0162] Continuing the example, for each unlabeled sample, after data augmentation processing, it can be divided into weak data augmentation samples and strong data augmentation samples, and after being input into the picture classification model for prediction, the third output result corresponding to the weak data augmentation sample and the fourth output result corresponding to the strong data augmentation sample can be obtained. According to the foregoing, by inputting the third output result into the preset knowledge transfer model for fusion, the fifth output result corresponding to the weak data augmentation sample can be determined.

[0163] Then, the fourth output result and the fifth output result are input into the second loss function for calculation to determine the second loss value corresponding to the picture classification model respectively.

[0164] Further, the second loss value and the first loss value are judged in size, if the second loss value is less than the first loss value, then the fourth output result and the fifth output result are determined as the output result satisfying the preset condition, that is, the same pseudo label is marked for the weak data augmentation sample and the strong data augmentation sample.

[0165] It should be noted that when the constructed picture classification model is more, a mixed method of fixed threshold and dynamic threshold can be used, some of the models can use the fixed threshold method to determine the pseudo label, and the other models can use the dynamic threshold method to determine the pseudo label.

[0166] In one embodiment shown, the same pseudo label can be marked for the second data augmentation sample based on the fourth output result and for the first data augmentation sample based on the fifth output result to determine the second data augmentation sample marked with the pseudo label and the first data augmentation sample marked with the pseudo label.

[0167] As known from the foregoing, in response to the second loss value being less than the first loss value, the fourth output result and the fifth output result can be determined as the output result satisfying the preset condition to facilitate further determination of the pseudo label.

[0168] For example, the pseudo label can be marked for the strong data augmentation sample based on the fourth output result, and the pseudo label can be marked for the weak data augmentation sample based on the fifth output result. Although the weak data augmentation sample and the strong data augmentation sample are different in the degree of data change, the final classification results are consistent, and therefore the marked pseudo labels are the same.

[0169] In step 205, the N picture classification models are trained based on a semi-supervised learning method according to the labeled samples and the pseudo-labeled samples, and N trained picture classification models are obtained.

[0170] In the embodiment, after the pseudo label corresponding to the unlabeled sample is determined, the N picture classification models are trained based on semi-supervised learning, and N trained picture classification models are obtained.

[0171] In the above process, the unlabeled sample is predicted by the picture classification model, the prediction result is fused by the knowledge transfer model, the sample is screened according to the fused result, the screened result is marked with a pseudo label, and then input into the picture classification model for training, so as to select the sample with high confidence for training, and improve the generalization ability of the model.

[0172] In an embodiment, the N picture classification models are iteratively trained based on a semi-supervised learning method according to the labeled samples, the pseudo-labeled second data augmentation samples, and the pseudo-labeled first data augmentation samples, until a third loss function converges, and N trained picture classification models are obtained; wherein the value of the third loss function is determined based on the value of the first loss function and the value of the second loss function.

[0173] For example, the N picture classification models are iteratively trained based on a semi-supervised learning method according to the labeled samples, the pseudo-labeled weak data augmentation samples, and the pseudo-labeled strong data augmentation samples, until a third loss function converges, and N trained picture classification models are obtained.

[0174] For example, the third loss function can be determined according to the following formula:

[0175] Loss=αloss label +βloss unlabel

[0176] Wherein, α and β represent different weight values, and loss label represents the value of the first loss function, and loss unlabel represents the value of the second loss function.

[0177] Through the above technical solutions, on the one hand, since the unlabeled samples are not only easy to obtain but also have more data and can obtain more useful information than the labeled samples, based on the semi-supervised learning method, the defects of insufficient labeled samples can be made up, the labeling cost can be reduced, and the accuracy and generalization ability of the model can be improved. On the other hand, by performing knowledge transfer on the results of the unlabeled samples output by each picture classification model, noise data in the training can be filtered, and higher-quality pseudo-labeled samples can be screened out, so as to improve the precision of the picture classification model and solve the problem of poor generalization ability of the picture classification model. In addition, the accuracy of the model can be improved through end-to-end training, and the finally trained picture classification model can be used to complete the picture classification task.

[0178] Corresponding to the above picture classification method, the disclosure also provides a technical solution of picture classification model training.

[0179] Please refer to FIG. 3 , FIG. 3 is a flowchart of a picture classification model training method provided by an exemplary embodiment, and the method comprises the following steps:

[0180] Step 301, obtaining a training sample set, wherein the training sample set comprises unlabeled samples and labeled samples.

[0181] In an embodiment shown, the obtained unlabeled samples can be subjected to data enhancement to obtain first data enhanced samples and second data enhanced samples.

[0182] Step 302, inputting the unlabeled samples into N picture classification models respectively to determine corresponding first output results.

[0183] In an embodiment shown, the above step 302 further comprises:

[0184] The first data enhanced samples can be inputted into the N picture classification models respectively to determine corresponding third output results, and the second data enhanced samples can be inputted into the N picture classification models respectively to determine corresponding fourth output results.

[0185] Step 303, inputting the N first output results into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, corresponding second output results for each picture classification model.

[0186] In an embodiment shown, the above step 303 further comprises:

[0187] The N third output results are input into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a fifth output result corresponding to the first data enhancement sample for each picture classification model.

[0188] In step 304, based on a preset algorithm, an output result meeting a preset condition in the N second output results is determined, and a pseudo label is labeled for the unlabeled sample corresponding to the output result meeting the preset condition, so as to determine a pseudo labeled sample; wherein the pseudo label is determined based on the second output result.

[0189] In an embodiment shown, based on the preset algorithm, the output result meeting the preset condition in the N second output results is determined, including:

[0190] It is judged whether the second output result corresponding to the N picture classification models respectively reaches a first threshold value set for the N picture classification models respectively;

[0191] The number of times that the second output result reaches the first threshold value is determined.

[0192] In response to the number of times reaching a second threshold value, the second output result reaching the first threshold value is determined as the output result meeting the preset condition.

[0193] In another embodiment shown, based on the preset algorithm, the output result meeting the preset condition in the N second output results is determined, including:

[0194] A first loss value corresponding to each picture classification model respectively is determined; wherein the first loss value includes a loss value determined when the picture classification model is iteratively trained based on the labeled sample until a first loss function corresponding to the picture classification model converges;

[0195] The fourth output result and the fifth output result are input into a second loss function to respectively calculate a second loss value corresponding to each picture classification model respectively;

[0196] In response to the second loss value being less than the first loss value, the fourth output result and the fifth output result are determined as the output result meeting the preset condition.

[0197] Further, in an embodiment shown, for the unlabeled sample corresponding to the output result meeting the preset condition, a pseudo label is labeled to determine a pseudo labeled sample, including:

[0198] Based on the fourth output result being the second data enhanced sample, and based on the fifth output result being the first data enhanced sample labeled with the same pseudo label, to determine the second data enhanced sample labeled with the pseudo label, and the first data enhanced sample labeled with the pseudo label.

[0199] Step 305, based on the semi-supervised learning method, respectively training the N picture classification models according to the labeled sample and the pseudo labeled sample, to obtain the trained N picture classification models.

[0200] In the illustrated embodiment, the above step 305 further comprises:

[0201] According to the labeled sample, the second data enhanced sample labeled with the pseudo label, and the first data enhanced sample labeled with the pseudo label, based on the semi-supervised learning method, iteratively training the N picture classification models until the third loss function converges, to obtain the trained N picture classification models; wherein the value of the third loss function is determined based on the value of the first loss function and the value of the second loss function.

[0202] The specific details of training the picture classification model are described in detail in the previously described picture classification method flow, and those skilled in the art can refer to the foregoing description, which will not be repeated here.

[0203] Please refer to FIG. 4 , FIG. 4 is a schematic diagram of another picture classification model training method provided by an exemplary embodiment, which comprises the following steps:

[0204] Step 401, obtaining a training sample set.

[0205] Among them, the training sample set includes unlabeled samples and labeled samples.

[0206] Among them, the unlabeled sample is not only easy to obtain, but also does not need to be labeled, which can greatly reduce the labeling cost. Moreover, by adding unlabeled samples, the accuracy of the classification boundary can be improved, and the robustness of the model can be improved.

[0207] In addition, for the samples in the training sample set, the samples can also be preprocessed according to the foregoing embodiments, for example, the samples can be scaled or normalized.

[0208] As can be seen from the foregoing, because the number of samples constructed in some picture classification tasks is limited, the number of training samples can be increased by data enhancement operation.

[0209] At step 402, data augmentation is performed on the obtained unlabeled samples to obtain weak data augmented samples and strong data augmented samples.

[0210] The data augmentation refers to data processing on existing data to generate more values from limited data, thereby improving the effect of the model. According to different degrees of data change, the data augmented samples can be divided into weak data augmented samples and strong data augmented samples.

[0211] For example, common data augmentation methods mainly include cropping, flipping, rotating, scaling, shifting, and Gaussian noise, and the present disclosure does not limit the same.

[0212] In the above process, data augmentation on the samples can solve the problem of insufficient training samples, thereby improving the generalization ability of the model.

[0213] At step 403, the weak data augmented samples are respectively input into N picture classification models to determine corresponding third output results, and the strong data augmented samples are respectively input into the N picture classification models to determine corresponding fourth output results.

[0214] When constructing the N picture classification models, a person skilled in the art can select any known picture classification model, such as ResNet series, ResNeXt series, and Transformer series, and the present disclosure does not limit the same.

[0215] It should be noted that for different picture classification models, the threshold value used to determine the classification result according to the predicted score can be different.

[0216] For example, assuming that the classification threshold value of model A is 0.85, and the classification threshold value given by model B is 0.75, for the same picture, even if the predicted scores given are both 0.8, the classification results are not the same, which also means that the prediction accuracy of different models is different.

[0217] As described above, before determining the pseudo label corresponding to the unlabeled sample according to the knowledge transfer model, the output results of the picture classification models need to be input into the knowledge transfer model.

[0218] At step 404, the N third output results are input into a preset knowledge transfer model for fusion.

[0219] It should be noted that, since the data change degree of the strong data augmentation sample is large, the classification result accuracy given by the model is low in the initial stage of model training, so the weak data augmentation sample can be classified first. Therefore, for the sample processed by data augmentation, the output result input to the knowledge transfer model can be the third output result corresponding to the weak data augmentation sample.

[0220] In step 405, the knowledge transfer model determines the fifth output result corresponding to each weak data augmentation sample for each picture classification model.

[0221] Knowledge transfer refers to transferring knowledge between multiple fields or tasks, which simply means transferring the knowledge of field B to field A to improve the classification effect of field A.

[0222] For example, assuming N is 4, when the knowledge transfer model is fused, the third output result determined by model B, model C and model D for the weak data augmentation sample can be used to improve or optimize the learning effect of model A, and the fused fifth output result is output. The fifth output result of model B can be obtained by knowledge transfer of model A, model C and model D.

[0223] In the above process, by using knowledge transfer, existing knowledge data can be reused, and there is no need to spend cost to re-collect and label new data sets, which can enhance the generalization ability of the model.

[0224] Next, see steps 406-410, the method based on the dynamic threshold is used to determine the corresponding pseudo label for the unlabeled sample, wherein the pseudo label refers to the label used for labeling the unlabeled sample.

[0225] It should be noted that the person skilled in the art can also determine the corresponding pseudo label for the unlabeled sample according to the actual situation based on the method based on the fixed threshold, which is not described here.

[0226] In step 406, the fourth output result and the fifth output result are input to the second loss function, and the second loss value corresponding to each picture classification model is calculated respectively.

[0227] For each picture classification model, the fourth output result corresponding to the strong data augmentation sample can be obtained according to the above step 403, and the fifth output result corresponding to the weak data augmentation sample can be obtained according to the above step 405. By inputting the fourth output result and the fifth output result to the second loss function preset for the picture classification model, the second loss value corresponding to each picture classification model can be calculated.

[0228] According to the foregoing, the method based on the dynamic threshold value needs to compare the second loss value and the first loss value, so that the step 407 of determining the first loss value can be performed.

[0229] In step 407, the N picture classification models are iteratively trained based on the labeled samples until the first loss function corresponding to the picture classification model converges, and the first loss value corresponding to each picture classification model when it converges is determined.

[0230] At this point, the second loss value and the first loss value corresponding to each picture classification model are obtained, and step 408 can be performed.

[0231] In step 408, the size of the first loss value and the second loss value is determined.

[0232] If the second loss value is less than the first loss value, the fourth output result and the fifth output result are determined as the output result satisfying the preset condition, that is, the same pseudo label is marked for the weak data augmentation sample and the strong data augmentation sample.

[0233] In step 409, in response to the second loss value being less than the first loss value, the unlabeled sample inputting the model for training again is screened out, and the corresponding pseudo label is determined.

[0234] In response to the second loss value being less than the first loss value, the fourth output result and the fifth output result can be determined as the output result satisfying the preset condition so as to further determine the pseudo label.

[0235] In step 410, the pseudo label is marked for the weak data augmentation sample, and the pseudo label is marked for the strong data augmentation sample.

[0236] The pseudo label can be marked for the strong data augmentation sample based on the fourth output result, and the pseudo label can be marked for the weak data augmentation sample based on the fifth output result. It should be noted that although the weak data augmentation sample and the strong data augmentation sample differ in the degree of data change, the final classification result tends to be consistent, so the pseudo label marked is also the same.

[0237] After the pseudo label corresponding to the unlabeled sample is determined, the N picture classification models can be respectively subjected to semi-supervised learning together with the labeled samples.

[0238] In step 411, the N picture classification models are trained based on the semi-supervised learning method according to the labeled samples and the pseudo labeled samples.

[0239] According to the labeled samples, the strong data augmentation samples marked with the pseudo label, and the weak data augmentation samples marked with the pseudo label, the N picture classification models are iteratively trained based on the semi-supervised learning method.

[0240] In the above process, the unlabeled samples are predicted by the picture classification model, and the prediction results are fused by using the knowledge transfer model. According to the fused results, the samples are screened, the screened results are marked with pseudo labels, and then input into the picture classification model for training again, so as to select samples with high confidence for training, and improve the generalization ability of the model. In step 412, N picture classification models trained are obtained.

[0241] Through the above iterative training, until the third loss function converges, the N picture classification models trained can be obtained; wherein the value of the third loss function is determined based on the value of the first loss function and the value of the second loss function.

[0242] The above third loss function can be determined according to the following formula:

[0243] Loss = a loss label + b loss unlabel

[0244] Wherein a and b represent different weight values, and loss label represents the value of the first loss function, and loss unlabel represents the value of the second loss function. The specific details of the above picture classification model training method are described in detail in the picture classification method process described above. Those skilled in the art can refer to the foregoing description, and will not be repeated here.

[0245] Through the above technical solution, on the one hand, since the unlabeled samples are not only easy to obtain, but also have more data, more useful information can be obtained, so based on the semi-supervised learning method, not only the defects of insufficient labeled samples can be made up, but also the labeling cost can be reduced, and the accuracy and generalization ability of the model can be improved. On the other hand, by transferring the knowledge of the unlabeled samples output by each picture classification model, the noise data in the training can be filtered, and the pseudo labeled samples with higher quality can be selected, so as to improve the precision of the picture classification model and solve the problem of poor generalization ability of the picture classification model. In addition, through end-to-end training, the accuracy of the model can be improved, and the picture classification model finally trained can be used to complete the picture classification task.

[0246] Exemplary Apparatus

[0247] After introducing the method of the example embodiment of the present disclosure, next, please see FIG. 5 , FIG. 5 is a block diagram of a picture classification device provided by an example embodiment.

[0248] The implementation process of the functions and roles of each module in the following device is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here. For the device embodiment, since it basically corresponds to the method embodiment, please refer to the part of the method embodiment for the related description.

[0249] As shown in FIG. 5 The picture classification device 500 can include a picture acquisition module 501, an input module 502, and an output module 503. Among them:

[0250] The picture acquisition module 501 is configured to acquire a picture to be identified;

[0251] The input module 502 is configured to input the picture into a pre-trained picture classification model;

[0252] The output module 503 is configured to output the result of picture classification based on the trained picture classification model.

[0253] In an embodiment, the picture classification device 500 further includes a model training module 504 of the picture classification model, and the model training module 504 includes:

[0254] The first acquisition module 5041 acquires a training sample set, and the training sample set includes unlabeled samples and labeled samples;

[0255] The first determination module 5042 inputs the unlabeled samples into N picture classification models respectively, and determines the corresponding first output results;

[0256] The first fusion module 5043 inputs the N first output results into a preset knowledge transfer model for fusion, so that the knowledge transfer model determines the corresponding second output results for each picture classification model;

[0257] The first screening module 5044 determines the output results in the N second output results that meet the preset conditions based on a preset algorithm;

[0258] The first labeling module 5045 labels the unlabeled samples corresponding to the output results that meet the preset conditions with pseudo-labels to determine pseudo-labeled samples, wherein the pseudo-labels are determined based on the second output results;

[0259] The first training module 5046 trains the N picture classification models based on a semi-supervised learning method according to the labeled samples and the pseudo-labeled samples, and obtains the trained N picture classification models.

[0260] In an embodiment, the first screening module 5044 further:

[0261] determine whether the second output results respectively corresponding to the N picture classification models reach first thresholds respectively set for the N picture classification models;

[0262] determine a number of times that the second output results reach the first thresholds;

[0263] in response to the number reaching a second threshold, determine the second output results reaching the first thresholds as the output results meeting the preset condition.

[0264] In an embodiment, the model training module 504 further includes:

[0265] a first enhancement module 5047 configured to perform data enhancement on the obtained unlabeled samples to obtain first data enhanced samples and second data enhanced samples.

[0266] In an embodiment, the first determination module 5042 is further configured to:

[0267] input the first data enhanced samples into N picture classification models respectively to determine corresponding third output results;

[0268] input the second data enhanced samples into the N picture classification models respectively to determine corresponding fourth output results.

[0269] In an embodiment, the first fusion module 5043 is further configured to:

[0270] input the N third output results into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, fifth output results respectively corresponding to the first data enhanced samples for each picture classification model.

[0271] In an embodiment, the first screening module 5044 is further configured to:

[0272] determine first loss values respectively corresponding to each picture classification model; wherein the first loss values include loss values determined when the picture classification models are iteratively trained based on the labeled samples until first loss functions corresponding to the picture classification models converge;

[0273] input the fourth output results and the fifth output results into a second loss function to respectively calculate second loss values respectively corresponding to each picture classification model;

[0274] in response to the second loss values being smaller than the first loss values, determine the fourth output results and the fifth output results as the output results meeting the preset condition.

[0275] In an embodiment, the first marking module 5045 is further configured to:

[0276] based on the fourth output result being the second data enhanced sample and based on the fifth output result being the first data enhanced sample, the same pseudo label is marked for the second data enhanced sample and the first data enhanced sample to determine the second data enhanced sample marked with the pseudo label and the first data enhanced sample marked with the pseudo label.

[0277] In an embodiment, the first training module 5046 further:

[0278] According to the labeled sample, the second data enhanced sample marked with the pseudo label, and the first data enhanced sample marked with the pseudo label, the N picture classification models are iteratively trained based on a semi-supervised learning method respectively until a third loss function converges, and the N trained picture classification models are obtained; wherein the value of the third loss function is determined based on the value of the first loss function and the value of the second loss function.

[0279] The specific details of each module of the picture classification device 500 have been described in detail in the previously described correlation prediction model training method process, and therefore will not be described again here.

[0280] Corresponding to the foregoing correlation prediction method, please refer to FIG. 6 , FIG. 6 is a block diagram of a picture classification model training device provided by an exemplary embodiment.

[0281] The implementation process of the functions and roles of each module in the following device is specifically described in the implementation process of the corresponding steps in the above picture classification model training method, and will not be described again here. For the device embodiment, since it basically corresponds to the method embodiment, please refer to the part of the method embodiment for the related description.

[0282] As shown in FIG. 6 , the picture classification model training device 600 can include a second acquisition module 601, a second determination module 602, a second fusion module 603, a second screening module 604, a second marking module 605, and a second training module 606, wherein:

[0283] The second acquisition module 601 is configured to acquire a training sample set, wherein the training sample set includes unlabeled samples and labeled samples;

[0284] The second determination module 602 is configured to input the unlabeled samples into N picture classification models respectively, and determine corresponding first output results;

[0285] The second fusion module 603 is configured to input the N first output results into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a corresponding second output result for each picture classification model;

[0286] The second screening module 604 is configured to determine, based on a preset algorithm, an output result in the N second output results that satisfies a preset condition;

[0287] The second marking module 605 is configured to mark a pseudo label for the unlabeled sample corresponding to the output result that satisfies the preset condition, to determine a pseudo labeled sample; wherein the pseudo label is determined based on the second output result.

[0288] The second training module 606 is configured to train the N picture classification models based on a semi-supervised learning method according to the labeled sample and the pseudo labeled sample, to obtain trained N picture classification models.

[0289] The specific details of each module of the picture classification model training apparatus 600 have been described in detail in the description of the picture classification model training method flow, and thus will not be described here.

[0290] It should be noted that although the picture classification apparatus 500 and the modules or units of the picture classification model training apparatus 600 are mentioned in the foregoing detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0291] Exemplary Medium

[0292] After introducing the apparatus of the exemplary embodiments of the present disclosure, next, please refer to FIG. 7 , FIG. 7 is a schematic diagram of a computer readable storage medium provided in an exemplary embodiment.

[0293] In the exemplary embodiments of the present disclosure, a computer readable storage medium is also provided, on which a program product capable of implementing the method described above is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code for causing the terminal device to perform the steps described in the “Exemplary Method” section above according to various exemplary embodiments of the present disclosure when the program product is run on the terminal device.

[0294] ReferenceFIG. 7 As shown, a readable storage medium 70 for implementing the above method according to the embodiments of the present disclosure is described, which can employ a portable compact disc read-only memory (CD-ROM) and include program codes, and can be run on a terminal device, such as a personal computer. However, the readable storage medium of the present disclosure is not limited thereto, and in the present document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with an instruction execution system, apparatus or device.

[0295] The readable storage medium can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0296] The computer readable signal medium can include a data signal propagated in a baseband or propagated as a carrier wave in a propagated data signal, in which the readable program codes are carried. Such a propagated data signal can employ various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit the program for use by or in conjunction with an instruction execution system, apparatus or device.

[0297] The program codes contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0298] Program code to implement an operation of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0299] Exemplary Computing Device

[0300] Having introduced the method, medium and apparatus of the exemplary embodiments of the present disclosure, next, please refer to FIG. 8 , FIG. 8 is a schematic diagram of an electronic device capable of implementing the above method according to an exemplary embodiment.

[0301] The electronic device 800 according to this embodiment of the present disclosure will be described below with reference to FIG. 8 FIG. 8 The electronic device 800 shown is merely an example and should not bring any limitation to the function and use range of the embodiments of the present disclosure.

[0302] As shown in FIG. 8 , the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 can include, but are not limited to, the at least one processing unit 801, the at least one storage unit 802, and a bus 803 connecting different system components, including the storage unit 802 and the processing unit 801.

[0303] The storage unit stores program code which can be executed by the processing unit 801, so that the processing unit 801 performs the steps of the various embodiments described in the present specification.

[0304] The storage unit 802 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 8021 and / or a cache memory 8022, and can further include a read-only memory (ROM) 8023.

[0305] ​The storage unit 802 also includes a number of program modules 8025 that are stored in the memory 8024, including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or a combination of which can embody the functionality described herein.

[0306] The bus 803 can represent one or more of several types of bus structures, including a storage unit bus or bus for storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus structures.

[0307] The electronic device 800 can also communicate with one or more external devices 804 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with the electronic device 800; and / or one or more devices that enable the electronic device 800 to communicate with one or more other computing devices. Such communication can be via an input / output (I / O) interface 805. Similarly, the electronic device 800 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through a network adapter 806. The network adapter 806 can be any of a variety of modems, including cable modem, digital subscriber line (DSL), and / or the like. As shown, the network adapter 806 is communicatively coupled to the other components of the electronic device 800 through the bus 803. It should be appreciated that the electronic device 800 can be connected to other types of hardware and / or software modules (including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.) that can also be considered "non-volatile storage" under some aspects.

[0308] Those skilled in the art will readily understand that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0309] It should be noted that although several elements / modules or sub-elements / modules of the apparatus are referred to in the foregoing detailed description, such division into elements / modules or sub-elements / modules is merely exemplary and not mandatory. Indeed, according to an embodiment of the present disclosure, features and functions of two or more elements / modules described above can be embodied in a single element / module. Conversely, features and functions of one element / module described above can be further divided into multiple elements / modules.

[0310] Moreover, while operations of the methods of the present disclosure are described in a particular order in the drawings, this is not required or implied, and the desired results can be achieved without performing all of the operations shown, or performing the operations in the particular order shown. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or divided into multiple steps.

[0311] While the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it is to be understood that the disclosure is not limited to the specific embodiments disclosed and that the division of aspects is not meant to imply that features from these aspects cannot be combined to benefit. The disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A picture classification method, comprising: obtaining a picture to be identified; inputting the picture into a pre-trained picture classification model; outputting a result of picture classification based on the trained picture classification model; a training process of the picture classification model comprises: obtaining a training sample set, the training sample set comprising unlabeled samples and labeled samples; performing data enhancement on the obtained unlabeled samples to obtain first data enhanced samples and second data enhanced samples; inputting the unlabeled samples into N picture classification models respectively to determine corresponding first output results, comprising: inputting the first data enhanced samples into the N picture classification models respectively to determine corresponding third output results, and inputting the second data enhanced samples into the N picture classification models respectively to determine corresponding fourth output results; inputting the N first output results into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a corresponding second output result for each picture classification model, comprising: inputting the N third output results into the preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a fifth output result corresponding to the first data enhanced samples for each picture classification model respectively; determining, based on a preset algorithm, an output result in the N second output results that meets a preset condition, and labeling a pseudo label for the unlabeled sample corresponding to the output result that meets the preset condition, to determine a pseudo labeled sample; wherein the pseudo label is determined based on the second output result; the determining, based on the preset algorithm, of the output result in the N second output results that meets the preset condition comprises: determining a first loss value corresponding to each picture classification model respectively; wherein the first loss value comprises a loss value determined when a first loss function corresponding to the picture classification model converes based on iterative training of the picture classification model based on the labeled samples; inputting the fourth output result and the fifth output result into a second loss function to calculate a second loss value corresponding to each picture classification model respectively; in response to the second loss value being less than the first loss value, determining the fourth output result and the fifth output result as the output result that meets the preset condition; training the N picture classification models based on a semi-supervised learning method respectively according to the labeled samples and the pseudo labeled samples, to obtain N trained picture classification models.

2. The method of claim 1, wherein labeling a pseudo label for the unlabeled sample corresponding to the output result that meets the preset condition to determine a pseudo labeled sample comprises: labeling the same pseudo label for the second data enhanced sample based on the fourth output result and for the first data enhanced sample based on the fifth output result, to determine a second data enhanced sample labeled with a pseudo label and a first data enhanced sample labeled with a pseudo label.

3. The method of claim 2, wherein the N image classification models are trained based on a semi-supervised learning method according to the labeled samples and the pseudo-labeled samples, and the trained N image classification models are obtained, comprising: the N image classification models are iteratively trained based on a semi-supervised learning method according to the labeled samples, the pseudo-labeled second data augmented samples, and the pseudo-labeled first data augmented samples, until a third loss function converges, and the trained N image classification models are obtained; wherein the value of the third loss function is determined based on the value of the first loss function and the value of a second loss function.

4. An image classification model training method, comprising: obtaining a training sample set, wherein the training sample set comprises unlabeled samples and labeled samples; performing data augmentation on the obtained unlabeled samples to obtain first data augmented samples and second data augmented samples; inputting the unlabeled samples into N image classification models respectively to determine corresponding first output results, comprising: inputting the first data augmented samples into the N image classification models respectively to determine corresponding third output results, and inputting the second data augmented samples into the N image classification models respectively to determine corresponding fourth output results; inputting the N first output results into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a corresponding second output result for each image classification model, comprising: inputting the N third output results into the preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a fifth output result corresponding to the first data augmented samples for each image classification model respectively; determining, based on a preset algorithm, an output result in the N second output results that meets a preset condition, and labeling a pseudo-label for the unlabeled sample corresponding to the output result that meets the preset condition, to determine a pseudo-labeled sample; wherein the pseudo-label is determined based on the second output result; the determination of the output result in the N second output results that meets the preset condition based on the preset algorithm comprises: determining a first loss value corresponding to each image classification model respectively; wherein the first loss value comprises a loss value determined when the image classification model is iteratively trained based on the labeled samples until a first loss function corresponding to the image classification model converges; inputting the fourth output result and the fifth output result into a second loss function to calculate a second loss value corresponding to each image classification model respectively; in response to the second loss value being less than the first loss value, the fourth output result and the fifth output result are determined as the output result that meets the preset condition; and training the N image classification models based on a semi-supervised learning method according to the labeled samples and the pseudo-labeled samples, to obtain trained N image classification models.

5. An image classification device, comprising: an image acquisition module configured to acquire an image to be recognized; and an input module configured to input the image into a pre-trained image classification model. ​ ​ ​ ​ ​ ​ ​ ​ ​ An output module, based on the trained picture classification model, outputs a result of picture classification; Further comprising a model training module of the picture classification model, the model training module comprises: A first acquisition module acquires a training sample set, the training sample set comprising unlabeled samples and labeled samples; A first enhancement module performs data enhancement on the acquired unlabeled samples to obtain first data enhanced samples and second data enhanced samples; A first determination module inputs the unlabeled samples into N picture classification models respectively to determine corresponding first output results, including: inputting the first data enhanced samples into the N picture classification models respectively to determine corresponding third output results, and inputting the second data enhanced samples into the N picture classification models respectively to determine corresponding fourth output results; A first fusion module inputs the N first output results into a preset knowledge transfer model for fusion, so that the knowledge transfer model determines corresponding second output results for each picture classification model, including: inputting the N third output results into the preset knowledge transfer model for fusion, so that the knowledge transfer model determines corresponding fifth output results of the first data enhanced samples for each picture classification model respectively; A first screening module determines output results meeting preset conditions in the N second output results based on a preset algorithm, including: determining a first loss value corresponding to each picture classification model respectively; wherein the first loss value comprises a loss value determined when the picture classification model is iteratively trained based on the labeled samples until a first loss function corresponding to the picture classification model converges; inputting the fourth output results and the fifth output results into a second loss function to calculate a second loss value corresponding to each picture classification model respectively; in response to the second loss value being less than the first loss value, determining the fourth output results and the fifth output results as the output results meeting the preset conditions; A first labeling module labels pseudo labels for the unlabeled samples corresponding to the output results meeting the preset conditions to determine pseudo labeled samples; wherein the pseudo labels are determined based on the second output results; A first training module trains the N picture classification models based on a semi-supervised learning method according to the labeled samples and the pseudo labeled samples to obtain trained N picture classification models.

6. The apparatus of claim 5, the first labeling module comprising: labeling the same pseudo labels for the second data enhanced samples based on the fourth output results and for the first data enhanced samples based on the fifth output results to determine second data enhanced samples labeled with pseudo labels and first data enhanced samples labeled with pseudo labels.

7. The apparatus of claim 6, the first training module comprising: According to the labeled sample, the pseudo-labeled second data enhanced sample, and the pseudo-labeled first data enhanced sample, the N picture classification models are iteratively trained based on a semi-supervised learning method respectively until a third loss function converges, and the N trained picture classification models are obtained; wherein the value of the third loss function is determined based on the value of the first loss function and the value of the second loss function.

8. An image classification model training apparatus, comprising: a second acquisition module configured to acquire a training sample set, the training sample set comprising unlabeled samples and labeled samples; a first enhancement module configured to perform data enhancement on the acquired unlabeled samples to obtain first data enhanced samples and second data enhanced samples; a second determination module configured to input the unlabeled samples into N image classification models respectively to determine corresponding first output results, including inputting the first data enhanced samples into the N image classification models respectively to determine corresponding third output results, and inputting the second data enhanced samples into the N image classification models respectively to determine corresponding fourth output results; a second fusion module configured to input the N first output results into a preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a corresponding second output result for each image classification model, including inputting the N third output results into the preset knowledge transfer model for fusion, so as to determine, by the knowledge transfer model, a corresponding fifth output result for the first data enhanced sample for each image classification model respectively; a second screening module configured to determine, based on a preset algorithm, output results in the N second output results that meet a preset condition, including determining a first loss value corresponding to each image classification model respectively; wherein the first loss value comprises a loss value determined when the image classification model is iteratively trained based on the labeled samples until a first loss function corresponding to the image classification model converges; inputting the fourth output results and the fifth output results into a second loss function to calculate a second loss value corresponding to each image classification model respectively; and in response to the second loss value being less than the first loss value, determining the fourth output results and the fifth output results as the output results that meet the preset condition; a second labeling module configured to label pseudo-labels for the unlabeled samples corresponding to the output results that meet the preset condition to determine pseudo-labeled samples; wherein the pseudo-labels are determined based on the second output results; a second training module configured to train the N image classification models based on a semi-supervised learning method respectively according to the labeled samples and the pseudo-labeled samples, and obtain N trained image classification models.

9. A storage medium having computer instructions stored thereon, the instructions being executed by a processor to implement the steps of the method of any one of claims 1-3 or claim 4.

10. A computing device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of any of claims 1-3 or claim 4 by executing the executable instructions.

Images