Training method of image classification model, image classification method and related equipment

Abstract

The application discloses a training method of an image classification model, an image classification method and related equipment. The image classification model comprises a first image classification submodel and a second image classification submodel. The training method comprises the following steps: acquiring an image set; performing classification and identification on labeled images and unlabeled images in the image set by using a target image classification submodel, respectively, to obtain first classification reference information of the labeled images and the unlabeled images; determining a classification loss of the target image classification submodel based on the first classification reference information corresponding to the labeled images, category labels and second classification reference information of the unlabeled images; the second classification reference information of the unlabeled images is obtained by performing classification and identification on the unlabeled images by using other image classification submodels except the target image classification submodel; and adjusting model parameters of the image classification model based on the classification losses of the first image classification submodel and the second image classification submodel.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method for an image classification model, an image classification method, and related equipment. Background Technology

[0002] Semi-supervised learning (SSL) is a key research area in pattern recognition and machine learning, combining supervised and unsupervised learning methods. In recent years, semi-supervised learning has been widely applied in fields such as image classification.

[0003] Current image classification models typically consist of a teacher network and a student network. The teacher network is usually pre-trained and remains unchanged during semi-supervised learning, only used to generate pseudo-labels for sample images. The student network uses these pseudo-labeled sample images for semi-supervised learning to obtain the final image classification model. However, in this process, since the pseudo-labels are entirely generated by the teacher network from the information learned from the sample images, it is not conducive to the student network fully mining and utilizing the information contained in the sample images. Especially in the early stages of training, the confidence of the pseudo-labels generated by the teacher network is low, resulting in poor training performance of the image classification model, which in turn affects the accuracy and stability of the final image classification. Summary of the Invention

[0004] The purpose of this application is to provide a training method for an image classification model, an image classification method, and related equipment to solve the problem that the training effect of existing image classification models is not good, which affects the accuracy and stability of the final image classification.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a method for training an image classification model, wherein the image classification model includes a first image classification sub-model and a second image classification sub-model, and the method includes:

[0007] Obtain an image set for training the image classification model, the image set including labeled images, unlabeled images, and category labels corresponding to the labeled images;

[0008] The labeled image and the unlabeled image are classified and identified by the target image classification sub-model in the image classification model to obtain the first classification reference information of the labeled image and the first classification reference information of the unlabeled image; the target image classification sub-model is either the first image classification sub-model or the second image classification sub-model.

[0009] Based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image, the classification loss of the target image classification sub-model is determined; the second classification reference information of the unlabeled image is obtained by classifying and recognizing the unlabeled image through other image classification sub-models besides the target image classification sub-model in the image classification model; the classification loss of the target image classification sub-model refers to the classification loss of the first image classification sub-model or the classification loss of the second image classification sub-model;

[0010] The model parameters of the image classification model are adjusted based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model.

[0011] As can be seen, in this embodiment of the application, under the semi-supervised learning framework, each image in the image classification model is classified and identified by each image classification sub-model in the image classification model, resulting in multiple classification reference information for each image, with one classification reference information corresponding to one image classification sub-model. Then, each image classification sub-model is used as the target image classification sub-model. Based on the classification reference information of labeled images in the image set corresponding to the target image classification sub-model, the category labels of labeled images, and the classification reference information of unlabeled images in the image set corresponding to another image classification sub-model, the classification loss of the target image classification sub-model is determined, that is, using one The information learned by the target image classification sub-model from the image set provides guidance for another target image classification sub-model, transforming the one-way teacher-student relationship between the target image classification sub-models into a mutual teacher-student relationship. Furthermore, by adjusting the model parameters of the image classification model based on the classification loss of each target image classification sub-model, the mutual teacher-student relationship between the target image classification sub-models can be fully utilized, enabling them to learn from each other and complement each other. This allows the information contained in the image set to be fully explored and utilized, thereby improving the training effect of the image classification model and obtaining a more accurate and reliable image classification model.

[0012] Secondly, embodiments of this application provide an image classification method, including:

[0013] An image classification model is used to classify and identify the image to be processed, thereby obtaining a classification reference information set for the image to be processed. The classification reference information set includes first target classification reference information and second target classification reference information. The image classification model includes a first image classification sub-model and a second image classification sub-model. The first image classification sub-model is used to classify and identify the image to be processed to obtain the first target classification reference information, and the second image classification sub-model is used to classify and identify the image to be processed to obtain the second target classification reference information. The image classification model is trained based on the training method described in the first aspect.

[0014] Based on the classification reference information set of the image to be processed, the category to which the image to be processed belongs is determined.

[0015] As can be seen, in this embodiment, the image classification model is trained by utilizing the teacher-student relationship between various image classification sub-models on the basis of semi-supervised learning, through complementary learning and mutual teaching and learning among the various image classification sub-models. Therefore, the image classification model has high accuracy and reliability. Furthermore, using the image classification model to classify and recognize the image to be processed helps to improve the accuracy and reliability of the image classification results.

[0016] Thirdly, embodiments of this application provide a training apparatus for an image classification model, wherein the image classification model includes a first image classification sub-model and a second image classification sub-model, and the training apparatus includes:

[0017] The acquisition unit is used to acquire an image set for training the image classification model, the image set including labeled images, unlabeled images, and category labels corresponding to the labeled images;

[0018] The classification unit is used to classify and identify the labeled image and the unlabeled image respectively through the target image classification sub-model in the image classification model, and obtain the first classification reference information of the labeled image and the first classification reference information of the unlabeled image; the target image classification sub-model is the first image classification sub-model or the second image classification sub-model.

[0019] A determining unit is configured to determine the classification loss of the target image classification sub-model based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image; the second classification reference information of the unlabeled image is obtained by classifying and recognizing the unlabeled image through other image classification sub-models in the image classification model besides the target image classification sub-model; the classification loss of the target image classification sub-model refers to the classification loss of the first image classification sub-model or the classification loss of the second image classification sub-model; an adjusting unit is configured to adjust the model parameters of the image classification model based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model.

[0020] Fourthly, embodiments of this application provide an image classification apparatus, comprising:

[0021] A classification unit is used to classify and identify the image to be processed using an image classification model to obtain a classification reference information set for the image to be processed; wherein, the classification reference information set includes first target classification reference information and second target classification reference information, the image classification model includes a first image classification sub-model and a second image classification sub-model, the first image classification sub-model is used to classify and identify the image to be processed to obtain the first target classification reference information, the second image classification sub-model is used to classify and identify the image to be processed to obtain the second target classification reference information, and the image classification model is trained based on the training method described in the first aspect;

[0022] The determining unit is used to determine the category to which the image to be processed belongs based on the classification reference information set of the image to be processed.

[0023] Fifthly, embodiments of this application provide an electronic device, including:

[0024] processor;

[0025] Memory used to store the processor's executable instructions;

[0026] The processor is configured to execute the instructions to implement the method as described in the first or second aspect.

[0027] In a sixth aspect, embodiments of this application provide a computer-readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method described in the first or second aspect. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0029] Figure 1 A flowchart illustrating a training method for an image classification model provided in one embodiment of this application;

[0030] Figure 2 A flowchart illustrating a training method for an image classification model provided in another embodiment of this application;

[0031] Figure 3 A flowchart illustrating an image classification method provided in one embodiment of this application;

[0032] Figure 4 A schematic diagram of the structure of a training device for an image classification model provided in one embodiment of this application;

[0033] Figure 5 A schematic diagram of the structure of an image classification device provided in one embodiment of this application;

[0034] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] The terms "first," "second," etc., used in this specification and claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this specification and claims, "and / or" indicates at least one of the objects in the two sections, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0037] As mentioned earlier, existing image classification model training methods involve a pre-trained teacher network generating pseudo-labels for sample images, followed by a student network performing semi-supervised learning using these pseudo-labeled sample images to form the final image classification model. However, since the pseudo-labels are entirely generated by the teacher network from information learned from the sample images, this hinders the student network from fully exploring and utilizing the information contained in the sample images. Especially in the early stages of training, the confidence level of the pseudo-labels generated by the teacher network is low, leading to poor training performance of the image classification model and consequently affecting the accuracy and stability of the final image classification.

[0038] In view of this, this application proposes a training method for an image classification model. Under a semi-supervised learning framework, the one-way teacher-student relationship between the image classification sub-models of the image classification model is improved into a mutual teacher-student relationship. Information learned by one image classification sub-model from sample images is used to provide pseudo-labels for another image classification sub-model to perform semi-supervised learning. This allows the image classification sub-models to learn from each other and complement each other, thereby enabling the information contained in the sample images to be fully explored and utilized, thus improving the training effect of the image classification model and obtaining a more accurate and reliable image classification model.

[0039] This application also proposes an image classification method, which can accurately classify and identify images using a trained image classification model.

[0040] It should be understood that the training method and image classification method of the image classification model provided in the embodiments of this application can be executed by an electronic device or software installed in an electronic device. The electronic device referred to herein may include terminal devices, such as smartphones, tablets, laptops, desktop computers, smart voice interaction devices, smart home appliances, smartwatches, vehicle terminals, aircraft, etc.; or, the electronic device may also include a server, such as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0041] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0042] Please refer to Figure 1 The following is a flowchart illustrating a training method for an image classification model, provided as an embodiment of this application. The method may include the following steps:

[0043] S102, Obtain the image set used to train the image classification model.

[0044] The image set includes labeled images, unlabeled images, and category labels corresponding to the labeled images.

[0045] Labeled images are images with corresponding category labels, while unlabeled images are images without corresponding category labels. In practical applications, to further improve the classification accuracy of image classification models, image sets can include multiple labeled images and multiple unlabeled images, and the multiple labeled images can belong to different categories.

[0046] In this context, the category label of a labeled image represents the true category to which the labeled image belongs, specifically the true category to which the content presented by the labeled image belongs. For example, the category to which the labeled image belongs can be people, animals, landscapes, etc.; or, the category to which the labeled image belongs can be a subcategory of a larger category. For example, for the category of people, the category to which the labeled image belongs could be depressed, happy, angry, etc., or it could be real faces, fake faces, etc.; or, for the category of animals, the category to which the labeled image belongs could be cats, dogs, horses, mules, etc. In practical applications, the category label corresponding to the labeled image can have any appropriate form. For example, the category label corresponding to the labeled image can be obtained by one-hot encoding of the true category to which the labeled image belongs, or it can be obtained by word embedding of the true category to which the labeled image belongs. This application embodiment does not limit the form of the category label.

[0047] S104, using the target image classification sub-model in the image classification model, classify and identify labeled and unlabeled images respectively, and obtain the first classification reference information for labeled images and the first classification reference information for unlabeled images.

[0048] In order to train a highly accurate and reliable image classification model with a limited number of labeled images, such as Figure 2 As shown, the image classification model in this embodiment may include a first image classification sub-model and a second image classification sub-model. Both the first and second image classification sub-models can classify and identify each image in the image set to obtain corresponding classification reference information. Based on this, a semi-supervised learning method is used to train the image classification model to obtain the final image classification model. In practical applications, the first and second image classification sub-models may have the same network structure. Alternatively, to simplify the model structure and achieve compression and acceleration of the image classification model, the first and second image classification sub-models may have different network structures. For example, the second image classification sub-model may use a more streamlined structure compared to the first image classification sub-model.

[0049] In this embodiment, the target image classification sub-model is either a first image classification sub-model or a second image classification sub-model. That is, the first image classification sub-model and the second image classification sub-model can be used as the target image classification sub-model, respectively. Through the above-described S104, the classification reference information of labeled images corresponding to each image classification sub-model and the classification reference information of unlabeled images corresponding to each image classification sub-model are identified.

[0050] For ease of distinction, the classification reference information of a labeled image corresponding to the target image classification sub-model is called the first classification reference information of the labeled image, and the classification reference information of a labeled image corresponding to another image classification sub-model in the image classification model besides the target image classification sub-model is called the second classification reference information of the labeled image. Similarly, the classification reference information of an unlabeled image corresponding to the target image classification sub-model is called the first classification reference information of the unlabeled image, and the classification reference information of an unlabeled image corresponding to another image classification sub-model in the image classification model besides the target image classification sub-model is called the second classification reference information of the unlabeled image.

[0051] In practical applications, the classification reference information for labeled images may include at least one of the following: the probability that a labeled image is identified as belonging to each of multiple preset categories, the category to which the labeled image belongs, etc. Similarly, the classification reference information for unlabeled images may include at least one of the following: the probability that an unlabeled image is identified as belonging to each of multiple preset categories, the category to which the unlabeled image belongs, etc. For example, the multiple preset categories include cat, dog, horse, and mule. The classification reference information for each image may include the probability that each image is identified as belonging to cat, dog, horse, or mule, respectively, and the category to which each image belongs may be the category with the highest probability among these multiple preset categories.

[0052] To enable each image classification sub-model in the image classification model to fully understand and learn from the images in the image set, thereby improving the expressive power of the image classification model, the images in the image set can be obtained by data augmentation processing of the initial images. That is, prior to S102 above, the training method for the image classification model provided in this application embodiment may further include: performing data augmentation processing on the images in the initial image set to obtain an image set for training the image classification model, thereby making the images in the obtained image set contain perturbation information. The initial image set includes initial unlabeled images and initial labeled images.

[0053] Specifically, for the initial unlabeled image, data augmentation processing of various enhancement levels can be applied to the initial unlabeled image to obtain multiple unlabeled images, each corresponding to an enhancement intensity. Correspondingly, in S104 above, the labeled image can be classified and identified using a target image classification sub-model to obtain first classification reference information for the labeled image; and each of the multiple unlabeled images can be classified and identified using the target image classification sub-model to obtain first classification reference information for each unlabeled image.

[0054] More specifically, the initial unlabeled image undergoes data augmentation processing at various levels of enhancement. This can be implemented as follows: weakly-augmented processing is applied to the initial unlabeled image to obtain a first-type unlabeled image, and strongly-augmented processing is applied to the initial unlabeled image to obtain a second-type unlabeled image. That is, the unlabeled images include both the first-type and second-type unlabeled images, with the enhancement level corresponding to the first-type unlabeled image being lower than that corresponding to the second-type unlabeled image.

[0055] For example, weak enhancement processing may include, but is not limited to, at least one of the following processing methods: translation, flipping, etc., while strong enhancement processing may include, but is not limited to, at least one of the following processing methods: occlusion, color transformation, random erase, etc.

[0056] Understandably, because weak enhancement processing introduces less perturbation into the initial unlabeled image, it avoids distorting the first-class unlabeled image. This allows the target image classification sub-model to learn the noise in the first-class unlabeled image based on accurate first-class classification reference information, thus improving the learning effect of the target image classification sub-model. Furthermore, considering that using only weakly enhanced images may cause the target image classification sub-model to fall into overfitting and fail to extract the essential features of the first-class unlabeled image, while strong enhancement processing introduces greater perturbation and may distort the first-class unlabeled image, it can still retain features sufficient to identify the category. By applying weak and strong enhancement processing to the initial unlabeled image respectively and then inputting it into the target image classification sub-model, the target image classification sub-model can learn from unlabeled images with different enhancement levels, which helps improve the learning effect of the target image classification sub-model and enhance its expressive power.

[0057] Optionally, to further improve the expressive power of the target image classification sub-model, before S104 above, the training method of the image classification model provided in this application embodiment may further include: performing weak enhancement processing on the initial labeled images in the initial image set to obtain labeled images.

[0058] S106. Based on the first classification reference information of the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image, determine the classification loss of the target image classification sub-model.

[0059] The second classification reference information for unlabeled images is obtained by classifying and recognizing unlabeled images through other image classification sub-models besides the target image classification sub-model in the image classification model.

[0060] In other words, in S106 above, for the first image classification sub-model, the classification loss of the first image classification sub-model can be determined based on the classification reference information of the labeled images in the image set corresponding to the first image classification sub-model, the category labels of the labeled images, and the classification reference information of the unlabeled images in the image set corresponding to the second image classification sub-model; for the second image classification sub-model, the classification loss of the second image classification sub-model can be determined based on the classification reference information of the labeled images in the image set corresponding to the second image classification sub-model, the category labels of the labeled images, and the classification reference information of the unlabeled images in the image set corresponding to the first image classification sub-model.

[0061] In this way, the first and second image classification sub-models can use the information they have learned to provide guidance to each other, changing the one-way teacher-student relationship between the first and second image classification sub-models into a mutual teacher-student relationship. This is conducive to complementary learning and mutual improvement among the various image classification sub-models, allowing the information contained in the images in the image set to be fully explored and utilized, thereby improving the training effect of the image classification model.

[0062] In this embodiment of the application, for each image classification sub-model, the classification loss of the image classification sub-model is used to represent the difference between the classification reference information obtained by the image classification sub-model in classifying and recognizing the input image and the category represented by the category label corresponding to the input image.

[0063] Considering that each image classification sub-model performs a semi-supervised learning task based on the input image set, which combines a supervised learning task based on labeled images and their corresponding category labels with an unsupervised learning task based on unlabeled images, each learning task may produce a certain classification loss. Therefore, the classification loss of the target image classification sub-model can include the supervised loss and the unsupervised loss of the target image classification sub-model. The supervised loss of the target image classification sub-model is used to represent the classification loss produced by the target image classification sub-model in the supervised learning task, and the unsupervised loss of the target image classification sub-model is used to represent the classification loss produced by the target image classification sub-model in the unsupervised learning task.

[0064] In one alternative implementation, the supervised loss of the target image classification sub-model can be determined based on the first classification reference information of the labeled images in the image set and the category labels corresponding to the labeled images, while the unsupervised loss of the target image classification sub-model can be determined based on the first classification reference information and the second classification reference information of the unlabeled images in the image set.

[0065] In another optional implementation, to enable the target image classification sub-model to fully understand and learn from the input images, thereby improving its expressive power, unlabeled images are input into the target image classification sub-model after undergoing various levels of data augmentation processing on the initial unlabeled images. This results in the first classification reference information for each unlabeled image, including the first classification reference information for each unlabeled image. Correspondingly, the supervised loss of the target image classification sub-model can be determined based on the first classification reference information of the labeled images in the image set and the corresponding category labels of the labeled images. The unsupervised loss of the target image classification sub-model can be determined based on the first classification reference information and the second classification reference information of the unlabeled images in the image set.

[0066] More specifically, the aforementioned unlabeled images include a first type of unlabeled images and a second type of unlabeled images. The enhancement level corresponding to the first type of unlabeled images is less than the enhancement level corresponding to the second type of unlabeled images. Accordingly, the aforementioned S106 may specifically include the following steps:

[0067] S161, Based on the second classification reference information of the first type of unlabeled images, generate the first pseudo-label corresponding to the first type of unlabeled images.

[0068] Since the first type of unlabeled images does not have corresponding category labels, a first-class pseudo-label is generated based on the second classification reference information of the first type of unlabeled images. This is equivalent to manually labeling the first type of unlabeled images to indicate their predicted category, thus providing guidance for the unsupervised learning task of the target image classification sub-model. In practical applications, the first pseudo-label corresponding to the first type of unlabeled images can be used to indicate the predicted category to which the first type of unlabeled images belong. Of course, the first-class pseudo-label corresponding to the first type of unlabeled images can also be used to indicate the first target object region in the first type of unlabeled images and the predicted category to which the first target object region belongs. Here, the first target object region refers to the region where the target object is located in the first type of unlabeled images as identified by the target image classification sub-model. For example, in a face classification and recognition scenario, the first target object region refers to the face region in the first type of unlabeled images.

[0069] Specifically, the first classification reference information of the first type of unlabeled image includes the probability that the first type of unlabeled image is identified as belonging to each of the multiple preset categories. In this case, as an optional solution, the above S141 can be implemented as follows: based on the second classification reference information of the first type of unlabeled image, determine the preset category corresponding to the maximum probability from the multiple preset categories; if the probability corresponding to the preset category is greater than the preset probability threshold, then generate the first type of pseudo label of the first type of unlabeled image based on the preset category corresponding to the maximum probability.

[0070] That is, such as Figure 2 As shown, based on the classification reference information output by the second image classification sub-model for the first type of unlabeled image, a preset category with the highest probability is determined from multiple preset categories. If the probability corresponding to the preset category is greater than a preset probability threshold, a pseudo-label for the first type of unlabeled image corresponding to the first image classification sub-model is generated based on the preset category corresponding to the highest probability. Also, based on the classification reference information output by the first image classification sub-model for the first type of unlabeled image, a preset category with the highest probability is determined from multiple preset categories. If the probability corresponding to the preset category is greater than a preset probability threshold, a pseudo-label for the first type of unlabeled image corresponding to the second image classification sub-model is generated based on the preset category corresponding to the highest probability.

[0071] For example, the pseudo-label of the first class of unlabeled images corresponding to the first image classification sub-model can be determined by the following formula (1), and the pseudo-label of the first class of unlabeled images corresponding to the second image classification sub-model can be determined by the following formula (2):

[0072]

[0073]

[0074] in, This represents the ζ-th unlabeled image of class I in the image set. Corresponding to the pseudo-labels of the first image classification sub-model, ONE_HOT represents one-hot encoding, and q2 represents the second image classification sub-model. This indicates that the second image classification sub-model is for the first class of unlabeled images. Output classification reference information, This indicates the highest probability in the reference information for this category. This represents the preset category corresponding to the maximum probability, and γ represents the preset probability threshold. Represents the first type of unlabeled image The pseudo-label corresponding to the image classification sub-model is q1, which represents the first image classification sub-model. This indicates that the first image classification sub-model is for the first class of unlabeled images. Output classification reference information, arg represents the maximum probability in the classification reference information. This indicates the preset category corresponding to the highest probability.

[0075] Understandably, since the enhancement level of the first type of unlabeled image is relatively small, that is, the perturbation introduced into the initial unlabeled image is small, the first type of unlabeled image will not be distorted. In addition, when the maximum probability in the classification reference information corresponding to the first type of unlabeled image is greater than the preset probability threshold, the corresponding pseudo label is generated based on the preset category corresponding to the maximum probability. This can greatly reduce the possibility of introducing noise or errors into the pseudo label, thereby ensuring that each image classification sub-model learns the noise in the first type of unlabeled image on the basis of obtaining accurate classification and recognition results, which is conducive to improving the learning effect of each image classification sub-model.

[0076] S162, the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images are used to determine the unsupervised loss of the target image classification sub-model.

[0077] Specifically, the unsupervised loss of the first image classification sub-model can be determined based on the classification reference information output by the first image classification sub-model for the second class of unlabeled images in the image set and the pseudo-labels of the unlabeled images corresponding to the first image classification sub-model; and the unsupervised loss of the second image classification sub-model can be determined based on the classification reference information output by the second image classification sub-model for the second class of unlabeled images corresponding to the unlabeled images in the image set and the pseudo-labels of the unlabeled images corresponding to the second image classification sub-model.

[0078] More specifically, in S162 above, the unsupervised loss of the target image classification sub-model can be determined based on the first classification reference information of each second-class unlabeled image in the image set, the first-class pseudo-label of each unlabeled image, and a preset loss function. In practical applications, the preset loss function can be set according to actual needs, such as including but not limited to at least one of the following: cross-entropy loss function, classification loss function, and bounding box regression loss function.

[0079] For example, for each unlabeled image of the first class in the image set, the unsupervised sub-loss corresponding to the unlabeled image of the second class can be determined based on the first classification reference information of the unlabeled image of the second class, the first pseudo-label corresponding to the unlabeled image of the first class, and the preset loss function; further, the weighted sum of the unsupervised sub-losses corresponding to each unlabeled image of the first class in the image set is determined as the unsupervised loss of the target image classification sub-model.

[0080] Optionally, considering that the confidence of pseudo-labels generated in the early stage of training is usually not high, which can easily lead to poor training effect of image classification model, the corresponding loss weight can be set for the first class of unlabeled images based on the confidence of the first pseudo-label corresponding to the unlabeled image. For example, a higher loss weight can be given to the first class of unlabeled images with pseudo-labels with high confidence, while a lower loss weight can be given to the first class of unlabeled images with pseudo-labels with low confidence. This can help to combat the noise in the first class of pseudo-labels to a certain extent, which is beneficial to improving the training effect of image classification model.

[0081] Specifically, prior to S162 above, the training method for the image classification model provided in this application embodiment may further include: determining the loss weights corresponding to the first type of unlabeled images based on the first type of pseudo-labels and the second type of pseudo-labels corresponding to the first type of unlabeled images, wherein the second type of pseudo-labels corresponding to the first type of unlabeled images are generated based on the first classification reference information of the first type of unlabeled images, and the specific generation method is similar to that based on the first type of pseudo-labels of the first type of unlabeled images. Correspondingly, in S162 above, the unsupervised sub-loss corresponding to the first type of unlabeled images is determined based on the first classification reference information and the first type of pseudo-labels corresponding to the second type of unlabeled images, and the unsupervised loss of the target image classification sub-model is determined based on the loss weights and the unsupervised loss corresponding to the first type of unlabeled images. For example, the unsupervised sub-loss corresponding to each first type of unlabeled image in the image set can be weighted and summed based on the loss weights corresponding to each first type of unlabeled image in the image set to obtain the unsupervised loss of the target image classification sub-model.

[0082] For example, theoretically, the classification reference information obtained after the same image is predicted by different image classification sub-models should be the same. Consequently, the pseudo-labels of the same image corresponding to different image classification sub-models should also be the same. To this end, the predicted category indicated by the first pseudo-label corresponding to the first unlabeled image and the predicted category indicated by the second pseudo-label corresponding to the first unlabeled image can be compared. If the two are inconsistent, it can be determined that the confidence of the two pseudo-labels of the first unlabeled image is low, and thus the first unlabeled image can be assigned a low loss weight (i.e., the first preset weight). If the two are consistent, it can be determined that the confidence of the two pseudo-labels of the first unlabeled image is high, and thus the first unlabeled image can be assigned a high loss weight (i.e., the second preset weight).

[0083] For example, a first-class pseudo-label corresponding to a first-class unlabeled image can be used to indicate a first target object region in the first-class unlabeled image and the predicted category to which the first target object region belongs. A second-class pseudo-label corresponding to a first-class unlabeled image can be used to indicate a second target object region in the first-class unlabeled image and the predicted category to which the second target object region belongs. To ensure that the loss weights assigned to the first-class unlabeled image are more closely matched with the confidence levels of the two pseudo-labels corresponding to the first-class unlabeled image, the intersection-union ratio (IU) between the first and second target object regions can be determined, and the predicted categories to which the first and second target object regions belong can be compared to obtain the comparison results. Further, based on the IU and the comparison results, the loss weights corresponding to the first-class unlabeled image are determined.

[0084] For example, if the intersection-union ratio (IU) is less than or equal to a preset ratio, or if the comparison results show that the predicted category of the first target object region is inconsistent with the predicted category of the second target object region, then it can be determined that the confidence levels of the first and second pseudo-labels corresponding to the first type of unlabeled image are both low, and thus a first preset weight can be assigned to the first type of unlabeled image; if the IU is greater than a preset ratio, and the comparison results show that the predicted category of the first target object region is consistent with the predicted category of the second target object region, then it can be determined that the confidence levels of the first and second pseudo-labels corresponding to the first type of unlabeled image are both low, and thus a second preset weight can be assigned to the first type of unlabeled image, and the second preset weight is greater than the first preset weight.

[0085] Accordingly, in S162 above, the unsupervised loss of the target image classification sub-model can be determined by the following formula (3).

[0086]

[0087] in, N represents the unsupervised loss of the target image classification sub-model. uB represents the number of unlabeled images of the first category in the image set, where B represents the image set. Let b represent the b-th unlabeled image of type I in the image set. b∈h indicates that the confidence levels of both the type I and type II pseudo-labels corresponding to the b-th unlabeled image of type I are high, and b∈B\h indicates that the confidence levels of both the type I and type II pseudo-labels corresponding to the b-th unlabeled image of type I are low. Represents the classification loss function. This represents the bounding box regression loss function. Represents the first type of unlabeled image For the corresponding first type of pseudo-label, δ represents the loss weight corresponding to the first type of unlabeled image with a pseudo-label of high confidence.

[0088] It is understandable that the confidence level of pseudo-labels generated in the early stages of training is usually low, which can easily lead to poor training results of image classification models. Furthermore, the pseudo-labels generated after classifying and recognizing the same image through different image classification sub-models should theoretically be the same. Based on this, by determining the confidence level of the pseudo-labels based on the pseudo-labels of the first class of unlabeled images corresponding to each image classification sub-model, the corresponding loss weights can be set for the first class of unlabeled images. This can, to some extent, combat the noise in the pseudo-labels and improve the training effect of the image classification model.

[0089] S163, based on the first classification reference information of the labeled image and the category label corresponding to the labeled image, determine the supervised loss of the target image classification sub-model.

[0090] For example, the supervised loss of the target image classification sub-model can be determined by the following formula (4):

[0091]

[0092] in, N represents the supervised loss of the target image classification sub-model. l This indicates the number of labeled images in the image set. This represents the l-th labeled image in the image set. Indicates a labeled image Corresponding category tags, Represents the classification loss function. This represents the bounding box regression loss function.

[0093] S164. Based on the unsupervised and supervised losses of the target image classification sub-model, determine the classification loss of the target image classification sub-model.

[0094] For example, the classification loss of the target image classification sub-model is determined by the following formula (5):

[0095]

[0096] in, This represents the classification loss of the target image classification sub-model. This represents the supervised loss of the target image classification sub-model. λ represents the unsupervised loss of the target image classification sub-model. u This represents the loss weight corresponding to the unsupervised loss.

[0097] It is understandable that each image classification sub-model in an image classification model performs a semi-supervised learning task based on the image set. This combines supervised learning based on labeled images and their corresponding class labels with unsupervised learning based on unlabeled images and their corresponding pseudo-labels. Each learning task may incur a certain classification loss. Therefore, for each image classification sub-model, based on the classification reference information output by the sub-model for labeled images and the corresponding class labels, the supervised loss of the sub-model is determined. This ensures that the supervised loss accurately reflects the classification loss generated by the sub-model during supervised learning. Utilizing the theoretically similar pattern of classification reference information obtained by inputting the same image after different data augmentation processes into the same image classification sub-model, based on the classification reference information corresponding to the first-class unlabeled image with the lower augmentation strength, a pseudo-label corresponding to another image classification sub-model is generated for the first-class unlabeled image. Then… By utilizing the pseudo-labels of the first-class unlabeled images corresponding to each image classification sub-model and the classification reference information of the corresponding second-class unlabeled images with greater enhancement strength corresponding to each image classification sub-model, the unsupervised loss of each image classification sub-model is determined. This not only ensures that the unsupervised loss accurately reflects the classification loss generated by the corresponding image classification sub-model during the unsupervised learning task, but also helps each image classification sub-model to use the classification reference information of the corresponding first-class unlabeled images with less enhancement strength to supervise the classification reference information of the corresponding second-class unlabeled images with greater enhancement strength during the unsupervised learning process, thereby improving the classification accuracy of each image classification sub-model.

[0098] This application embodiment illustrates a specific implementation for determining the classification loss of a target image classification sub-model. It should be understood that the classification loss of the target image classification sub-model can also be determined in other ways, and this application embodiment does not limit this approach.

[0099] S108, Based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model, adjust the model parameters of the image classification model.

[0100] In one alternative implementation, such as Figure 2 As shown, the above S108 may include the following steps:

[0101] S181, the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model are weighted and summed to obtain the classification loss of the image classification model.

[0102] The classification loss of the image classification model represents the difference between the classification reference information obtained by the image classification model in classifying the input image and the true category to which the input image belongs. For example, the classification loss of the image classification model can be determined by the following formula (6):

[0103]

[0104] in, This represents the classification loss of the image classification model. This represents the classification loss of the first image classification sub-model. This represents the classification loss of the second image classification sub-model. This represents the supervised loss of the first image classification sub-model. This represents the unsupervised loss of the first image classification sub-model. This represents the supervised loss of the second image classification sub-model. λ represents the unsupervised loss of the second image classification sub-model. u This represents the loss weight corresponding to the unsupervised loss.

[0105] S182 uses the backpropagation algorithm to adjust the model parameters of the image classification model based on the classification loss of the image classification model.

[0106] The model parameters of the image classification model can include the model parameters of the first image classification sub-model and the model parameters of the second image classification sub-model. For each image classification sub-model, taking a neural network as an example, its model parameters may include, but are not limited to, the number of neurons in each network layer of the image classification sub-model, the connection relationships between neurons in different network layers and the weights of the connection edges, and the biases corresponding to the neurons in each network layer.

[0107] Since the classification loss of an image classification model can reflect the difference between the classification reference information output by the image classification model when classifying and recognizing the input image and the true category to which the input image belongs, a backpropagation algorithm can be used to adjust the model parameters of the first and second image classification sub-models based on the classification loss of the image classification model in order to obtain a high-accuracy image classification model.

[0108] More specifically, when adjusting the model parameters of the first and second image classification sub-models using the backpropagation algorithm, the prediction loss caused by each network layer of the first and second image classification sub-models can be determined based on the classification loss of the image classification model, the current model parameters of the first and second image classification sub-models, and the backpropagation algorithm. Then, with the goal of reducing the classification loss of the image classification model, the relevant parameters of each network layer in the first image classification sub-model and the relevant parameters of each network layer in the second image classification sub-model are adjusted layer by layer.

[0109] This application embodiment illustrates one specific implementation of S182 described above. Of course, it should be understood that S182 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0110] It should be noted that the above process is only a single model parameter adjustment process. In practical applications, it may be necessary to adjust the model parameters of the image classification model multiple times. Therefore, steps S102 to S108 can be repeated multiple times until the preset training stopping condition is met, thereby obtaining the final image classification model. The preset training stopping condition may be that the classification loss of the image classification model is less than a preset loss threshold, or it may be that the number of adjustments reaches a preset number, etc. This application embodiment does not limit this.

[0111] It is understandable that, since there may be some difference between the classification reference information obtained by each image classification sub-model for the input image and the true category to which the input image belongs, the classification loss generated by each image classification sub-model will affect the classification accuracy of the image classification model. Therefore, by weighted summing of the classification losses of each image classification sub-model as the classification loss of the image classification model, the classification loss of the image classification model can more accurately reflect the classification bias of the image classification model. In this way, the model parameters of the image classification model can be adjusted using the classification loss of the image classification model, which is beneficial to improving the classification accuracy of the image classification model.

[0112] The image classification model training method provided in this application embodiment, under a semi-supervised learning framework, classifies and identifies each image in the image set through various image classification sub-models in the image classification model, obtaining multiple classification reference information for each image, with each classification reference information corresponding to one image classification sub-model. Then, each image classification sub-model is used as a target image classification sub-model. Based on the classification reference information of labeled images in the image set corresponding to the target image classification sub-model, the category labels corresponding to labeled images, and the classification reference information of unlabeled images in the image set corresponding to another image classification sub-model, the classification loss of the target image classification sub-model is determined, i.e. By utilizing information learned from an image set by one target image classification sub-model, guidance can be provided to another target image classification sub-model, transforming the one-way teacher-student relationship between the target image classification sub-models into a mutual teacher-student relationship. Furthermore, by adjusting the model parameters of the image classification model based on the classification loss of each target image classification sub-model, the mutual teacher-student relationship between the target image classification sub-models can be fully utilized, enabling complementary learning and mutual improvement among them. This allows the information contained in the image set to be fully mined and utilized, thereby improving the training effect of the image classification model and obtaining a more accurate and reliable image classification model.

[0113] The above embodiments describe a training method for an image classification model. This method can be used to train image classification models for different application scenarios. The image set used for training the image classification model and the labels of each image within it can be selected according to the specific application scenario. The application scenarios applicable to the training method provided in this application embodiment include, but are not limited to, object detection, facial expression classification, natural animal classification, and handwritten digit recognition. Taking natural animal classification as an example, the category labels corresponding to labeled images are used to mark the target object contained in the labeled image and the category to which the target object belongs, such as cat, dog, horse, etc. The image classification model trained using the training method provided in this application embodiment can detect the region where the target object is located in the image to be processed and identify the category to which the target object belongs.

[0114] Based on the training method of the image classification model shown in the above embodiments of this application, the trained image classification model can be applied to any scenario that requires image classification and recognition. The application process based on the image classification model is described in detail below.

[0115] This application also provides an image classification method based on an image classification model, which can classify and recognize images to be processed based on the image classification model trained by the above training method.

[0116] Please refer to Figure 3 The following is a flowchart illustrating an image classification method according to an embodiment of this application. The method may include the following steps:

[0117] S302, the image to be processed is classified and identified using an image classification model to obtain a classification reference information set for the image to be processed.

[0118] The classification reference information set for the image to be processed includes first target classification reference information and second target classification reference information. The image classification model includes a first image classification sub-model and a second image classification sub-model. The first image classification sub-model is used to classify and recognize the image to be processed, obtaining the first target classification reference information; the second image classification sub-model is used to classify and recognize the image to be processed, obtaining the second target classification reference information.

[0119] S304, Based on the classification reference information set of the image to be processed, determine the category to which the image to be processed belongs.

[0120] Optionally, the category to which the image to be processed belongs can be determined based on the classification reference information of any image classification sub-model to which the image to be processed belongs. For example, the category with the highest probability in the first target classification reference information of the image to be processed can be determined as the category to which the image to be processed belongs, or the category with the highest probability in the second target classification reference information of the image to be processed can be determined as the category to which the image to be processed belongs.

[0121] Optionally, the category to which the image to be processed belongs can be determined by comprehensively considering the aforementioned multiple classification reference information of the image to be processed. For example, if the category corresponding to the maximum classification probability in the first target classification reference information of the image to be processed is consistent with the category corresponding to the maximum classification probability in the second target classification reference information of the image to be processed, then that category can be determined as the category to which the image to be processed belongs; or, for example, the category to which the image to be processed belongs can be determined based on the intersection between the first target category set in the first target classification reference information of the image to be processed and the second target category set in the second target classification reference information of the image to be processed, wherein the first target category set includes the categories corresponding to the probabilities exceeding a preset probability threshold in the first target classification reference information, the second target category set includes the categories corresponding to the probabilities exceeding a preset probability threshold in the second target classification reference information, and so on.

[0122] The image classification method provided in this application has high accuracy and reliability because the image classification model is trained by utilizing the mutual teacher-student relationship between various image classification sub-models on the basis of semi-supervised learning, through complementary learning and mutual teaching and learning among the various image classification sub-models. Furthermore, using the image classification model to classify and recognize the image to be processed helps to improve the accuracy and reliability of the image classification results.

[0123] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0124] In addition, with the above Figure 1 Corresponding to the training method of the image classification model shown, this application also provides a training apparatus for an image classification model. Please refer to... Figure 4 The diagram below illustrates the structure of a training device 400 for an image classification model, as provided in one embodiment of this application. The device 400 includes:

[0125] The acquisition unit 410 is used to acquire an image set for training the image classification model, the image set including labeled images, unlabeled images, and category labels corresponding to the labeled images;

[0126] The classification unit 420 is used to classify and identify the labeled image and the unlabeled image respectively through the target image classification sub-model in the image classification model, and obtain the first classification reference information of the labeled image and the first classification reference information of the unlabeled image; the target image classification sub-model is the first image classification sub-model or the second image classification sub-model.

[0127] The determining unit 430 is used to determine the classification loss of the target image classification sub-model based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image; the second classification reference information of the unlabeled image is obtained by classifying and recognizing the unlabeled image through other image classification sub-models in the image classification model besides the target image classification sub-model; the classification loss of the target image classification sub-model refers to the classification loss of the first image classification sub-model or the classification loss of the second image classification sub-model; the adjusting unit 440 is used to adjust the model parameters of the image classification model based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model.

[0128] Optionally, the acquisition unit is further configured to acquire an initial unlabeled image;

[0129] The training device 400 further includes:

[0130] An enhancement unit is used to perform data enhancement processing on the initial unlabeled image to a variety of degrees to obtain the unlabeled image. There are multiple unlabeled images, and each unlabeled image corresponds to a certain degree of enhancement.

[0131] Optionally, the unlabeled images include a first type of unlabeled images and a second type of unlabeled images, wherein the enhancement level of the first type of unlabeled images is less than the enhancement level of the second type of unlabeled images;

[0132] The determining unit determines the classification loss of the target image classification sub-model based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image, including:

[0133] Based on the second classification reference information of the first type of unlabeled images, generate the first type of pseudo-labels corresponding to the first type of unlabeled images;

[0134] Based on the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images, the unsupervised loss of the target image classification sub-model is determined;

[0135] Based on the first classification reference information of the labeled image and the category label corresponding to the labeled image, the supervised loss of the target image classification sub-model is determined;

[0136] The classification loss of the target image classification sub-model is determined based on the unsupervised loss and the supervised loss of the target image classification sub-model.

[0137] Optionally, the determining unit is further configured to determine the loss weight corresponding to the first type of unlabeled image based on the first type of pseudo-label corresponding to the first type of unlabeled image and the second type of pseudo-label corresponding to the first type of unlabeled image before determining the unsupervised loss of the target image classification sub-model based on the first classification reference information of the second type of unlabeled image and the first type of pseudo-label corresponding to the first type of unlabeled image, wherein the second type of pseudo-label corresponding to the first type of unlabeled image is generated based on the first classification reference information of the first type of unlabeled image;

[0138] The determining unit determines the unsupervised loss of the target image classification sub-model based on the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images, including:

[0139] Based on the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images, the unsupervised sub-loss corresponding to the first type of unlabeled images is determined.

[0140] Based on the loss weights corresponding to the first type of unlabeled images and the unsupervised sub-loss corresponding to the first type of unlabeled images, the unsupervised loss of the target image classification sub-model is determined.

[0141] Optionally, the first pseudo-label corresponding to the first type of unlabeled image is used to indicate the first target object region in the first type of unlabeled image and the predicted category to which the first target object region belongs; the second pseudo-label corresponding to the first type of unlabeled image is used to indicate the second target object region in the first type of unlabeled image and the predicted category to which the second target object region belongs.

[0142] The determining unit determines the loss weights corresponding to the first type of unlabeled images based on the first type of pseudo-labels and the second type of pseudo-labels corresponding to the first type of unlabeled images, including:

[0143] Determine the intersection-union ratio between the first target object region and the second target object region, and compare the predicted category to which the first target object region belongs with the predicted category to which the second target object region belongs to obtain the comparison result;

[0144] Based on the intersection-union ratio and the comparison results, the loss weights corresponding to the first type of unlabeled images are determined.

[0145] Optionally, the determining unit determines the loss weights corresponding to the first type of unlabeled images based on the intersection-union ratio and the alignment result, including:

[0146] If the intersection-union ratio is less than or equal to a preset ratio or the comparison result shows that the predicted category of the first target object region is inconsistent with the predicted category of the second target object region, then the loss weight corresponding to the first type of unlabeled image is determined to be the first preset weight.

[0147] If the intersection-union ratio is greater than the preset ratio, and the comparison result shows that the predicted category of the first target object region is consistent with the predicted category of the second target object region, then the loss weight corresponding to the first type of unlabeled image is determined to be the second preset weight, wherein the second preset weight is greater than the first preset weight.

[0148] Optionally, both the first classification reference information and the second classification reference information of the first type of unlabeled images include the probability that the first type of unlabeled images is identified as belonging to each of the multiple preset categories;

[0149] The determining unit generates a first-class pseudo-label corresponding to the first-class unlabeled image based on the second classification reference information of the first-class unlabeled image, including:

[0150] Based on the second classification reference information of the first type of unlabeled images, determine the preset category corresponding to the highest probability from the plurality of preset categories;

[0151] If the maximum probability is greater than a preset probability threshold, then a first-class pseudo-label is generated for the first-class unlabeled image based on the preset category corresponding to the maximum probability.

[0152] Optionally, the adjustment unit adjusts the model parameters of the image classification model based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model, including:

[0153] The classification loss of the image classification model is obtained by weighted summing of the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model.

[0154] The model parameters of the image classification model are adjusted based on the classification loss of the image classification model using the backpropagation algorithm.

[0155] Obviously, the training device for the image classification model provided in this application embodiment can be used as... Figure 1 The entity that performs the training method for the image classification model shown, for example, Figure 1 In the training method of the image classification model shown, step S102 can be performed by... Figure 4The acquisition unit in the training device of the image classification model shown is executed, step S104 can be executed by the classification unit in the training device of the image classification model, step S106 can be executed by the determination unit in the training device of the image classification model, and step S108 can be executed by the adjustment unit in the training device of the image classification model.

[0156] According to another embodiment of this application, Figure 4 The units in the training device of the image classification model shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the training device of the image classification model may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0157] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can run an application capable of performing tasks such as... Figure 1 The computer program (including program code) for each step involved in the corresponding method shown, to construct, as Figure 4 The image classification model training apparatus shown herein, and the image classification model training method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0158] The image classification model training device provided in this application embodiment, under a semi-supervised learning framework, classifies and identifies each image in the image set through various image classification sub-models in the image classification model, obtaining multiple classification reference information for each image, with each classification reference information corresponding to one image classification sub-model; then, each image classification sub-model is used as a target image classification sub-model, and the classification loss of the target image classification sub-model is determined based on the classification reference information of labeled images in the image set corresponding to the target image classification sub-model, the category labels corresponding to labeled images, and the classification reference information of unlabeled images in the image set corresponding to another image classification sub-model, i.e. By utilizing information learned from an image set by one target image classification sub-model, guidance can be provided to another target image classification sub-model, transforming the one-way teacher-student relationship between the target image classification sub-models into a mutual teacher-student relationship. Furthermore, by adjusting the model parameters of the image classification model based on the classification loss of each target image classification sub-model, the mutual teacher-student relationship between the target image classification sub-models can be fully utilized, enabling complementary learning and mutual improvement among them. This allows the information contained in the image set to be fully mined and utilized, thereby improving the training effect of the image classification model and obtaining a more accurate and reliable image classification model.

[0159] In addition, with the above Figure 3 Corresponding to the image classification method shown, this application also provides an image classification apparatus. Please refer to... Figure 5 The image classification device 500 provided in one embodiment of this application is shown in the following schematic diagram. The device 500 includes:

[0160] The classification unit 510 is used to classify and identify the image to be processed using an image classification model to obtain a classification reference information set of the image to be processed; wherein, the classification reference information set includes first target classification reference information and second target classification reference information, the image classification model includes a first image classification sub-model and a second image classification sub-model, the first image classification sub-model is used to classify and identify the image to be processed to obtain the first target classification reference information, the second image classification sub-model is used to classify and identify the image to be processed to obtain the second target classification reference information, and the image classification model is trained based on the training method described in the embodiments of this application;

[0161] The determining unit 520 is used to determine the category to which the image to be processed belongs based on the classification reference information set of the image to be processed.

[0162] Obviously, the image classification device provided in this application embodiment can serve as... Figure 3 The entity that performs the image classification method shown, for example, Figure 3 In the image classification method shown, step S302 can be performed by... Figure 5 The classification unit in the image classification device shown performs the step S304, which can be performed by the determination unit in the image classification device.

[0163] According to another embodiment of this application, Figure 5 The units in the image classification device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the image classification device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0164] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can run an application capable of performing tasks such as... Figure 3 The computer program (including program code) for each step involved in the corresponding method shown, to construct, as Figure 5 The image classification apparatus shown herein, and the image classification method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0165] The image classification device provided in this application has a high accuracy and reliability because the image classification model is trained by utilizing the mutual teacher-student relationship between various image classification sub-models on the basis of semi-supervised learning, through complementary learning and mutual teaching and learning among the various image classification sub-models. Furthermore, using the image classification model to classify and recognize the image to be processed helps to improve the accuracy and reliability of the image classification results.

[0166] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 6At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0167] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0168] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0169] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a training device for the image classification model at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0170] Obtain an image set for training the image classification model, the image set including labeled images, unlabeled images, and category labels corresponding to the labeled images;

[0171] The labeled image and the unlabeled image are classified and identified by the target image classification sub-model in the image classification model to obtain the first classification reference information of the labeled image and the first classification reference information of the unlabeled image; the target image classification sub-model is either the first image classification sub-model or the second image classification sub-model.

[0172] Based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image, the classification loss of the target image classification sub-model is determined; the second classification reference information of the unlabeled image is obtained by classifying and recognizing the unlabeled image through other image classification sub-models besides the target image classification sub-model in the image classification model; the classification loss of the target image classification sub-model refers to the classification loss of the first image classification sub-model or the classification loss of the second image classification sub-model;

[0173] The model parameters of the image classification model are adjusted based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model.

[0174] Alternatively, the processor reads the corresponding computer program from non-volatile memory into memory and runs it, forming an image classification device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0175] An image classification model is used to classify and identify the image to be processed, thereby obtaining a classification reference information set for the image to be processed. The classification reference information set includes first target classification reference information and second target classification reference information. The image classification model includes a first image classification sub-model and a second image classification sub-model. The first image classification sub-model is used to classify and identify the image to be processed to obtain the first target classification reference information, and the second image classification sub-model is used to classify and identify the image to be processed to obtain the second target classification reference information. The image classification model is trained using the training method described in the embodiments of this application.

[0176] Based on the classification reference information set of the image to be processed, the category to which the image to be processed belongs is determined.

[0177] The above is as stated in this application. Figure 1 The illustrated embodiment discloses a method for training an image classification model, or the method described above in this application. Figure 3The image classification device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0178] The electronic device can also perform Figure 1 The method, and the implementation of the training device for the image classification model in Figure 1 , Figure 2 The illustrated embodiment may also perform the functions of the electronic device, or the electronic device may also perform the functions of the embodiment shown. Figure 3 The method, and realize the image classification device in Figure 3 The functions of the embodiments shown are not described again in this application.

[0179] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0180] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0181] Obtain an image set for training the image classification model, the image set including labeled images, unlabeled images, and category labels corresponding to the labeled images;

[0182] The labeled image and the unlabeled image are classified and identified by the target image classification sub-model in the image classification model to obtain the first classification reference information of the labeled image and the first classification reference information of the unlabeled image; the target image classification sub-model is either the first image classification sub-model or the second image classification sub-model.

[0183] Based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image, the classification loss of the target image classification sub-model is determined; the second classification reference information of the unlabeled image is obtained by classifying and recognizing the unlabeled image through other image classification sub-models besides the target image classification sub-model in the image classification model; the classification loss of the target image classification sub-model refers to the classification loss of the first image classification sub-model or the classification loss of the second image classification sub-model;

[0184] The model parameters of the image classification model are adjusted based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model.

[0185] Alternatively, the computer-readable storage medium stores one or more programs, which include instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 3 The method of the illustrated embodiment is specifically used to perform the following operations:

[0186] An image classification model is used to classify and identify the image to be processed, thereby obtaining a classification reference information set for the image to be processed. The classification reference information set includes first target classification reference information and second target classification reference information. The image classification model includes a first image classification sub-model and a second image classification sub-model. The first image classification sub-model is used to classify and identify the image to be processed to obtain the first target classification reference information, and the second image classification sub-model is used to classify and identify the image to be processed to obtain the second target classification reference information. The image classification model is trained using the training method described in the embodiments of this application.

[0187] Based on the classification reference information set of the image to be processed, the category to which the image to be processed belongs is determined.

[0188] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0189] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0190] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0191] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0192] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A training method for an image classification model, characterized in that, The image classification model includes a first image classification sub-model and a second image classification sub-model, and the method includes: Obtain an image set for training the image classification model, the image set including labeled images, unlabeled images, and category labels corresponding to the labeled images; The labeled image and the unlabeled image are classified and identified by the target image classification sub-model in the image classification model to obtain the first classification reference information of the labeled image and the first classification reference information of the unlabeled image; the target image classification sub-model is either the first image classification sub-model or the second image classification sub-model. Based on the first classification reference information of the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image, the classification loss of the target image classification sub-model is determined; the second classification reference information of the unlabeled image is obtained by classifying and recognizing the unlabeled image through other image classification sub-models besides the target image classification sub-model in the image classification model; the classification loss of the target image classification sub-model refers to the classification loss of the first image classification sub-model or the classification loss of the second image classification sub-model; Based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model, the model parameters of the image classification model are adjusted. The unlabeled images include a first type of unlabeled images and a second type of unlabeled images. The classification loss of the target image classification sub-model includes an unsupervised loss, which is obtained based on the loss weight corresponding to the first type of unlabeled images and the unsupervised sub-loss corresponding to the first type of unlabeled images. The first type of unlabeled image has a first type of pseudo-label and a second type of pseudo-label. The first type of pseudo-label is obtained based on the second classification reference information of the first type of unlabeled image, and the second type of pseudo-label is obtained based on the first classification reference information of the first type of unlabeled image. The first type of pseudo-label is used to indicate the first target object region in the first type of unlabeled image and the predicted category to which the first target object region belongs. The second type of pseudo-label is used to indicate the second target object region in the first type of unlabeled image and the predicted category to which the second target object region belongs. The unsupervised sub-loss is obtained based on the first classification reference information and the first pseudo-label of the second type of unlabeled image; the loss weight is obtained in the following way: The intersection-union ratio (IUR) between the first target object region and the second target object region is determined, and the predicted category to which the first target object region belongs is compared with the predicted category to which the second target object region belongs to obtain the comparison result; based on the IUR and the comparison result, the loss weight corresponding to the first type of unlabeled image is determined.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the initial unlabeled image; The initial unlabeled image is subjected to data augmentation processing of various degrees to obtain the unlabeled image. There are multiple unlabeled images, and each unlabeled image corresponds to a certain degree of augmentation.

3. The method according to claim 2, characterized in that, The enhancement level of the first type of unlabeled image is less than that of the second type of unlabeled image; determining the classification loss of the target image classification sub-model based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image includes: Based on the second classification reference information of the first type of unlabeled images, generate the first type of pseudo-labels corresponding to the first type of unlabeled images; Based on the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images, the unsupervised loss of the target image classification sub-model is determined; Based on the first classification reference information of the labeled image and the category label corresponding to the labeled image, the supervised loss of the target image classification sub-model is determined; The classification loss of the target image classification sub-model is determined based on the unsupervised loss and the supervised loss of the target image classification sub-model.

4. The method according to claim 3, characterized in that, Before determining the unsupervised loss of the target image classification sub-model based on the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images, the method further includes: Based on the first pseudo-label corresponding to the first type of unlabeled image and the second pseudo-label corresponding to the first type of unlabeled image, the loss weight corresponding to the first type of unlabeled image is determined. The unsupervised loss of the target image classification sub-model, determined based on the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images, includes: Based on the first classification reference information of the second type of unlabeled images and the first type of pseudo-labels corresponding to the first type of unlabeled images, the unsupervised sub-loss corresponding to the first type of unlabeled images is determined. Based on the loss weights corresponding to the first type of unlabeled images and the unsupervised sub-loss corresponding to the first type of unlabeled images, the unsupervised loss of the target image classification sub-model is determined.

5. The method according to claim 4, characterized in that, The step of determining the loss weights corresponding to the first type of unlabeled images based on the intersection-union ratio and the alignment results includes: If the intersection-union ratio is less than or equal to a preset ratio or the comparison result shows that the predicted category of the first target object region is inconsistent with the predicted category of the second target object region, then the loss weight corresponding to the first type of unlabeled image is determined to be the first preset weight. If the intersection-union ratio is greater than the preset ratio, and the comparison result shows that the predicted category of the first target object region is consistent with the predicted category of the second target object region, then the loss weight corresponding to the first type of unlabeled image is determined to be the second preset weight, wherein the second preset weight is greater than the first preset weight.

6. The method according to claim 3, characterized in that, The first classification reference information and the second classification reference information of the first type of unlabeled images both include the probability that the first type of unlabeled images are identified as belonging to each of the multiple preset categories; Based on the second classification reference information of the first type of unlabeled images, a first type of pseudo-label is generated corresponding to the first type of unlabeled images, including: Based on the second classification reference information of the first type of unlabeled images, determine the preset category corresponding to the highest probability from the plurality of preset categories; If the maximum probability is greater than a preset probability threshold, then a first-class pseudo-label is generated for the first-class unlabeled image based on the preset category corresponding to the maximum probability.

7. The method according to any one of claims 1 to 6, characterized in that, The step of adjusting the model parameters of the image classification model based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model includes: The classification loss of the image classification model is obtained by weighted summing of the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model. The model parameters of the image classification model are adjusted based on the classification loss of the image classification model using the backpropagation algorithm.

8. An image classification method, characterized in that, include: An image classification model is used to classify and identify the image to be processed, thereby obtaining a classification reference information set for the image to be processed. The classification reference information set includes first target classification reference information and second target classification reference information. The image classification model includes a first image classification sub-model and a second image classification sub-model. The first image classification sub-model is used to classify and identify the image to be processed to obtain the first target classification reference information, and the second image classification sub-model is used to classify and identify the image to be processed to obtain the second target classification reference information. The image classification model is trained based on the training method described in any one of claims 1 to 7. Based on the classification reference information set of the image to be processed, the category to which the image to be processed belongs is determined.

9. A training device for an image classification model, characterized in that, The image classification model includes a first image classification sub-model and a second image classification sub-model, and the training device includes: The acquisition unit is used to acquire an image set for training the image classification model, the image set including labeled images, unlabeled images, and category labels corresponding to the labeled images; The classification unit is used to classify and identify the labeled image and the unlabeled image respectively through the target image classification sub-model in the image classification model, and obtain the first classification reference information of the labeled image and the first classification reference information of the unlabeled image; the target image classification sub-model is the first image classification sub-model or the second image classification sub-model. A determining unit is configured to determine the classification loss of the target image classification sub-model based on the first classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, and the second classification reference information of the unlabeled image; the second classification reference information of the unlabeled image is obtained by classifying and recognizing the unlabeled image through other image classification sub-models besides the target image classification sub-model in the image classification model; the classification loss of the target image classification sub-model refers to the classification loss of the first image classification sub-model or the classification loss of the second image classification sub-model; an adjusting unit is configured to adjust the model parameters of the image classification model based on the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model. The unlabeled images include a first type of unlabeled images and a second type of unlabeled images. The classification loss of the target image classification sub-model includes an unsupervised loss, which is obtained based on the loss weight corresponding to the first type of unlabeled images and the unsupervised sub-loss corresponding to the first type of unlabeled images. The first type of unlabeled image has a first type of pseudo-label and a second type of pseudo-label. The first type of pseudo-label is obtained based on the second classification reference information of the first type of unlabeled image, and the second type of pseudo-label is obtained based on the first classification reference information of the first type of unlabeled image. The first type of pseudo-label is used to indicate the first target object region in the first type of unlabeled image and the predicted category to which the first target object region belongs. The second type of pseudo-label is used to indicate the second target object region in the first type of unlabeled image and the predicted category to which the second target object region belongs. The unsupervised sub-loss is obtained based on the first classification reference information and the first pseudo-label of the second type of unlabeled image; the loss weight is obtained in the following way: The intersection-union ratio (IUR) between the first target object region and the second target object region is determined, and the predicted category to which the first target object region belongs is compared with the predicted category to which the second target object region belongs to obtain the comparison result; based on the IUR and the comparison result, the loss weight corresponding to the first type of unlabeled image is determined.

10. An image classification device, characterized in that, include: A classification unit is used to classify and identify an image to be processed using an image classification model to obtain a classification reference information set for the image to be processed; wherein, the classification reference information set includes first target classification reference information and second target classification reference information, the image classification model includes a first image classification sub-model and a second image classification sub-model, the first image classification sub-model is used to classify and identify the image to be processed to obtain the first target classification reference information, the second image classification sub-model is used to classify and identify the image to be processed to obtain the second target classification reference information, and the image classification model is trained based on the training method of any one of claims 1 to 7; The determining unit is used to determine the category to which the image to be processed belongs based on the classification reference information set of the image to be processed.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 7; or the processor is configured to execute the instructions to implement the method as described in claim 8.

12. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1 to 7; or, the processor is configured to execute the instructions to implement the method as described in claim 8.

Images