Label generation, training of image classification model, image classification method and device

Abstract

This application discloses a method for label generation and an image classification model, as well as an image classification method and apparatus. The label generation method includes: classifying and recognizing unlabeled images using a teacher model and a student model respectively to obtain classification reference information corresponding to the teacher model and the student model; the classification reference information corresponding to each model includes N detection objects contained in the unlabeled image and the probability of each detection object belonging to each of the multiple categories; determining the number of detection objects of each category in the teacher model based on the classification reference information corresponding to the teacher model, and determining the number of detection objects of each category in the student model based on the classification reference information corresponding to the student model; determining the probability threshold of each category based on the number of detection objects of each category in the teacher model and the student model; and generating pseudo-labels for the unlabeled images corresponding to the student model based on the classification reference information corresponding to the teacher model and the probability thresholds of each category.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a label generation, image classification model training, image classification method and apparatus. Background Technology

[0002] In recent years, deep learning techniques based on convolutional neural networks have achieved remarkable results in image classification, object detection, and instance segmentation. However, large-scale neural networks rely on large amounts of labeled data. In practical applications, constructing large-scale labeled datasets suitable for supervised learning is difficult, time-consuming, and dependent on expert experience. However, obtaining unlabeled image data is easy; large amounts of unlabeled images can be quickly acquired using automated computer programs. By utilizing a small number of labeled images and a large number of unlabeled images, deep learning tasks can be effectively performed, and high-performance deep learning models can be trained, effectively addressing the problem of insufficient labeled image quantity. This approach is known as semi-supervised learning. To save on the cost of manual annotation, semi-supervised learning has been proposed and extensively studied.

[0003] In object detection tasks, class imbalance is common, with significant differences in the learning difficulty of each class. This problem is more pronounced in semi-supervised object detection tasks, where confirmation bias and error accumulation are more significant. Specifically, in semi-supervised object detection, labeled images provide limited supervisory information, causing the model to tend to identify more easily identifiable objects. This results in generated false labels containing more easily identifiable objects and fewer difficult ones, further exacerbating the imbalance problem. This phenomenon hinders the utilization of unlabeled images (low utilization of difficult categories), leading to poor training performance and ultimately affecting the model's classification accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus and method for label generation and image classification model, which solves the problem of poor model training effect caused by low utilization of difficult categories in pseudo-labels during the training process of existing image classification models.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a label generation method, which is applied in a semi-supervised object detection architecture, the semi-supervised object detection architecture including a teacher model and a student model, and the label generation method includes:

[0007] The teacher model is used to classify and identify the unlabeled image, thereby obtaining classification reference information corresponding to the teacher model. Similarly, the student model is used to classify and identify the unlabeled image, thereby obtaining classification reference information corresponding to the student model. Both the classification reference information corresponding to the teacher model and the classification reference information corresponding to the student model include N detection objects contained in the identified unlabeled image and the probability that each of the N detection objects belongs to each of the multiple categories; N is a positive integer.

[0008] Based on the classification reference information corresponding to the teacher model, the number of detection objects for each category in the multiple categories corresponding to the teacher model is determined; and based on the classification reference information corresponding to the student model, the number of detection objects for each category in the multiple categories corresponding to the student model is determined.

[0009] Based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, the probability threshold for each category is determined.

[0010] Based on the classification reference information corresponding to the teacher model and the probability threshold of each category, pseudo-labels are generated for the student model corresponding to the unlabeled image.

[0011] As can be seen, in this embodiment of the application, for a semi-supervised object detection framework including a teacher model and a student model, based on the classification reference information obtained by the teacher model and the student model respectively classifying and recognizing the same unlabeled image, the number of detection objects corresponding to the same category for each model is counted. This allows for the evaluation of the learning status (or learning difficulty) of the teacher model and the student model for the same category, thereby determining the probability threshold suitable for each category. Furthermore, by using the classification reference information corresponding to the teacher model and the probability threshold for each category, pseudo-labels corresponding to the unlabeled image are generated for the student model. This avoids the large-scale filtering of detection objects in difficult categories, alleviates the problem of learning imbalance of the student model for different categories, and thus improves the training effect of the image classification model with a semi-supervised object detection framework, enabling the image classification model to have a higher classification accuracy.

[0012] Secondly, embodiments of this application provide a training method for an image classification model. The training method is used to train an image classification model with a semi-supervised object detection architecture. The image classification model includes a teacher model and a student model. The training method includes:

[0013] Obtain an image set for training the image classification model, the image set including labeled images and unlabeled images, and obtain the category labels corresponding to the labeled images and the pseudo labels corresponding to the unlabeled images, wherein the pseudo labels corresponding to the unlabeled images are generated based on the label generation method described in the first aspect;

[0014] Using the student model, the labeled image and the unlabeled image are classified and identified respectively, and the target classification reference information corresponding to the labeled image and the target classification reference information corresponding to the unlabeled image are obtained.

[0015] Based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image, the model parameters of the image classification model are adjusted.

[0016] As can be seen, in this embodiment, for an image classification model with a semi-supervised object detection framework, the student model in the image classification model classifies and identifies labeled and unlabeled images in the image set respectively, obtaining classification reference information corresponding to labeled images and classification reference information corresponding to unlabeled images. Then, the pseudo-labels corresponding to unlabeled images generated by the label generation method provided in this embodiment guide the learning process of the student model. Based on the classification reference information corresponding to each image in the image set, the category labels corresponding to labeled images in the image set, and the pseudo-labels corresponding to unlabeled images, the model parameters of the image classification model are adjusted, which helps to alleviate the problem of imbalance in the student model's learning of different categories, thereby improving the training effect of the image classification model with a semi-supervised object detection framework and making the image classification model have a high classification accuracy.

[0017] Thirdly, embodiments of this application provide an image classification method, including:

[0018] The image to be processed is classified and identified by the target model in the image classification model, and the classification reference information corresponding to the target model is obtained.

[0019] Based on the classification reference information corresponding to the target model, the detection objects contained in the image to be processed and the category to which the detection objects belong are determined;

[0020] The image classification model is a model with a semi-supervised object detection framework, the image classification model includes a teacher model and a student model, and the object model includes the teacher model and / or the student model;

[0021] The image classification model is trained based on the target classification reference information of each image in the image set, the category labels corresponding to labeled images in the image set, and the pseudo labels corresponding to unlabeled images; the target classification reference information of each image is obtained by classifying and recognizing each image in the image set through the student model in the image classification model; the pseudo labels corresponding to unlabeled images are determined by the first method.

[0022] As can be seen, in the embodiments of this application, for the image classification model with a semi-supervised object detection framework, since the pseudo-labels corresponding to the unlabeled images used in the semi-supervised learning process of the image classification model are obtained by the dynamic evaluation of the learning status (or learning difficulty) of each category by the teacher model and the student model respectively, it is beneficial to alleviate the problem of the student model's learning imbalance for different categories, thereby obtaining a higher classification accuracy of the trained image classification model; furthermore, using the image classification model to classify and recognize the image to be processed helps to improve the accuracy and reliability of the image classification results.

[0023] Fourthly, embodiments of this application provide a label generation device, which is applied in a semi-supervised object detection architecture, the semi-supervised object detection architecture including a teacher model and a student model, and the label generation device includes:

[0024] The classification unit is used to classify and recognize unlabeled images using the teacher model to obtain classification reference information corresponding to the teacher model, and to classify and recognize unlabeled images using the student model to obtain classification reference information corresponding to the student model. Both the classification reference information corresponding to the teacher model and the classification reference information corresponding to the student model include N detection objects contained in the identified unlabeled image and the probability that each of the N detection objects belongs to each of the multiple categories; N is a positive integer.

[0025] The determining unit is used to determine the number of detection objects for each category in the multiple categories corresponding to the teacher model based on the classification reference information corresponding to the teacher model, and to determine the number of detection objects for each category in the multiple categories corresponding to the student model based on the classification reference information corresponding to the student model.

[0026] The determining unit is further configured to determine the probability threshold for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model.

[0027] The generation unit is used to generate pseudo-labels for the unlabeled images of the student model based on the classification reference information corresponding to the teacher model and the probability threshold of each category.

[0028] Fifthly, embodiments of this application provide a training apparatus for an image classification model. The training apparatus is used to train an image classification model with a semi-supervised object detection architecture. The image classification model includes a teacher model and a student model. The training apparatus includes:

[0029] The acquisition unit is configured to acquire an image set for training the image classification model, the image set including labeled images and unlabeled images, and to acquire the category labels corresponding to the labeled images and the pseudo labels corresponding to the unlabeled images, wherein the pseudo labels corresponding to the unlabeled images are generated based on the label generation method described in the first aspect.

[0030] The classification unit is used to classify and identify the labeled image and the unlabeled image respectively using the student model, and obtain the target classification reference information corresponding to the labeled image and the target classification reference information corresponding to the unlabeled image;

[0031] The adjustment unit is used to adjust the model parameters of the image classification model based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image.

[0032] Sixthly, embodiments of this application provide an image classification apparatus, comprising:

[0033] The classification unit is used to classify and identify the image to be processed using the target model in the image classification model, and to obtain the classification reference information corresponding to the target model.

[0034] The determining unit is used to determine the detection objects contained in the image to be processed and the category to which the detection objects belong, based on the classification reference information corresponding to the target model;

[0035] The image classification model is a model with a semi-supervised object detection framework, the image classification model includes a teacher model and a student model, and the object model includes the teacher model and / or the student model;

[0036] The image classification model is trained based on the target classification reference information of each image in the image set, the category labels corresponding to labeled images in the image set, and the pseudo labels corresponding to unlabeled images; the target classification reference information of each image is obtained by classifying and recognizing each image in the image set through the student model in the image classification model; the pseudo labels corresponding to unlabeled images are determined by the method described in the first aspect.

[0037] In a seventh aspect, embodiments of this application provide an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as described in any one of the first, second, and third aspects.

[0038] Eighthly, embodiments of this application provide a computer-readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method as described in any one of the first, second, and third aspects. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 The flowchart illustrates a tag generation method provided in one embodiment of this application;

[0041] Figure 2 A flowchart illustrating a label generation method provided for another embodiment of this application;

[0042] Figure 3 A flowchart illustrating a training method for an image classification model provided in one embodiment of this application;

[0043] Figure 4 A schematic diagram illustrating a training method for an image classification model provided in another embodiment of this application;

[0044] Figure 5 A flowchart illustrating an image classification method provided in one embodiment of this application;

[0045] Figure 6 A schematic diagram of a label generation apparatus provided in one embodiment of this application;

[0046] Figure 7 A schematic diagram of the structure of a training device for an image classification model provided in one embodiment of this application;

[0047] Figure 8 A schematic diagram of the structure of an image classification device provided in one embodiment of this application;

[0048] Figure 9 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] The terms "first," "second," etc., used in this specification and claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this specification and claims, "and / or" indicates at least one of the objects in the two sections, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0051] As mentioned earlier, class imbalance is common in object detection tasks, with significant differences in the learning difficulty of each class. This problem is even more pronounced in semi-supervised object detection tasks, where confirmation bias and error accumulation are more significant. In other words, in semi-supervised object detection tasks, the supervisory information provided by labeled images is scarce, leading the model to tend to identify more easily identifiable objects. This results in generated pseudo-labels containing more easily identifiable objects and fewer difficult objects, further exacerbating the imbalance problem. This phenomenon hinders the utilization of unlabeled images (low utilization of difficult categories), leading to poor training performance and ultimately affecting the model's classification accuracy.

[0052] Research on existing semi-supervised object detection methods reveals that the main reason for the generated pseudo-labels containing more easy-to-class objects and fewer difficult-to-class objects is that these methods do not differentiate between object categories based on their learning difficulty. Instead, they use a fixed threshold to select appropriate categories from the model's classification results for unlabeled images and generate pseudo-labels accordingly. This results in a large number of difficult-to-class objects being filtered out, causing the model to tend to identify more easily identifiable objects.

[0053] Based on this, this application proposes a label generation method for a semi-supervised object detection framework including a teacher model and a student model. Based on the classification reference information obtained by the teacher and student models from classifying and recognizing the same unlabeled image, the number of detection objects corresponding to the same category for each model is counted. This allows for the evaluation of the learning status (or learning difficulty) of the teacher and student models for the same category, thereby determining the probability threshold suitable for each category. Furthermore, using the classification reference information corresponding to the teacher model and the probability threshold for each category, pseudo-labels are generated for the unlabeled image corresponding to the student model. This avoids the large-scale filtering of detection objects in difficult categories, alleviates the problem of learning imbalance between different categories by the student model, and thus improves the training effect of the image classification model with the semi-supervised object detection framework, resulting in a higher classification accuracy.

[0054] This application also proposes a training method for an image classification model. For an image classification model with a semi-supervised object detection framework, a student model in the image classification model classifies and identifies labeled and unlabeled images in the image set, respectively, obtaining classification reference information for labeled and unlabeled images. Then, the pseudo-labels for the unlabeled images generated by the label generation method provided in this application guide the learning process of the student model. Based on the classification reference information for each image in the image set, the category labels for labeled images, and the pseudo-labels for unlabeled images, the model parameters of the image classification model are adjusted. This helps alleviate the problem of imbalance in the student model's learning of different categories, thereby improving the training effect of the image classification model with a semi-supervised object detection framework and enabling the image classification model to have a higher classification accuracy.

[0055] This application also proposes an image classification method. The trained image classification model can accurately classify and recognize images. Since the pseudo-labels corresponding to the unlabeled images used in the semi-supervised learning process of the image classification model are dynamically evaluated by the teacher model and the student model for each category's learning status (or learning difficulty), this helps alleviate the problem of learning imbalance between different categories by the student model. Consequently, the trained image classification model has a high classification accuracy. Furthermore, using the image classification model to classify and recognize the images to be processed helps improve the accuracy and reliability of the image classification results.

[0056] It should be understood that the label generation method, image classification model training method, and image classification method provided in the embodiments of this application can all be executed by electronic devices or software installed in electronic devices. The term "electronic device" here may include terminal devices, such as smartphones, tablets, laptops, desktop computers, smart voice interaction devices, smart home appliances, smartwatches, vehicle terminals, aircraft, etc.; or, the electronic device may also include a server, such as a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0057] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0058] Please refer to Figure 1 The image below is a flowchart illustrating a label generation method according to an embodiment of this application. This method is applied to a semi-supervised object detection architecture, which includes a teacher model and a student model. The method may include the following steps:

[0059] S102, classify and recognize unlabeled images using the teacher model to obtain classification reference information corresponding to the teacher model, and classify and recognize unlabeled images using the student model to obtain classification reference information corresponding to the student model.

[0060] In this embodiment, the teacher model and the student model can have the same network structure, or, in order to simplify the model structure and achieve compression and acceleration of the image classification model, the teacher model and the student model can have different network structures. For example, the student model adopts a more streamlined structure compared to the teacher model.

[0061] In addition, in practical applications, the teacher model and the student model can be two independent models; or, the teacher model and the student model can be integrated into an image classification model. Based on this, the image classification model can be trained using a semi-supervised learning method to obtain the final image classification model.

[0062] Both the teacher and student models can classify and recognize unlabeled images, obtaining corresponding classification reference information. That is, one set of classification reference information corresponds to one model in the semi-supervised object detection architecture. Specifically, the teacher model can classify and recognize unlabeled images, obtaining classification reference information for the teacher model corresponding to the unlabeled image; the student model can classify and recognize unlabeled images, obtaining classification reference information for the student model corresponding to the unlabeled image.

[0063] The unlabeled image corresponds to the classification reference information for the teacher model, and the unlabeled image corresponds to the classification reference information for the student model. Both include the N (N is a positive integer) detection objects contained in the identified unlabeled image and the probability of each of the N detection objects belonging to each of the multiple categories. Here, a detection object refers to a portion of the content presented in the unlabeled image, and the probability of each detection object belonging to a certain category represents the probability that a portion of the content presented in the unlabeled image belongs to that category. For example, the multiple categories include cat, dog, horse, and mule. The classification reference information corresponding to the unlabeled image includes the N detection objects contained in the unlabeled image and the probability of each of these N detection objects belonging to cat, dog, horse, and mule, respectively.

[0064] To ensure that each model in the semi-supervised object detection framework can fully understand and learn from unlabeled images, thereby improving the expressive power of each model, such as... Figure 2 As shown, the unlabeled image in this application embodiment can be obtained after data augmentation processing of the initial unlabeled image. That is, before S102 above, the label generation method provided in this application embodiment may further include: performing data augmentation processing on the initial unlabeled image to add perturbation to the initial unlabeled image and obtain an unlabeled image containing perturbation.

[0065] Specifically, for an initial unlabeled image, data augmentation processing of various degrees can be performed on the initial unlabeled image to obtain multiple unlabeled images, where each unlabeled image corresponds to one of the multiple degrees of augmentation.

[0066] More specifically, the initial unlabeled image undergoes data augmentation processing at various levels of enhancement. This can be implemented as follows: weakly-augmented processing is applied to the initial unlabeled image to obtain a first-type unlabeled image, and strongly-augmented processing is applied to the initial unlabeled image to obtain a second-type unlabeled image. That is, the aforementioned multiple unlabeled images include both the first-type and second-type unlabeled images, with the enhancement level of the first-type unlabeled image being less than that of the second-type unlabeled image. For example, weak enhancement processing may include, but is not limited to, at least one of the following processing methods: translation, flipping, etc., while strong enhancement processing may include, but is not limited to, at least one of the following processing methods: occlusion, color transformation, random erase, etc.

[0067] Accordingly, in S102 above, the first type of unlabeled image can be classified and identified by the teacher model to obtain the classification reference information of the first type of unlabeled image corresponding to the teacher model, and the second type of unlabeled image can be classified and identified by the student model to obtain the classification reference information of the second type of unlabeled image corresponding to the student model.

[0068] Understandably, since the enhancement levels of translation, flipping, and other enhancement processes are relatively small, meaning the perturbation introduced into the initial unlabeled image is small, it is unlikely to distort the first type of unlabeled image. This allows both the teacher and student models to learn the noise in the first type of unlabeled image based on accurate classification results, which is beneficial to improving the learning performance of both models. Furthermore, considering that using only weakly enhanced images may cause the teacher and / or student models to fall into overfitting and fail to extract the essential features of the first type of unlabeled image, while strong enhancement processes introduce larger perturbations that may distort the second type of unlabeled image, but still retain enough features to identify the category, by performing weak and strong enhancement processes on the initial unlabeled image respectively, and then having the teacher model classify the first type of unlabeled image and the student model classify the second type of unlabeled image, it is beneficial to improve the learning performance of both models and enhance their expressive capabilities.

[0069] S104. Based on the classification reference information corresponding to the teacher model, determine the number of detection objects for each category in the teacher model, and based on the classification reference information corresponding to the student model, determine the number of detection objects for each category in the student model.

[0070] In one optional implementation, S104 may include: for the teacher model, based on the classification reference information corresponding to the teacher model, selecting the category with the highest probability from the probability of each detected object belonging to each category, determining the predicted category of the teacher model for each object, and determining the number of detected objects in the teacher model for each category among multiple categories based on the predicted category of the teacher model for each detected object; for the student model, based on the classification reference information corresponding to the student model, selecting the category with the highest probability from the probability of each detected object belonging to each category, determining the predicted category of the student model for each object, and determining the number of detected objects in the student model for each category among multiple categories based on the predicted category of the student model for each detected object.

[0071] For example, taking multiple categories including cats, dogs, horses, and mules, for the classification reference information corresponding to each model, if the probabilities of a detected object belonging to cats, dogs, horses, and mules are 0.1, 0.2, 0.6, and 0.1 respectively, then it can be determined that the predicted category of the detected object corresponding to the model is horse. By counting the predicted categories of each detected object, the number of detected objects under each category is determined. For example, if the predicted categories of the detected objects are horse, horse, dog, cat, and dog, then the count shows that the number of detected objects under the category of horse is 2, the number of detected objects under the category of dog is 2, and the number of detected objects under the category of cat is 1.

[0072] In practical applications, the identified unlabeled image contains at least one detected object. After determining the predicted category of a certain model for each detected object, the number of detected objects corresponding to each category in that model can be determined by counting the detected objects belonging to that category. For example, the number of detected objects corresponding to each category in a certain model can be determined by the following formula (1):

[0073]

[0074] in, This represents the number of objects detected by the i-th model in the semi-supervised object detection architecture for the k-th category. This represents an unlabeled image. This represents the set of detected objects contained in the unlabeled image detected by the i-th model. This represents the j-th detection object o in the set of detection objects. j , It is an exponential function, which means that if the condition in parentheses is true, it equals 1, and |·| represents the base.

[0075] Furthermore, for ease of subsequent use, such as Figure 2 As shown, for each model in the semi-supervised object detection architecture, the number of detected objects for each category in that model can be recorded in the form of a dictionary. The key of the dictionary is the category identifier (e.g., category number), and the value represents the number of detected objects for that category in that model. This yields the dictionary corresponding to that model. For example, the dictionary corresponding to the i-th model is shown below:

[0076]

[0077] in, Let represent the dictionary corresponding to the i-th model in the semi-supervised object detection architecture, and k represent the category identifier of the k-th category, 1≤k≤z. This represents the number of detected objects corresponding to the k-th category in the i-th model. This indicates an unlabeled image.

[0078] It is worth noting that the above is only illustrated with an unlabeled image; the dictionary in question... This dictionary only corresponds to one unlabeled image for the i-th model. In actual training, multiple unlabeled images are used. In this case, we can statistically analyze the detection objects belonging to the same category in each unlabeled image to obtain the detection object set corresponding to each category for that model. This allows us to obtain the dictionary corresponding to the unlabeled image set for that model, i.e., Where represents the dictionary corresponding to the set of unlabeled images for the i-th model. Represents a set of unlabeled images. This represents the j-th unlabeled image in the set of unlabeled images. This represents the dictionary that corresponds to the i-th model and the j-th unlabeled image.

[0079] This application embodiment illustrates one specific implementation of S104 described above. It should be understood that S104 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0080] S106. Based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, determine the probability threshold for each category.

[0081] like Figure 2 As shown, considering that the recognition results of the same image after being classified and recognized by different models should theoretically be the same, the number of detected objects corresponding to the same category in different models should also be the same. Therefore, the difference in the number of detected objects for each category in different models can intuitively and accurately reflect the difference in classification and recognition of each category by different models. This allows us to evaluate the learning status (or learning difficulty) of the student model in learning from the teacher model for each category. Based on the learning status (or learning difficulty) of each category, we can set a corresponding threshold probability for each category. Based on the probability threshold corresponding to each category and the classification reference information obtained by the teacher model for unlabeled images, we can generate pseudo-labels for unlabeled images for the student model. This can avoid the large-scale filtering of detected objects in difficult categories, which is beneficial to the problem of imbalance in the learning of different categories by the student model. This is conducive to improving the training effect of image classification models with semi-supervised object detection architecture, so that the image classification model has a higher classification accuracy.

[0082] Based on this, in one alternative implementation, the above 106 may include the following steps:

[0083] S161. Based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, determine the learning rate corresponding to the target category.

[0084] For each category, the learning rate refers to the learning state of the student model learning from the teacher model for that category. A higher learning rate for that category indicates a better learning state of the student model, making that category easier to identify. Conversely, a lower learning rate for that category indicates a better learning state of the student model learning from the teacher model, making that category harder to identify.

[0085] Optionally, for each category, the learning rate can be determined as the ratio between the number of detection objects corresponding to that category in the student model and the number of detection objects corresponding to that category in the teacher model.

[0086] In other words, the learning rate for each category can be determined using the following formula (2):

[0087]

[0088] in, This represents the learning rate corresponding to the i-th category. This represents the number of objects detected in the teacher model corresponding to the i-th category. Let z represent the number of detected objects corresponding to the i-th category in the student model, and z represent the total number of categories.

[0089] Optionally, in order to ensure that the learning rate for each category can accurately reflect the learning status of the student model learning from the teacher model, the learning rate for each category can be determined based on the ratio between the proportions of the number of detection objects in each category.

[0090] For example, the learning rate for each category can be determined as follows: For each category, a first proportion of detected objects for that category is determined based on the ratio between the number of detected objects for that category in the teacher model and the first total number of detected objects, where the first total number of detected objects is the sum of the number of detected objects for each category in the teacher model; a second proportion of detected objects for that category is determined based on the ratio between the number of detected objects for that category in the student model and the second total number of detected objects, where the second total number of detected objects is the sum of the number of detected objects for each category in the student model; further, the learning rate for that category is determined based on the ratio between the first proportion of detected objects and the second proportion of detected objects.

[0091] In other words, the learning rate for each category can be determined using the following formulas (3) to (5):

[0092]

[0093]

[0094]

[0095] in, This represents the learning rate corresponding to the i-th category. This represents the proportion of the i-th category in the detection objects corresponding to the teacher model. This represents the proportion of the i-th category in the detection objects corresponding to the student model. This indicates the number of objects detected by the teacher model corresponding to the i-th category. This indicates that the g-th category corresponds to the number of objects detected by the teacher model. This represents the total number of detection objects corresponding to the teacher model. This indicates the number of objects detected by the student model corresponding to the i-th category. This indicates that the g-th category corresponds to the detection data of the student model. This represents the total number of objects detected for the student model, and z represents the total number of categories.

[0096] Optionally, considering that the classification reference information usually includes detected objects identified as background categories, while in practical applications, only the classification and identification of detected objects that are not background categories are usually considered, based on this, before S106 above, the label generation method provided in this application embodiment may further include: removing categories belonging to the image background from multiple categories. Then, the learning rate corresponding to each category is determined by any of the above methods.

[0097] For example, the learning rate for each category can be determined using the following formulas (6) to (8):

[0098]

[0099]

[0100]

[0101] in, This represents the learning rate corresponding to the i-th category. This represents the proportion of the i-th category in the detection objects corresponding to the teacher model. This represents the proportion of the i-th category in the detection objects corresponding to the student model. This indicates that the g-th category corresponds to the number of objects detected by the teacher model. The number of detected objects corresponding to the teacher model is represented by h, and the image background is represented by h. This indicates the number of detected objects in the teacher model corresponding to the category that belongs to the image background. This indicates the number of objects detected by the student model corresponding to the i-th category. This indicates that the g-th category corresponds to the detection data of the student model. This represents the total number of detection objects corresponding to the student model. This represents the number of detected objects in the student model that belong to the category of image background, and z represents the total number of categories.

[0102] S162, Based on the learning rate corresponding to the target category, determine the probability threshold corresponding to the target category.

[0103] Optionally, based on the learning rate corresponding to each category, a preset correspondence between the learning rate and the probability threshold can be queried to obtain the corresponding probability threshold, which can then be used as the probability threshold for each category. This preset correspondence can be set according to actual needs, and this embodiment does not limit it.

[0104] Optionally, to avoid the pseudo-labels generated based on probability thresholds filtering out more detection objects belonging to categories that are not easy to identify, thus causing an imbalance in the student model's learning of different categories, the above S162 can be specifically implemented as follows: determine the maximum learning rate among the learning rates corresponding to the above multiple categories, and determine the weight coefficient corresponding to each category based on the ratio between the learning rate corresponding to each category and the maximum learning rate; further, determine the probability threshold corresponding to each category based on the product between the weight coefficient corresponding to each category and the preset probability upper limit value.

[0105] It is worth noting that in the actual training process of image classification models for semi-supervised object detection architecture, as the model parameters are continuously adjusted, the classification reference information obtained by the image classification model after classifying and recognizing the same unlabeled image in each iteration (i.e., each adjustment) is different. Consequently, the learning rates corresponding to the multiple categories obtained in each iteration are different. Therefore, in each iteration, based on the classification reference information obtained in that iteration, the learning rates corresponding to the multiple categories in that iteration and the maximum learning rate can be determined. This allows the probability thresholds corresponding to each category in that iteration to be obtained, so as to generate pseudo-labels for the unlabeled images in that iteration.

[0106] For example, the probability threshold for each category can be determined using the following formulas (9) and (10):

[0107]

[0108]

[0109] in, This represents the weight coefficient corresponding to the k-th category in the e-th iteration. This represents the learning rate for the k-th category during the e-th iteration. This represents the maximum learning rate among the learning rates corresponding to multiple classes during the e-th iteration. z represents the total number of categories, and T is a hyperparameter used to adjust the weighting coefficients. Let τ represent the probability threshold corresponding to the k-th category in the e-th iteration, and τ represent the preset upper probability value. In practical applications, the preset probability threshold can be set according to actual needs, and this embodiment does not limit it.

[0110] Understandable, The smaller the value, the fewer objects belonging to the k-th category are identified by the student model compared to the teacher model. Therefore, objects belonging to this category are more likely to be poorly understood, and many objects belonging to this category may even be ignored during pseudo-label generation. Determining the probability threshold in the above way allows for the automatic assignment of a smaller probability threshold to this category, thus enabling the generated pseudo-labels to help the student model extract more usable unlabeled images; conversely, a smaller probability threshold indicates that the student model is less likely to be identified by the teacher model compared to the teacher model. A larger value indicates that the student model identifies more objects belonging to the k-th category than the teacher model. Therefore, objects belonging to this category are easily classified and identified by the student model in its current learning state, potentially leading to confirmation bias. Determining the probability threshold using the above method allows for the automatic allocation of a larger probability threshold to this category. This results in pseudo-labels that help the student model selectively extract high-quality unlabeled images. Thus, by determining the probability threshold, the generated pseudo-labels help alleviate the learning imbalance between different categories in the student model, thereby improving the training effect of the image classification model with a semi-supervised object detection architecture and achieving higher classification accuracy.

[0111] This application embodiment illustrates one specific implementation of S106 described above. It should be understood that S106 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0112] S108: Based on the classification reference information corresponding to the teacher model and the probability threshold of each category, generate pseudo-labels for the unlabeled images of the student model.

[0113] Since unlabeled images do not have corresponding category labels, pseudo-labels are generated for the student model based on the classification reference information obtained by the teacher model for unlabeled images and the probability thresholds corresponding to each category. This is equivalent to adding artificial labels to the unlabeled images, thereby providing guidance for the student model's unsupervised learning task.

[0114] In practical applications, the pseudo-labels corresponding to unlabeled images can be used to indicate the predicted category to which the unlabeled image belongs. Of course, the pseudo-labels can also be used to indicate the detected objects in the unlabeled image and the predicted category to which those objects belong. For example, in a face classification and recognition scenario, the detected object can be a face in an unlabeled image, and the predicted category to which the detected object belongs is the predicted category to which the face belongs.

[0115] In one optional implementation, S108 may include: based on the classification reference information corresponding to the teacher model, selecting the category with the highest probability from the probability of each detected object belonging to each category, and determining the predicted category of each object corresponding to the teacher model; if the probability of each object corresponding to the predicted category of the teacher model is greater than the probability threshold of the predicted category, then generating pseudo-labels for the unlabeled images of the student model based on the predicted category of each object corresponding to the teacher model.

[0116] For example, pseudo-labels for unlabeled images can be generated for student models using the following formula (11):

[0117]

[0118] in, Generate unlabeled images for student models The corresponding pseudo-labels are: ONE_HOT indicates one-hot encoding, and q1 indicates the teacher model. The teacher model is for unlabeled images. Output classification reference information, This indicates the highest probability in the reference information for this category. This indicates that the detected object corresponds to the predicted category of the teacher model. This represents the probability threshold corresponding to the predicted category during the e-th iteration.

[0119] Understandably, by generating pseudo-labels for the unlabeled images of the student models only when the probability of each detected object corresponding to the predicted category of the teacher model is greater than the probability threshold of that predicted category, the possibility of introducing noise or errors into the pseudo-labels can be greatly reduced. This ensures that the student models learn the noise in the unlabeled images based on accurate classification and recognition results, which is beneficial to improving the learning effect of the student models.

[0120] This application embodiment illustrates one specific implementation of the above-described S108. It should be understood that S108 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0121] The label generation method provided in this application targets a semi-supervised object detection framework including a teacher model and a student model. Based on the classification reference information obtained by the teacher model and the student model from classifying and recognizing the same unlabeled image, the method counts the number of detection objects corresponding to the same category for each model. This allows the method to evaluate the learning status (or learning difficulty) of the teacher model and the student model for the same category, thereby determining the probability threshold suitable for each category. Furthermore, by using the classification reference information corresponding to the teacher model and the probability threshold for each category, pseudo-labels are generated for the unlabeled image corresponding to the student model. This avoids the large-scale filtering of detection objects in difficult categories, alleviates the problem of learning imbalance between different categories by the student model, and improves the training effect of the image classification model with the semi-supervised object detection framework, resulting in a higher classification accuracy for the image classification model.

[0122] Based on the label generation method shown in the above embodiments of this application, the pseudo-labels generated for unlabeled images can be applied to image classification models with semi-supervised object detection architectures for semi-supervised learning, resulting in image classification models that can accurately classify and recognize the images to be processed. The training process of the image classification model is described in detail below.

[0123] Please refer to Figure 3 The image above is a flowchart illustrating a training method for an image classification model, provided as an embodiment of this application. This method trains an image classification model with a semi-supervised object detection architecture, comprising a teacher model and a student model. The method may include the following steps:

[0124] S302, obtain the image set used to train the image classification model, and obtain the category labels corresponding to the labeled images and the pseudo labels corresponding to the unlabeled images in the image set.

[0125] The image set includes labeled images and unlabeled images, and the pseudo-labels corresponding to the unlabeled images are generated based on the label generation method described in the embodiments of this application.

[0126] S304. Using the student model in the image classification model, labeled and unlabeled images are classified and identified respectively, and target classification reference information corresponding to labeled images and classification reference information corresponding to unlabeled images are obtained.

[0127] Specifically, the student model can perform object detection on each image in the image set, identify candidate bounding boxes of the detected objects, and then use non-maximum suppression (NMS) to eliminate redundant candidate bounding boxes to determine the detected objects contained in each image. Then, based on the image features of the detected objects contained in each image, the model can classify and identify the detected objects contained in each image to obtain classification reference information of the detected objects contained in each image.

[0128] S306. Based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image, adjust the model parameters of the image classification model.

[0129] The model parameters of an image classification model can include the model parameters of the teacher model and the model parameters of the student model. For each model, taking a neural network as an example, its model parameters may include, but are not limited to, the number of neurons in each network layer, the connection relationships between neurons in different network layers, the weights of the connection edges, and the biases of the neurons in each network layer.

[0130] Specifically, such as Figure 4 As shown, by having the teacher model generate unlabeled images corresponding to the student model, the student model can perform semi-supervised learning tasks under the guidance of the teacher network, based on the classification reference information of each image in the image set, the category label corresponding to each labeled image in the image set, and the pseudo-label corresponding to each unlabeled image. That is, it includes supervised learning tasks based on labeled images and their corresponding category labels, as well as unsupervised learning tasks based on unlabeled images.

[0131] Considering that each learning task may produce a certain classification loss, in order to improve the training effect of the image classification model, in one optional implementation, the above S306 may include the following steps:

[0132] S341, based on the category labels corresponding to the labeled images and the target classification reference information corresponding to the labeled images, determine the supervised loss of the student model.

[0133] The supervised loss of the student model is used to represent the classification loss generated by the student model when performing supervised learning tasks.

[0134] Specifically, the supervised loss of the student model can be determined based on the category label corresponding to the labeled image, the target classification reference information corresponding to the labeled image, and a first preset loss function. In practical applications, the first preset loss function can be set according to actual needs, such as including but not limited to at least one of the following: cross-entropy loss function, classification loss function, and bounding box regression loss function.

[0135] For example, the supervised loss of the student model can be determined by the following formula (12):

[0136]

[0137] N represents the supervised loss of the student model. l This indicates the number of labeled images in the image set. This represents the l-th labeled image in the image set. Indicates a labeled image Corresponding category tags, Represents the classification loss function. This represents the bounding box regression loss function.

[0138] S342. Based on the pseudo-labels corresponding to the unlabeled images and the target classification reference information corresponding to the unlabeled images, determine the unsupervised loss of the student model.

[0139] The unsupervised loss of the student model is used to represent the classification loss generated by the student model when performing unsupervised learning tasks.

[0140] Specifically, the unsupervised loss of the student model can be determined based on the pseudo-labels corresponding to the unlabeled images, the target classification reference information corresponding to the unlabeled images, and the second preset loss function. In practical applications, the second preset loss function can be set according to actual needs, such as including but not limited to at least one of the following: cross-entropy loss function, classification loss function, and bounding box regression loss function.

[0141] For example, the supervised loss of the student model can be determined by the following formula (13):

[0142]

[0143] in, N represents the unsupervised loss of the student model. u This indicates the number of unlabeled images in the image set. This represents the b-th unlabeled image in the image set. Represents unlabeled images The corresponding pseudo-tags.

[0144] S343, Based on the supervised loss and unsupervised loss of the student model, determine the classification loss of the student model.

[0145] For example, the classification loss of the student model is determined as follows: Formula (14)

[0146]

[0147] in, This represents the classification loss of the student model. This represents the supervised loss of the student model. λ represents the unsupervised loss of the student model. u This represents the loss weight corresponding to the unsupervised loss.

[0148] Understandably, the student model performs semi-supervised learning tasks based on image sets, combining supervised learning based on labeled images and their corresponding class labels with unsupervised learning based on unlabeled images and their corresponding pseudo-labels. Each learning task may incur a certain classification loss. Therefore, based on the target classification reference information output by the student model for labeled images and the corresponding class labels, the supervised loss of the student model is determined, ensuring that the supervised loss accurately reflects the classification loss incurred by the student model during supervised learning tasks. Similarly, using the pseudo-labels corresponding to unlabeled images and the target classification reference information for unlabeled images, the unsupervised loss of the student model is determined, ensuring that the unsupervised loss accurately reflects the classification loss incurred by the student model during unsupervised learning tasks.

[0149] S344, adjust the model parameters of the student model based on the classification loss of the student model.

[0150] Specifically, since the classification loss of the student model can reflect the difference between the target classification reference information output by the student model when classifying and recognizing the input image and the corresponding category label of the input image, in order to obtain a student model with high accuracy, the backpropagation algorithm can be used to adjust the network parameters of the student model based on the classification loss of the student model.

[0151] More specifically, when adjusting the model parameters of the student model using the backpropagation algorithm, the classification loss caused by each network layer of the student model can be determined based on the classification loss of the student model and the current model parameters. Then, with the goal of reducing the classification loss of the student model, the relevant parameters of each network layer in the student model are adjusted layer by layer.

[0152] This application embodiment illustrates one specific implementation of S344 described above. It should be understood that S344 can also be implemented in other ways, and this application embodiment does not impose any limitations on this.

[0153] It should be noted that the above process is only a single model parameter adjustment process. In practical applications, it may be necessary to adjust the model parameters of the image classification model multiple times. Therefore, steps S302 to S304 can be repeated multiple times until the preset training stopping condition is met, thereby obtaining the final student model. The preset training stopping condition may be that the classification loss of the student model is less than a preset loss threshold, or it may be that the number of adjustments reaches a preset number, etc. This application embodiment does not limit this.

[0154] Furthermore, the aforementioned S304 may also include:

[0155] S345, Based on the student model's model parameters and the exponential moving average adjustment strategy, adjust the teacher model's model parameters.

[0156] For example, the model parameters of the teacher model may include the classification weights of the teacher model, and the model parameters of the student model may include the classification weights of the student model. The classification weights of each model are used to weight the classification reference information obtained by each model when classifying and recognizing the image to be processed using the image classification model, in order to obtain the final classification and recognition result. In this case, by adjusting the sum of the classification weights of the student models, the classification weights of the teacher model can be adjusted to the exponential moving average (EMA) of the classification weights of the student models.

[0157] This application embodiment illustrates one specific implementation of S304 described above. Of course, it should be understood that S304 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0158] The image classification model training method provided in this application embodiment targets an image classification model with a semi-supervised object detection framework. Through a student model within the image classification model, labeled and unlabeled images in an image set are classified and identified respectively, obtaining classification reference information for labeled and unlabeled images. Then, the pseudo-labels for the unlabeled images generated by the label generation method provided in this application embodiment guide the student model's learning process. Based on the classification reference information for each image in the image set, the category labels for labeled images, and the pseudo-labels for unlabeled images, the model parameters of the image classification model are adjusted. This helps alleviate the problem of imbalance in the student model's learning of different categories, thereby improving the training effect of the image classification model with a semi-supervised object detection framework and enabling the image classification model to achieve higher classification accuracy.

[0159] The above embodiments describe a training method for an image classification model. This method can be used to train image classification models for different application scenarios. The image set used to train the model and the labels of each image within it can be selected according to the specific application scenario. The application scenarios applicable to the training method provided in this application include, but are not limited to, object detection, facial expression classification, animal classification, and handwritten digit recognition. Taking animal classification as an example, the category labels corresponding to labeled images are used to mark the detection objects contained in the labeled images and the categories to which the detection objects belong, such as cats, dogs, and horses. The image classification model trained using the method provided in this application can detect the region where the detection object is located in the image to be processed and identify the category to which the detection object belongs.

[0160] Based on the training method of the image classification model shown in the above embodiments of this application, the trained image classification model can be applied to any scenario that requires image classification and recognition. The application process based on the image classification model is described in detail below.

[0161] This application also provides an image classification method based on an image classification model, which can classify and recognize images to be processed based on the image classification model trained by the above-described training method. Please refer to... Figure 5 The following is a flowchart illustrating an image classification method according to an embodiment of this application. The method may include the following steps:

[0162] S502, through the target model in the image classification model, classifies and identifies the image to be processed, and obtains the classification reference information corresponding to the target model.

[0163] The image classification model is a semi-supervised detection framework model, which includes a teacher model and a student model. The target model includes a teacher model and / or a student model. The teacher model is used to classify and recognize the image to be processed, obtaining classification reference information of the image to be processed corresponding to the teacher model; the student model is used to classify and recognize the image to be processed, obtaining classification reference information of the image to be processed corresponding to the student model.

[0164] S504, Based on the classification reference information corresponding to the target model, determine the detection objects contained in the image to be processed and the category to which the detection objects belong.

[0165] Optionally, the detected objects and their categories in the image to be processed can be determined based on classification reference information corresponding to either the teacher model or the student model. For example, the category to which the detected objects in the image to be processed belong can be determined by the category with the highest probability in the classification reference information of the teacher model; or, the category to which the detected objects in the image to be processed belong can be determined by the category with the highest probability in the classification reference information of the student model.

[0166] Optionally, the detection objects contained in the image to be processed and the categories to which those detection objects belong can be determined by comprehensively considering the aforementioned multiple classification reference information of the image to be processed. For example, if the category corresponding to the maximum classification probability in the classification reference information of the student model (hereinafter referred to as the first classification reference information for ease of description) of the image to be processed is consistent with the category corresponding to the maximum classification probability in the classification reference information of the teacher model (hereinafter referred to as the second classification reference information for ease of description), then that category can be determined as the category to which the detection objects contained in the image to be processed belong; or, for example, the category to which the detection objects contained in the image to be processed belong can be determined based on the intersection between the first target category set in the first classification reference information of the image to be processed and the second target category set in the second classification reference information of the image to be processed, wherein the first target category set includes the categories corresponding to the probabilities exceeding a preset probability threshold in the first classification reference information, the second target category set includes the categories corresponding to the probabilities exceeding a preset probability threshold in the second classification reference information, and so on.

[0167] The image classification method provided in this application, for image classification models with a semi-supervised object detection framework, is beneficial in alleviating the problem of imbalance in the learning of different categories by the student model. This is because the pseudo-labels corresponding to the unlabeled images used in the semi-supervised learning process of the image classification model are dynamically evaluated by the teacher model and the student model for each category, respectively. As a result, the trained image classification model has a high classification accuracy. Furthermore, using the image classification model to classify and recognize the images to be processed helps to improve the accuracy and reliability of the image classification results.

[0168] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0169] In addition, with the above Figure 1 Corresponding to the label generation method shown, this application also provides a label generation apparatus. Please refer to... Figure 6 This is a schematic diagram of a label generation device 600 provided in one embodiment of this application. The label generation device is applied in a semi-supervised object detection architecture, which includes a teacher model and a student model. The device 600 includes:

[0170] The classification unit 610 is used to classify and identify unlabeled images using the teacher model to obtain classification reference information corresponding to the teacher model, and to classify and identify unlabeled images using the student model to obtain classification reference information corresponding to the student model; both the classification reference information corresponding to the teacher model and the classification reference information corresponding to the student model include N detection objects contained in the identified unlabeled image and the probability that each of the N detection objects belongs to each of the multiple categories; N is a positive integer;

[0171] The determining unit 620 is used to determine the number of detection objects for each category in the multiple categories corresponding to the teacher model based on the classification reference information corresponding to the teacher model, and to determine the number of detection objects for each category in the multiple categories corresponding to the student model based on the classification reference information corresponding to the student model.

[0172] The determining unit 620 is further configured to determine a probability threshold for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model.

[0173] The generation unit 630 is used to generate pseudo-labels for the unlabeled images of the student model based on the classification reference information corresponding to the teacher model and the probability threshold of each category.

[0174] Optionally, when determining the probability threshold for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, the determining unit 620 performs the following steps:

[0175] The learning rate for each category is determined based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model.

[0176] Based on the learning rate for each category, a probability threshold for each category is determined.

[0177] Optionally, when determining the learning rate for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, the determining unit 620 performs the following steps:

[0178] Based on the ratio between the number of detection objects corresponding to each category in the teacher model and the first total number of detection objects, the first proportion of detection objects for each category is determined, where the first total number of detection objects is the sum of the number of detection objects corresponding to each category in the teacher model.

[0179] The second proportion of detection objects for each category is determined based on the ratio between the number of detection objects for each category in the student model and the second total number of detection objects. The second total number of detection objects is the sum of the number of detection objects for each category in the student model.

[0180] The learning rate for each category is determined based on the ratio between the proportion of the first detected object in each category and the proportion of the second detected object in each category.

[0181] Optionally, when determining the probability threshold for each category based on the learning rate of each category, the determining unit 620 performs the following steps:

[0182] Determine the maximum learning rate among the learning rates corresponding to the multiple categories;

[0183] The weight coefficient for each category is determined based on the ratio between the learning rate of each category and the maximum learning rate.

[0184] The probability threshold for each category is determined by multiplying the weight coefficient corresponding to each category with the preset probability upper limit.

[0185] Optionally, the device 600 further includes:

[0186] The elimination unit 640 is used to eliminate categories belonging to the image background from the plurality of categories before the determining unit determines the probability threshold of each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model.

[0187] Optionally, when determining the number of detection objects for each of the multiple categories in the teacher model based on the classification reference information corresponding to the teacher model, and when determining the number of detection objects for each of the multiple categories in the student model based on the classification reference information corresponding to the student model, the determining unit 620 performs the following steps:

[0188] Based on the classification reference information corresponding to the teacher model, the category with the highest probability is selected from the probability of each detected object belonging to each category, and the predicted category of the teacher model is determined for each detected object. Based on the predicted category of the teacher model for each detected object, the number of detected objects corresponding to each category in the teacher model is determined.

[0189] Based on the classification reference information corresponding to the student model, the category with the highest probability is selected from the probabilities of each detected object belonging to each category, and this category is determined as the predicted category of the student model corresponding to each detected object. Based on the predicted category of the student model corresponding to each detected object, the number of detected objects corresponding to each category in the student model is determined.

[0190] Optionally, when generating pseudo-labels for the unlabeled images of the student model based on the classification reference information corresponding to the teacher model and the probability threshold of each category, the generation unit 630 performs the following steps:

[0191] If the probability of each detected object corresponding to the predicted category of the teacher model is greater than the probability threshold of the predicted category, then a pseudo-label corresponding to the unlabeled image is generated for the student model based on the predicted category of each detected object corresponding to the teacher model.

[0192] Obviously, the label generation device provided in this application embodiment can serve as... Figure 1 The execution body of the label generation method shown, for example, Figure 1 In the label generation method shown, step S102 can be generated by... Figure 6 The classification unit in the label generation device shown is executed, steps S104 and S106 can be executed by the determination unit in the label generation device, and step S108 can be executed by the generation unit in the label generation device.

[0193] According to another embodiment of this application, Figure 6 The various units in the label generation device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the label generation device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0194] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can run an application capable of performing tasks such as... Figure 1 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 6 The label generation apparatus shown herein, and the label generation method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0195] The label generation apparatus provided in this application, for a semi-supervised object detection framework including a teacher model and a student model, is based on the classification reference information obtained by the teacher model and the student model from classifying and recognizing the same unlabeled image. It counts the number of detection objects corresponding to the same category for each model, and can evaluate the learning status (or learning difficulty) of the teacher model and the student model for the same category, thereby determining the probability threshold suitable for each category. Furthermore, using the classification reference information corresponding to the teacher model and the probability threshold for each category, pseudo-labels are generated for the unlabeled image corresponding to the student model. This avoids the large-scale filtering of detection objects in difficult categories, alleviates the problem of learning imbalance between different categories by the student model, and thus improves the training effect of the image classification model with the semi-supervised object detection framework, enabling the image classification model to have a higher classification accuracy.

[0196] In addition, with the above Figure 3 Corresponding to the training method of the image classification model shown, this application also provides a training apparatus for an image classification model. Please refer to... Figure 7 The diagram below illustrates the structure of a training device 700 for an image classification model, provided in one embodiment of this application. The device 700 includes:

[0197] The acquisition unit 710 is used to acquire an image set for training the image classification model, the image set including labeled images and unlabeled images, and to acquire the category labels corresponding to the labeled images and the pseudo labels corresponding to the unlabeled images, wherein the pseudo labels corresponding to the unlabeled images are generated based on the label generation method described in the first aspect.

[0198] The classification unit 720 is used to classify and identify the labeled image and the unlabeled image respectively using the student model, and obtain the target classification reference information corresponding to the labeled image and the target classification reference information corresponding to the unlabeled image.

[0199] The adjustment unit 730 is used to adjust the model parameters of the image classification model based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image.

[0200] Optionally, the model parameters of the image classification model include the model parameters of the student model;

[0201] When adjusting the model parameters of the image classification model based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image, the adjustment unit 730 performs the following steps:

[0202] Based on the category label corresponding to the labeled image and the target classification reference information corresponding to the labeled image, the supervised loss of the student model is determined;

[0203] Based on the pseudo-labels corresponding to the unlabeled images and the target classification reference information corresponding to the unlabeled images, the unsupervised loss of the student model is determined.

[0204] Based on the supervised loss and the unsupervised loss, determine the classification loss of the student model;

[0205] The model parameters of the student model are adjusted based on the classification loss of the student model.

[0206] Optionally, the model parameters of the image classification model may also include the model parameters of the student model;

[0207] The adjustment unit 730 is further configured to: adjust the model parameters of the teacher model based on the model parameters of the student model and the exponential moving average adjustment strategy.

[0208] Obviously, the training device for the image classification model provided in this application embodiment can be used as... Figure 3 The entity that performs the training method for the image classification model shown, for example, Figure 3 In the training method of the image classification model shown, step S302 can be performed by... Figure 7The acquisition unit 710 in the training device of the image classification model shown is executed, step S304 can be executed by the classification unit 720 in the training device of the image classification model, and step S306 can be executed by the adjustment unit 730 in the training device of the image classification model.

[0209] According to another embodiment of this application, Figure 7 The units in the training device of the image classification model shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the training device of the image classification model may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0210] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit, random access storage medium, and read-only storage medium, can be used to run an application capable of performing tasks such as... Figure 3 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 7 The image classification model training apparatus shown herein, and the image classification model training method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0211] The image classification model training device provided in this application provides a training device for an image classification model with a semi-supervised object detection framework. Through a student model in the image classification model, labeled and unlabeled images in an image set are classified and identified respectively, obtaining classification reference information corresponding to labeled images and unlabeled images. Then, pseudo-labels corresponding to unlabeled images generated by the label generation method provided in this application guide the learning process of the student model. Based on the classification reference information corresponding to each image in the image set, the category labels corresponding to labeled images in the image set, and the pseudo-labels corresponding to unlabeled images, the model parameters of the image classification model are adjusted. This helps alleviate the problem of imbalance in the student model's learning of different categories, thereby improving the training effect of the image classification model with a semi-supervised object detection framework and enabling the image classification model to have a high classification accuracy.

[0212] In addition, with the above Figure 5 Corresponding to the image classification method shown, this application also provides an image classification apparatus. Please refer to... Figure 8 The image classification device 800 provided in one embodiment of this application is shown in the following schematic diagram. The device 800 includes:

[0213] The classification unit 810 is used to classify and identify the image to be processed through the target model in the image classification model, and obtain the classification reference information corresponding to the target model.

[0214] The determining unit 820 is used to determine the detection objects contained in the image to be processed and the category to which the detection objects belong, based on the classification reference information corresponding to the target model.

[0215] The image classification model is a model with a semi-supervised object detection framework, the image classification model includes a teacher model and a student model, and the object model includes the teacher model and / or the student model;

[0216] The image classification model is trained based on target classification reference information of labeled images in the image set, target classification reference information of unlabeled images, category labels corresponding to the labeled images, and pseudo labels corresponding to the unlabeled images. The target classification reference information of the labeled images or the target classification information of the unlabeled images is obtained by classifying and recognizing the labeled images or unlabeled images through the student model in the image classification model. The pseudo labels corresponding to the unlabeled images are determined by the label generation method described in the embodiments of this application.

[0217] Obviously, the image classification device provided in this application embodiment can serve as... Figure 5 The entity that performs the image classification method shown, for example, Figure 5 In the image classification method shown, step S502 can be performed by... Figure 8 The classification unit 810 in the image classification device shown performs the step S504, which can be performed by the determination unit 820 in the image classification device.

[0218] According to another embodiment of this application, Figure 8The units in the image classification device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the image classification device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0219] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit, random access storage medium, and read-only storage medium, can be used to run an application capable of performing tasks such as... Figure 5 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 8 The image classification apparatus shown herein, and the image classification method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0220] The image classification apparatus provided in this application, for an image classification model with a semi-supervised object detection framework, uses pseudo-labels corresponding to unlabeled images in the semi-supervised learning process of the image classification model. These pseudo-labels are obtained by dynamically evaluating the learning status (or learning difficulty) of each category by the teacher model and the student model, respectively. This helps to alleviate the problem of learning imbalance of the student model for different categories, thereby resulting in a trained image classification model with high classification accuracy. Furthermore, using the image classification model to classify and recognize the image to be processed helps to improve the accuracy and reliability of the image classification results.

[0221] Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 9 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0222] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0223] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0224] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a tag generation device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0225] The unlabeled image is classified and identified using a teacher model in a semi-supervised object detection framework to obtain classification reference information corresponding to the teacher model. Similarly, the unlabeled image is classified and identified using a student model in the same framework to obtain classification reference information corresponding to the student model. Both the classification reference information for the teacher model and the student model include N detection objects contained in the identified unlabeled image and the probability that each of the N detection objects belongs to each of the multiple categories; N is a positive integer.

[0226] Based on the classification reference information corresponding to the teacher model, the number of detection objects for each category in the multiple categories corresponding to the teacher model is determined; and based on the classification reference information corresponding to the student model, the number of detection objects for each category in the multiple categories corresponding to the student model is determined.

[0227] Based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, the probability threshold for each category is determined.

[0228] Based on the classification reference information corresponding to the teacher model and the probability threshold of each category, pseudo-labels are generated for the student model corresponding to the unlabeled image.

[0229] Alternatively, the processor reads the corresponding computer program from non-volatile memory into memory and runs it, forming a training device for the image classification model at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0230] A set of images for training the image classification model is obtained, the set of images including labeled images and unlabeled images, and category labels corresponding to the labeled images and pseudo labels corresponding to the unlabeled images are obtained. The pseudo labels corresponding to the unlabeled images are generated based on the label generation method described in the embodiments of this application. The image classification model includes a teacher model and a student model.

[0231] Using the student model, the labeled image and the unlabeled image are classified and identified respectively, and the target classification reference information corresponding to the labeled image and the target classification reference information corresponding to the unlabeled image are obtained.

[0232] Based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image, the model parameters of the image classification model are adjusted.

[0233] Alternatively, the processor reads the corresponding computer program from non-volatile memory into memory and runs it, forming an image classification device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0234] The image to be processed is classified and identified by the target model in the image classification model, and the classification reference information corresponding to the target model is obtained.

[0235] Based on the classification reference information corresponding to the target model, the detection objects contained in the image to be processed and the category to which the detection objects belong are determined;

[0236] The image classification model is a model with a semi-supervised object detection framework, the image classification model includes a teacher model and a student model, and the object model includes the teacher model and / or the student model;

[0237] The image classification model is trained based on target classification reference information of labeled images, target classification reference information of unlabeled images, category labels corresponding to labeled images, and pseudo labels corresponding to unlabeled images. The target classification reference information of labeled images or the target classification information of unlabeled images is obtained by classifying and recognizing labeled or unlabeled images through a student model in the image classification model. The pseudo labels corresponding to unlabeled images are determined by the label generation method in the embodiments of this application.

[0238] The above is as stated in this application. Figure 1 The method performed by the label generation apparatus disclosed in the illustrated embodiment is as described above in this application. Figure 3 The illustrated embodiment discloses a method for training an image classification model, as well as the method described above in this application. Figure 5 At least one of the methods performed by the image classification apparatus disclosed in the illustrated embodiments can be applied to a processor, or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0239] The electronic device can also perform Figure 1 The method, and implement the label generation device in Figure 1 , Figure 2 The illustrated embodiment may also perform the functions of the electronic device, or the electronic device may also perform the functions of the embodiment shown. Figure 3 The method, and the implementation of the training device for the image classification model in Figure 3 , Figure 4 The illustrated embodiment may also perform the functions of the electronic device, or the electronic device may also perform the functions of the embodiment shown. Figure 5 The method, and realize the image classification device in Figure 5 The functions of the embodiments shown are not described in detail here.

[0240] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0241] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0242] The unlabeled image is classified and identified using a teacher model in a semi-supervised object detection framework to obtain classification reference information corresponding to the teacher model. Similarly, the unlabeled image is classified and identified using a student model in the same framework to obtain classification reference information corresponding to the student model. Both the classification reference information for the teacher model and the student model include N detection objects contained in the identified unlabeled image and the probability that each of the N detection objects belongs to each of the multiple categories; N is a positive integer.

[0243] Based on the classification reference information corresponding to the teacher model, the number of detection objects for each category in the multiple categories corresponding to the teacher model is determined; and based on the classification reference information corresponding to the student model, the number of detection objects for each category in the multiple categories corresponding to the student model is determined.

[0244] Based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, the probability threshold for each category is determined.

[0245] Based on the classification reference information corresponding to the teacher model and the probability threshold of each category, pseudo-labels are generated for the student model corresponding to the unlabeled image.

[0246] Alternatively, when executed by a portable electronic device that includes multiple applications, the instruction can enable the portable electronic device to perform... Figure 3 The method of the illustrated embodiment is specifically used to perform the following operations:

[0247] A set of images for training the image classification model is obtained, the set of images including labeled images and unlabeled images, and category labels corresponding to the labeled images and pseudo labels corresponding to the unlabeled images are obtained. The pseudo labels corresponding to the unlabeled images are generated based on the label generation method described in the embodiments of this application. The image classification model includes a teacher model and a student model.

[0248] Using the student model, the labeled image and the unlabeled image are classified and identified respectively, and the target classification reference information corresponding to the labeled image and the target classification reference information corresponding to the unlabeled image are obtained.

[0249] Based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image, the model parameters of the image classification model are adjusted.

[0250] Alternatively, when executed by a portable electronic device that includes multiple applications, the instruction can enable the portable electronic device to perform... Figure 5 The method of the illustrated embodiment is specifically used to perform the following operations:

[0251] The image to be processed is classified and identified by the target model in the image classification model, and the classification reference information corresponding to the target model is obtained.

[0252] Based on the classification reference information corresponding to the target model, the detection objects contained in the image to be processed and the category to which the detection objects belong are determined;

[0253] The image classification model is a model with a semi-supervised object detection framework, the image classification model includes a teacher model and a student model, and the object model includes the teacher model and / or the student model;

[0254] The image classification model is trained based on target classification reference information of labeled images in the image set, target classification reference information of unlabeled images, category labels corresponding to the labeled images, and pseudo labels corresponding to the unlabeled images. The target classification reference information of the labeled images or the target classification information of the unlabeled images is obtained by classifying and recognizing the labeled images or unlabeled images through the student model in the image classification model. The pseudo labels corresponding to the unlabeled images are determined by the label generation method described in the embodiments of this application.

[0255] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0256] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0257] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0258] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0259] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A label generation method, characterized in that, The label generation method is applied to a semi-supervised object detection architecture, which includes a teacher model and a student model. The label generation method includes: The teacher model is used to classify and identify unlabeled images, thereby obtaining classification reference information corresponding to the teacher model. Similarly, the student model is used to classify and identify unlabeled images, thereby obtaining classification reference information corresponding to the student model. Both the classification reference information corresponding to the teacher model and the classification reference information corresponding to the student model include N detection objects contained in the identified unlabeled image and the probability that each of the N detection objects belongs to each of the multiple categories; N is a positive integer. Based on the classification reference information corresponding to the teacher model, the number of detection objects for each category in the multiple categories corresponding to the teacher model is determined; and based on the classification reference information corresponding to the student model, the number of detection objects for each category in the multiple categories corresponding to the student model is determined. Based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, the probability threshold for each category is determined. Based on the classification reference information corresponding to the teacher model and the probability threshold of each category, pseudo-labels are generated for the student model corresponding to the unlabeled image.

2. The method according to claim 1, characterized in that, The determination of the probability threshold for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model includes: The learning rate for each category is determined based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model. Based on the learning rate for each category, a probability threshold for each category is determined.

3. The method according to claim 2, characterized in that, The method of determining the learning rate for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model includes: Based on the ratio between the number of detection objects corresponding to each category in the teacher model and the first total number of detection objects, the first proportion of detection objects for each category is determined, where the first total number of detection objects is the sum of the number of detection objects corresponding to each category in the teacher model. The second proportion of detection objects for each category is determined based on the ratio between the number of detection objects for each category in the student model and the second total number of detection objects. The second total number of detection objects is the sum of the number of detection objects for each category in the student model. The learning rate for each category is determined based on the ratio between the proportion of the first detected object in each category and the proportion of the second detected object in each category.

4. The method according to claim 2, characterized in that, The determination of the probability threshold for each category based on the learning rate for each category includes: Determine the maximum learning rate among the learning rates corresponding to the multiple categories; The weight coefficient for each category is determined based on the ratio between the learning rate of each category and the maximum learning rate. The probability threshold for each category is determined by multiplying the weight coefficient corresponding to each category with the preset probability upper limit.

5. The method according to claim 2, characterized in that, Before determining the probability threshold for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model, the method further includes: Remove the categories that belong to the image background from the multiple categories.

6. The method according to claim 1, characterized in that, The step of determining the number of detection objects for each category in the teacher model based on the classification reference information corresponding to the teacher model, and determining the number of detection objects for each category in the student model based on the classification reference information corresponding to the student model, includes: Based on the classification reference information corresponding to the teacher model, the category with the highest probability is selected from the probability of each detected object belonging to each category, and the predicted category of the teacher model is determined for each detected object. Based on the predicted category of the teacher model for each detected object, the number of detected objects corresponding to each category in the teacher model is determined. Based on the classification reference information corresponding to the student model, the category with the highest probability is selected from the probabilities of each detected object belonging to each category, and this category is determined as the predicted category of the student model corresponding to each detected object. Based on the predicted category of the student model corresponding to each detected object, the number of detected objects corresponding to each category in the student model is determined.

7. The method according to claim 1, characterized in that, The step of generating pseudo-labels for the unlabeled images of the student model based on the classification reference information corresponding to the teacher model and the probability threshold of each category includes: Based on the classification reference information corresponding to the teacher model, the category with the highest probability is selected from the probability of each detected object belonging to each of the multiple categories, and determined as the predicted category of the teacher model corresponding to each detected object; If the probability of each detected object corresponding to the predicted category of the teacher model is greater than the probability threshold of the predicted category, then a pseudo-label corresponding to the unlabeled image is generated for the student model based on the predicted category of each detected object corresponding to the teacher model.

8. A training method for an image classification model, characterized in that, The training method is used to train an image classification model with a semi-supervised object detection architecture, the image classification model including a teacher model and a student model, and the training method includes: Obtain an image set for training the image classification model, the image set including labeled images and unlabeled images, and obtain the category labels corresponding to the labeled images and the pseudo labels corresponding to the unlabeled images, wherein the pseudo labels corresponding to the unlabeled images are generated based on the label generation method according to any one of claims 1 to 7; Using the student model, the labeled image and the unlabeled image are classified and identified respectively, and the target classification reference information corresponding to the labeled image and the target classification reference information corresponding to the unlabeled image are obtained. Based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image, the model parameters of the image classification model are adjusted.

9. The method according to claim 8, characterized in that, The model parameters of the image classification model include the model parameters of the student model; The step of adjusting the model parameters of the image classification model based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image includes: Based on the category label corresponding to the labeled image and the target classification reference information corresponding to the labeled image, the supervised loss of the student model is determined; Based on the pseudo-labels corresponding to the unlabeled images and the target classification reference information corresponding to the unlabeled images, the unsupervised loss of the student model is determined. Based on the supervised loss and the unsupervised loss, determine the classification loss of the student model; The model parameters of the student model are adjusted based on the classification loss of the student model.

10. The method according to claim 9, characterized in that, The model parameters of the image classification model also include the model parameters of the student model; the method further includes: The model parameters of the teacher model are adjusted based on the model parameters of the student model and the exponential moving average adjustment strategy.

11. An image classification method, characterized in that, include: The image to be processed is classified and identified by the target model in the image classification model, and the classification reference information corresponding to the target model is obtained. Based on the classification reference information corresponding to the target model, the detection objects contained in the image to be processed and the category to which the detection objects belong are determined; The image classification model is a model with a semi-supervised object detection framework, the image classification model includes a teacher model and a student model, and the object model includes the teacher model and / or the student model; The image classification model is trained based on target classification reference information of labeled images in the image set, target classification reference information of unlabeled images, category labels corresponding to the labeled images, and pseudo labels corresponding to the unlabeled images; the target classification reference information of the labeled images or the target classification information of the unlabeled images is obtained by classifying and recognizing the labeled images or unlabeled images through the student model in the image classification model; the pseudo labels corresponding to the unlabeled images are determined by the method of any one of claims 1 to 7.

12. A label generating apparatus, characterized in that, The label generation device is applied in a semi-supervised object detection architecture, which includes a teacher model and a student model. The label generation device includes: The classification unit is used to classify and recognize unlabeled images using the teacher model to obtain classification reference information corresponding to the teacher model, and to classify and recognize unlabeled images using the student model to obtain classification reference information corresponding to the student model. Both the classification reference information corresponding to the teacher model and the classification reference information corresponding to the student model include N detection objects contained in the identified unlabeled image and the probability that each of the N detection objects belongs to each of the multiple categories; N is a positive integer. The determining unit is used to determine the number of detection objects for each category in the multiple categories corresponding to the teacher model based on the classification reference information corresponding to the teacher model, and to determine the number of detection objects for each category in the multiple categories corresponding to the student model based on the classification reference information corresponding to the student model. The determining unit is further configured to determine the probability threshold for each category based on the number of detection objects corresponding to each category in the teacher model and the number of detection objects corresponding to each category in the student model. The generation unit is used to generate pseudo-labels for the unlabeled images of the student model based on the classification reference information corresponding to the teacher model and the probability threshold of each category.

13. A training device for an image classification model, characterized in that, The training device is used to train an image classification model with a semi-supervised object detection architecture. The image classification model includes a teacher model and a student model. The training device includes: The acquisition unit is configured to acquire an image set for training the image classification model, the image set including labeled images and unlabeled images, and to acquire the category labels corresponding to the labeled images and the pseudo labels corresponding to the unlabeled images, wherein the pseudo labels corresponding to the unlabeled images are generated based on the label generation method according to any one of claims 1 to 7. The classification unit is used to classify and identify the labeled image and the unlabeled image respectively using the student model, and obtain the target classification reference information corresponding to the labeled image and the target classification reference information corresponding to the unlabeled image; The adjustment unit is used to adjust the model parameters of the image classification model based on the target classification reference information corresponding to the labeled image, the category label corresponding to the labeled image, the target classification reference information corresponding to the unlabeled image, and the pseudo label corresponding to the unlabeled image.

14. An image classification device, characterized in that, include: The classification unit is used to classify and identify the image to be processed using the target model in the image classification model, and to obtain the classification reference information corresponding to the target model. The determining unit is used to determine the detection objects contained in the image to be processed and the category to which the detection objects belong, based on the classification reference information corresponding to the target model; Wherein, the image classification model is a model with a semi-supervised object detection framework, the image classification model includes a teacher model and a student model, and the object model includes the teacher model and / or the student model; The image classification model is trained based on target classification reference information of labeled images in the image set, target classification reference information of unlabeled images, category labels corresponding to the labeled images, and pseudo labels corresponding to the unlabeled images; the target classification reference information of the labeled images or the target classification information of the unlabeled images is obtained by classifying and recognizing the labeled images or unlabeled images through the student model in the image classification model; the pseudo labels corresponding to the unlabeled images are determined by the method of any one of claims 1 to 7.

15. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 7; or, the processor is configured to execute the instructions to implement the method as described in any one of claims 8 to 10; or, the processor is configured to execute the instructions to implement the method as described in claim 11.

16. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device performs the method as described in any one of claims 1 to 7; or, the electronic device performs the method as described in any one of claims 8 to 10; or, the electronic device performs the method as described in claim 11.

Images