Open set target detection method based on self-distillation

Abstract

The invention designs an open set target detection method based on self-distillation. The method comprises the following steps: dividing a data set into an open set training set and a test set; performing weak data enhancement and strong data enhancement on the training data set image; obtaining a teacher model and a student model based on a Faster RCNN model structure; after weak enhancement data is input based on a teacher model, a reliable pseudo tag is obtained; a consistency distillation loss function is utilized to ensure the consistency of a detection result of the student model inputting the strong enhancement data and a pseudo label detection result obtained by the teacher model; further screening unknown classes of targets by utilizing teacher model pseudo label information, and calculating the real example comparison learning loss by utilizing the characteristics of unknown classes of pseudo labels; and training a student model by using the corresponding loss function and outputting a detection result. The method can effectively improve the detection and recognition precision of the open set detection model in a noise environment.

Description

Technical Field

[0001] The present invention belongs to the field of open set target detection in computer vision, and in particular relates to an open set target detection method based on self-distillation. Background Art

[0002] Object detection has always been a basic research field of computer vision. Compared with the traditional closed-set object detection method, the challenge faced by the open-set detection model is to detect objects of categories that do not appear in the training set. Since there are more categories of objects in the image than those given in the data, open-set detection is more in line with actual application scenarios and plays a key role in applications such as autonomous driving and defect detection.

[0003] Due to the lack of annotation information of unknown classes in the training set, existing open set learning methods mainly solve the open set recognition problem from the feature level and uncertainty measurement-based methods. Based on the learning and use of known classes in deep networks, existing methods perform representation learning in datasets rich in known classes, which may limit the model's migration to other datasets for open set detection.

[0004] There are three main problems with existing technologies: First, traditional closed-set object detection methods will misclassify objects of unknown categories as known categories in open-set scenarios. This is because they do not take into account the existence of unknown categories during training. The second is the limitation on feature representation learning. Existing methods perform feature representation learning in datasets rich in known classes, which may limit the ability of the model to migrate to other datasets for open-set detection. The last problem is that existing open-set object detection technology performs poorly in terms of model robustness, especially when faced with a lot of noise in unknown categories. There is a lack of effective means to improve the robustness of the model in noisy environments. Summary of the invention

[0005] The present invention proposes an open set target detection method based on self-distillation, which aims to solve the problem that the traditional target detection model misclassifies unknown classes into known classes when applied to open set scenarios and the poor robustness of the existing open set target detection technology model.

[0006] Technical solution of the present invention: An open set target detection method based on self-distillation comprises the following steps: Step 1: Divide the data set into training set and test set.

[0007] Step 2: Perform weak data enhancement and strong data enhancement on the images in the training dataset.

[0008] Step 3: Construct teacher model and student model based on the Faster RCNN model structure.

[0009] Step 4: The weakly enhanced data in step 2 is input into the teacher model to obtain reliable pseudo labels of known classes.

[0010] Step 5: Input the strongly enhanced data in step 2 into the student model to obtain the detection result; calculate the consistency distillation loss function of the detection result of the student model and the pseudo-label detection result obtained by the teacher model.

[0011] Step 6: Use the pseudo-label information of the teacher model to further screen the unknown class targets, and use the features of these unknown class pseudo-labels to calculate their instance contrast learning loss. Calculate the total loss function to train the student model and output the detection results.

[0012] Through the above-mentioned scheme, the self-distillation framework and the consistency distillation loss function are used to improve the robustness of the model. The smooth parameters of the teacher model are used to stably output high-quality pseudo-labels of unknown classes, and the detection accuracy of unknown classes is improved through example comparative learning.

[0013] By adopting the above technical solution, with stable unknown class features output by the teacher model and known class features output by the student model, categories can be better distinguished in the feature implicit space after instance contrastive learning.

[0014] Beneficial effects: (1) This paper designs a new open-set object detection model, which uses the semi-supervised idea to input strongly enhanced data and weakly enhanced data and uses the consistency distillation loss of self-distillation to help the open-set model adapt to the noise existing in unknown classes and improve the robustness of the model.

[0015] (2) The present invention uses a self-distillation-based model architecture and the smoothness of its teacher model to obtain high-quality pseudo-labeling of unknown classes. It further uses an example comparative learning method to make the feature level have a clearer decision boundary to distinguish between known classes and unknown classes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0017] Figure 1 Flow chart of the open set target detection method based on self-distillation of the present invention; Figure 2 This is a structural diagram of the open set target detection model based on self-distillation of the present invention; Figure 3 This is a test effect diagram of the benchmark model of the present invention; Figure 4 This is a diagram of the detection effect of the present invention in the test data set. DETAILED DESCRIPTION

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0019] An open set target detection method based on self-distillation includes the following steps: Figure 1 : Step 1: Divide the data set into training set and test set.

[0020] This embodiment uses the COCO public dataset and the PASCAL VOC dataset. The COCO public dataset contains 80 categories of target detection annotated data, the VOC dataset contains 20 categories of target detection annotated data, and the 80 categories of the COCO dataset cover the 20 categories of the VOC dataset.

[0021] Training set: This example uses VOC's trainval and test for closed set training and model evaluation.

[0022] Test set: The evaluation dataset VOC-COCO-{20,40,60} for the open set object detection task is constructed by reconstructing the VOC and COCO datasets.

[0023] Specifically, VOC-COCO-{20,40,60} contains 5,000 images from the VOC test dataset (20 known categories), and 5,000, 10,000, and 150,000 COCO images (corresponding to 20, 40, and 60 images of non-VOC dataset categories). This setting is more in line with real-world scenarios. The categories in the VOC dataset are known categories, and the open-set object detection task is evaluated by dividing the data containing unknown categories in COCO.

[0024] Step 2: Perform weak data enhancement and strong data enhancement on the images in the training dataset.

[0025] The images in the training dataset are weakly enhanced and strongly enhanced, and provided to the teacher model in step 4 and the student model in step 5, respectively.

[0026] The weak data enhancement includes: random rotation (horizontal or vertical), random cropping; The strong data enhancement includes: random selection of enhancement strategies, random color space transformation (adjusting image contrast, inverting pixel values, adjusting image color channels), random set transformation (rotating by a certain angle, shearing in the X / Y axis direction, randomly translating images in the X / Y direction), and random erasing.

[0027] Specifically, Color space operations: AutoContrast, which automatically adjusts the contrast of an image to make the brightness distribution of the image more even; Solarize, which inverts certain pixel values ​​in an image so that they look like solarized effects; Color, which adjusts the color channels of an image, which may change the saturation or hue of the color.

[0028] Geometric transformation operations: Rotate, randomly rotate the image to a certain angle; ShearX / ShearY, shear the image on the X-axis or Y-axis, which can change the direction of the straight lines in the image; TranslateX / TranslateY, randomly translate the image in the X-axis or Y-axis direction.

[0029] Random Erase Data Operation: An augmentation operation that randomly erases a portion of the image to simulate noise and occlusion in the data. Specify the range of the number of patches to erase (1 to 5 patches) and the scale range of the erased area (0 to 0.2).

[0030] Step 3: Construct teacher model and student model based on the Faster RCNN model structure.

[0031] like Figure 2 ,The present invention is based on a self-distillation architecture and includes two teacher and student models with the same structure and parameter scale, both of which adopt the structure of the two-stage object detection model Faster R-CNN.

[0032] Among them, the teacher model is responsible for obtaining reliable pseudo labels of known classes and providing high-quality pseudo label features of unknown classes; the student model calculates the consistency distillation loss based on the known class label information of the labeled data and the pseudo label information of the teacher model, and trains the model by comparing the learning loss and the supervised loss function through calculation examples.

[0033] Step 4: The weakly enhanced data in step 2 is input into the teacher model to obtain reliable pseudo labels of known classes.

[0034] Based on the weakly enhanced data in step 2, the teacher model obtains the target detection result through the RPN network and RoI Head of Faster RCNN. The detection result includes the bounding box and category score.

[0035] According to the obtained detection results, the candidate boxes are sorted from high to low according to the category score, and the threshold is used for restriction to filter out the candidate boxes with the highest score category confidence score greater than the threshold in the detection results as pseudo labels.

[0036] Step 5: Input the strongly enhanced data in step 2 into the student model to obtain the detection result; and calculate the consistency distillation loss function of the detection result of the student model and the pseudo-label detection result obtained by the teacher model.

[0037] In the self-distillation framework, the student model uses a strong data enhancement strategy. During the distillation process, the student model needs to adapt to the complex situations in real scenarios. Therefore, strong data enhancement is introduced to help the student model learn more robust feature representations. Secondly, the generalization ability of the model can be improved in the face of data changes and noise. The present invention mainly uses color space operations, geometric transformation operations and random erasure data operations.

[0038] The strongly enhanced image is input into the student model. Based on the pseudo-label information obtained by the teacher model in step 4, the candidate box of the pseudo-label is mapped through RoI-Align to obtain the features of the candidate area of ​​the student model. The candidate area features are input into the RoI Head to obtain the logits value of the classification result. The consistency distillation loss is used to ensure the consistency of the prediction results of the teacher model and the student model, thereby improving the robustness of the model.

[0039] 5.1 Consistency Distillation Loss Function The consistency distillation loss function aims to ensure that the student model can maintain similar detection results to the teacher model when inputting strongly enhanced data and there is a certain amount of noise in the image. It is used to improve the robustness of the model. The formula is as follows: in represents the category of the i-th pseudo label generated by the teacher model; is the category score (logits) of the student model obtained by mapping the bounding box of the teacher model’s pseudo-label. The bounding box is predicted by the teacher model on the unlabeled data and filtered by the confidence threshold to ensure high-quality pseudo-labels. is the number of pseudo labels; is the cross entropy loss, which represents the difference between the probability distribution predicted by the model and the true label.

[0040] The cross entropy loss function is used to perform consistent knowledge distillation to match the detection results of the teacher and student models.

[0041] The cross-entropy loss function calculates the difference between the probability distribution predicted by the model and the true label. is the first The one-hot encoding vector of the pseudo-label category, and is the probability distribution of the logits values ​​of the student model corresponding to these pseudo labels converted by the softmax function, then the cross entropy loss It can be expressed as: in is the total number of categories. is the true label The value of the category, one-hot encoding, the sample belongs to the category but is 1 if the value is true, otherwise it is 0. The model predicts The probability of a class.

[0042] By adopting the above technical solution, the consistency distillation loss matches the detection results obtained by inputting strong augmented data to the student model and the detection results of the teacher model. Since the strong augmented data simulates the noisy data in reality, it helps the robustness of the model.

[0043] Step 6: Use the pseudo-label information of the teacher model to further screen the unknown class targets, and use the features of these unknown class pseudo-labels to calculate their instance contrast learning loss. Calculate the total loss function to train the student model and output the detection results.

[0044] 6.1 Calculating instance contrastive learning loss In calculation step three, the IoU between the pseudo-label of the teacher model and the real annotation box of the known class in the training set is obtained, and all pseudo-labels less than 0.5 are screened out as pseudo-labels of unknown class targets. The feature outputs of these pseudo-labels in the teacher model are extracted and used as features of unknown classes.

[0045] The weakly enhanced data in step 2 is input into the student model, and the local features of RoI-Align of known classes are obtained.

[0046] The contrastive learning loss is calculated by using the features of known and unknown classes.

[0047] The instance contrastive learning loss aims to reduce the intra-class feature distance and increase the inter-class feature distance in the feature implicit space. The formula is as follows: N represents the number of all instance features, Represents the category score logits value of the i-th target instance before the activation function, represents the contrastive learning loss for a single instance.

[0048] The contrastive learning loss for a single instance is for: Indicates category The number of memory banks in the feature cache. Indicates that in addition to the category Other instances besides the instance, Represents the temperature coefficient.

[0049] The feature cache queue is a data structure for storing historical sample features. These features are usually calculated and stored in the early stages of model training. Its main function is to serve as a negative sample set to help the model learn features during the training process.

[0050] Furthermore, in the instance contrastive learning loss, the pseudo-label detection results obtained by the teacher model exclude the pseudo-labels with IoU>0.5 (Intersection over Union) with the known class true labels as the logits values ​​of the unknown class targets. The student model calculates the instance contrastive learning loss through the output known class logits values ​​and the unknown class logits values.

[0051] 6.2 Calculate the total loss function, train the neural network and output the detection results.

[0052] In order to ensure the training of the model, in addition to the consistency distillation loss proposed in step 5 and the instance contrastive learning loss proposed in step 6.1, it is also necessary to calculate the supervised training loss under the target detection task. The supervised loss function includes classification loss and bounding box regression loss.

[0053] In order to further improve the recognition ability of unknown classes, the present invention improves the classification loss and proposes an unknown class-aware classification loss function, the formula is as follows: in represents the cross entropy loss function for known class supervision information, It represents the classification loss for unknown classes. The classification loss formula for unknown classes is as follows: Indicates unknown category, represents the category score (logits value) of the unknown class instance, Represents a set of known class categories, and the above formula is expressed as calculating the conditional probability of the unknown class.

[0054] The consistency distillation loss function obtained in step 5 is summarized , the instance contrast learning loss function obtained in the previous step , supervised training loss unknown class-aware classification loss function and the loss function used in the model to measure the difference between the predicted bounding box and the true bounding box , forming the total loss function ,as follows: in, The calculation method is a prior art and is not limited in this embodiment.

[0055] The open set target detection model is trained using the total loss function to obtain the final detection results.

[0056] 6.3 Update model parameters Among them, the student model updates its parameters through the stochastic gradient descent algorithm according to the given loss function, and the teacher model updates its parameters through the EMA method based on the obtained student model parameters.

[0057] Using the VOC-COCO-20 / 40 / 60 test set, the detection effects of the models obtained by the proposed method and the traditional Faster RCNN method are compared. The open set detection results are shown in Table 2. The open set indicators WI (Wilderness Impact), AOSE (Absolute Open-set Error, absolute open set error), (Average Precision of unknown classes, the average accuracy of open set categories), and the average accuracy of known classes Four evaluation indicators are used to assess the results.

[0058] Compared with the prior art, the proposed method has a lower WI, indicating that the model can better identify unknown categories and will not easily misclassify unknown category samples into known categories; a lower AOSE, indicating that the model has better performance in an open set environment, can effectively identify known categories and reject unknown categories; a higher , indicating that the model can more accurately identify unknown categories and will not easily misclassify unknown categories as known categories; higher , indicating that the model has stronger classification and positioning capabilities, which means that the model can perform very well on known categories and accurately identify and locate samples of known categories.

[0059] Table 2 Results of self-distillation open set target detection method Figure 3 and Figure 4The detection result diagrams of the baseline model and the detection result diagrams of the method proposed in this invention are given respectively. From the comparison of Figure (a), it can be seen that the baseline model, that is, the traditional target detection model, has the problem of classifying unknown classes into known classes with high confidence, and the unknown classes can be well identified from the result diagram of the method in this article. From Figure (b), it can be seen that the method proposed in this article has excellent ability to detect unknown classes, and can detect the unknown category "snowboard" with high confidence. From Figure (f), it can be seen that in a dense environment, the model can also detect unknown classes while ensuring the accurate detection capability of known classes. Other result diagrams also show the excellent performance of the model in open set target detection.

[0060] In summary, the open set object detection method proposed in the present invention uses computer technology and deep learning, and the proposed self-distillation-based method and consistency distillation loss effectively improve the robustness of the model and can generate high-quality pseudo-labels of unknown classes. Moreover, the instance contrastive learning proposed in the present invention can more accurately distinguish known classes from unknown classes in the feature implicit space compared to the existing open set model.

[0061] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

[0062] Appendix: Description of evaluation indicators (1)WI (Wilderness Impact) The meaning of the WI indicator is: confirm how many unknown class targets are misclassified as known classes. Its definition is shown in the formula: Where Pk and Pk∪u represent the category accuracy under closed and open sets, respectively. Where FPU means that any detection belonging to the unknown category cu is classified as one of the known category set {ck}.

[0063] (2)AOSE (Absolute Open-set Error) The AOSE metric refers to "Absolute Open-Set Error". It is used to calculate the absolute number of times the model incorrectly classifies unknown categories as known categories. The lower the AOSE value, the better the model performs in an open set environment, and can effectively identify known categories and reject unknown categories. It helps researchers understand the robustness and generalization ability of the model when facing unknown categories.

[0064] (3)APu (Average Precision of unknown classes) The average accuracy of the open set category is calculated in the same way as the AP of a single category of the known class. The calculation process is as follows: 1. Predictions and ground-truth annotations: For each image in the test set, the model generates a series of predicted bounding boxes, each with an associated predicted class and confidence.

[0065] 2. Identification of unknown categories: In open-set object detection, in addition to known categories, there are also objects of unknown categories. The model needs to be able to identify the bounding boxes of these unknown categories.

[0066] 3. Calculate IoU: In the open set test phase, due to the division of the data set, the unknown classes in the training phase are labeled in the test or validation set. The IoU between the open set target and the true label can be calculated by comparing the true label. If the IoU is higher than a certain threshold, the prediction is considered correct.

[0067] 4. Precision and recall: For unknown categories, calculate the model's precision Pu and recall Ru. Precision refers to the proportion of all bounding boxes predicted as unknown categories that actually belong to the unknown category; recall refers to the proportion of all bounding boxes of real unknown categories that are correctly predicted by the model as unknown categories.

[0068] 5.PR curve: By changing the confidence threshold of the predicted bounding box, a series of confidence thresholds can be generated. The precision and recall values ​​are plotted on the Precision-Recall plane to form a curve.

[0069] 6. Calculate A Pu: A Pu is the area under the PR curve, which can be calculated by integrating the curve or using an approximate method. For example, COCO uses 101 recall points, or 100 intervals, to calculate the sum of the rectangular areas in the PR curve, while VOC uses 11 points.

[0070] (4) mAP (Mean Average Precision) mAP (Mean Average Precision) is an indicator for evaluating the accuracy of the target detection model. It takes into account the model's precision and recall. The following are the steps for calculating mAP: 1. Determine the IoU threshold: 0.5 is usually used as the Intersection over Union (IoU) threshold, which means that the overlapping area of ​​the predicted box and the true box is at least half of the area of ​​both.

[0071] 2. Calculate the Precision-Recall curve: For each category, sort according to the confidence score of the predicted box, and then calculate the precision and recall at different IoU thresholds.

[0072] 3. Calculate the average precision (AP): At each recall level, select the value with the highest accuracy as the average precision at that recall level. Then, through the Precision-Recall curve, calculate the area under the curve as the AP value. Usually, the 101-point interpolation method is used to calculate the area: that is, between Recall 0 and 1, with a step size of 0.01, a total of 101 points, and the Precision corresponding to each point takes the maximum Precision value after the point (on the right).

[0073] 4. Calculate mAP: Average the AP of all categories to get the final mAP value.

[0074] mAP = (AP1 + AP2 + … + APN) / N, Where N is the total number of categories and APi is the AP value of the i-th category.

Claims

1. An open set target detection method based on self-distillation, characterized in that: The following steps are included: Step 1: Divide the data set into training set and test set; Step 2: Perform weak data enhancement and strong data enhancement on the training dataset images; Step 3: Construct the teacher model and student model based on the Faster RCNN model structure; Step 4: The weakly enhanced data in step 2 is input into the teacher model to obtain reliable pseudo labels of known classes; Step 5: Input the strongly enhanced data in step 2 into the student model to obtain the test results; calculate the consistency distillation loss function of the test results of the student model and the pseudo-label test results obtained by the teacher model; Step 6: Use the pseudo-label information of the teacher model to further screen the unknown class targets, and use the features of these unknown class pseudo-labels to calculate their instance contrast learning loss, calculate the total loss function, train the student model and output the detection results.

2. The open set target detection method based on self-distillation as claimed in claim 1, characterized in that: In step one, Training set: Use VOC's trainval and test for closed set training and model evaluation; Test set: The evaluation dataset VOC-COCO-{20,40,60} for the open set object detection task is constructed by reconstructing the VOC and COCO datasets; specifically, VOC-COCO-{20,40,60} contains 5,000 test dataset images from VOC, and 5,000, 10,000, and 150,000 COCO images respectively.

3. The open set target detection method based on self-distillation as claimed in claim 1, characterized in that: In step 2, the images in the training dataset are weakly enhanced and strongly enhanced respectively, and provided to the teacher model in step 4 and the student model in step 5 respectively; The weak data enhancement includes: random rotation (horizontal or vertical), random cropping; The strong data enhancement includes: random selection of enhancement strategies, random color space transformation (adjusting image contrast, inverting pixel values, adjusting image color channels), random set transformation (rotating by a certain angle, shearing in the X / Y axis direction, randomly translating images in the X / Y direction), and random erasing.

4. The open set target detection method based on self-distillation as claimed in claim 1, characterized in that: In step three, a teacher model and a student model are constructed based on the Faster R-CNN model structure, which includes two teacher and student models with the same structure and parameter scale. Both adopt the structure of the two-stage target detection model Faster R-CNN. The teacher model is responsible for obtaining reliable pseudo labels of known classes and providing high-quality pseudo label features of unknown classes. The student model calculates the consistency distillation loss based on the known class label information of the labeled data and the pseudo label information of the teacher model, and trains the model by comparing the learning loss and the supervised loss function through calculation examples.

5. The open set target detection method based on self-distillation as claimed in claim 1, characterized in that: Step 4 is as follows: Based on the weakly enhanced data in step 2, the teacher model is input into the teacher model. The teacher model obtains the target detection result through the RPN network and RoI Head of Faster RCNN. The detection result includes the bounding box and category score. According to the obtained detection results, the candidate boxes are sorted from high to low according to the category score, and the threshold is used for restriction to filter out the candidate boxes with the highest score category confidence score greater than the threshold in the detection results as pseudo labels.

6. The open set target detection method based on self-distillation as claimed in claim 1, characterized in that: Step 5 is as follows: the strongly enhanced image is input into the student model. Based on the pseudo-label information obtained by the teacher model in step 4, the candidate box of the pseudo-label is mapped through RoI-Align to obtain the features of the candidate area of ​​the student model. The candidate area features are input into RoI Head to obtain the logits value of the classification result. The consistency distillation loss is used to ensure the consistency of the prediction results of the teacher model and the student model, thereby improving the robustness of the model.

7. The open set target detection method based on self-distillation as claimed in claim 6, characterized in that: The consistency distillation loss function is used to ensure that the student model can maintain similar detection results as the teacher model when inputting strongly enhanced data and there is a certain amount of noise in the image; and to improve the robustness of the model; the formula is as follows: in represents the category of the i-th pseudo label generated by the teacher model, is the category score (logits) of the student model obtained by mapping the bounding box of the teacher model’s pseudo-label. The bounding box is predicted by the teacher model on the unlabeled data and filtered by the confidence threshold to ensure high-quality pseudo-labels. is the number of pseudo labels; is the cross entropy loss, which represents the difference between the probability distribution predicted by the model and the true label; is the cross entropy loss.

8. The open set target detection method based on self-distillation as claimed in claim 1, characterized in that: Step 6: The example contrastive learning loss is calculated as follows: In step 3, the IoU between the pseudo-label of the teacher model and the real annotation box of the known class in the training set is obtained, and all pseudo-labels less than 0.5 are selected as pseudo-labels of unknown class targets. The feature outputs of these pseudo-labels in the teacher model are extracted and used as features of unknown classes. Input the weakly enhanced data in step 2 into the student model and obtain the local features of RoI-Align of the known class; Compare the learning loss between known and unknown features; The instance contrastive learning loss aims to reduce the intra-class feature distance and increase the inter-class feature distance from the feature implicit space. The formula is as follows: N represents the number of all instance features, Represents the category score logits value of the i-th target instance before the activation function, represents the contrastive learning loss for a single instance; The contrastive learning loss for a single instance is for: Indicates category The number of memory banks in the feature cache. Indicates that in addition to the category Other instances besides the instance, Represents the temperature coefficient.

9. The open set target detection method based on self-distillation as claimed in claim 1, characterized in that: Step 6: Total loss function The calculation method is as follows: in, is the consistency distillation loss function, is the instance contrastive learning loss function, is the classification loss function that is aware of the unknown class for supervised training loss, is the loss function used in the model to measure the difference between the predicted bounding box and the true bounding box; The open set target detection model is trained using the total loss function to obtain the final detection results.

10. The open set target detection method based on self-distillation according to claim 9, characterized in that: The unknown class-aware classification loss function is as follows: in represents the cross entropy loss function for known class supervision information, It represents the classification loss for unknown classes; the classification loss formula for unknown classes is as follows: Indicates unknown category, represents the category score of the unknown class instance, Represents a set of known class categories, and the above formula is expressed as calculating the conditional probability of the unknown class.

Images