Open world image classification method based on adaptive contrast learning

Abstract

The invention belongs to the technical field of image classification, and particularly relates to an open world image classification method based on adaptive contrast learning. The method comprises the following steps: firstly, training a pre-training classification model by using source domain data to obtain a source domain training classification model; clustering the feature vectors extracted from the source domain samples by the pre-training classification model to obtain a clustering center of each category; then, training the source domain training classification model during testing by using target domain data, enhancing the target domain data in the training process, and optimizing the model through comparative learning of a sample pair; meanwhile, training and optimizing the model through a category pair, a strong OOD category clustering center and an alignment pair of feature distribution of a source domain and a target domain; and finally, the cosine similarity between the feature vector extracted from the target domain sample by the source domain training classification model and each target domain category clustering center is calculated, and the target domain category corresponding to the maximum cosine similarity is the prediction category label of the target domain sample. According to the method, the adaptability and the recognition capability of the model to unknown categories are improved, the classification stability can be kept when domain transfer and unknown categories of samples are encountered, and misclassification is effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and in particular relates to an open-world image classification method based on adaptive contrast learning. Background Art

[0002] Image classification is a core task in computer vision, and its goal is to classify an image into one or more category labels based on its content. Simply put, the image classification task is to let the computer "see" an image and determine the category it belongs to. This task is widely used in many fields such as autonomous driving, medical diagnosis, smart security, and social media content recommendation.

[0003] Image classification based on deep learning usually assumes that the training data and test data have the same distribution, that is, the independent and identically distributed assumption. However, this assumption is often not true in real scenarios. For example, the data collected by the camera of the autonomous driving system may be significantly different from the training data due to weather, light changes or emergencies; in medical image analysis, due to the resolution of different devices, individual differences in patients, etc., the test data may be inconsistent with the distribution of training data. This distribution inconsistency problem is called the domain shift problem. Due to the existence of domain shift, the model performs well on the training data, but the classification performance will drop significantly in the test data in the real environment, especially in the open world scenario where new or never-seen categories (Out of Distribution, referred to as OOD data) appear. For open-world image classification, the model is required not only to accurately classify images of known categories, but also to classify images of unseen categories to improve the robustness and accuracy of classification.

[0004] In order to solve the domain shift problem, the test time training (TTT) method came into being. TTT uses unlabeled target domain data to adaptively adjust the model in the test phase to narrow the difference between the distribution of training data and test data, thereby improving the classification accuracy of the model. However, traditional TTT methods usually assume that the category sets in the training and test phases are consistent. When new categories appear in the test phase, traditional TTT methods cannot handle strong OOD data well (strong OOD data refers to data of completely unknown categories), making the model easily confused with unseen categories during classification, resulting in classification errors. Open world test time training (OWTTT) can improve the model's distinction between strong OOD and weak OOD data (weak OOD data refers to data of known categories), but it relies on the model's initial ability to extract features from weak OOD data. If this ability is lacking, in the context of significant domain shift, the model may find it difficult to distinguish weak OOD data from strong OOD data, but instead mistake weak OOD data for strong OOD noise, and thus be misclassified as strong OOD data in the test phase. In summary, the present invention proposes an open-world image classification method based on adaptive contrastive learning. Summary of the invention

[0005] In view of the deficiencies in the prior art, the technical problem that the present invention intends to solve is to provide an open-world image classification method based on adaptive contrast learning.

[0006] The present invention solves the technical problem by adopting the following technical solution:

[0007] An open-world image classification method based on adaptive contrastive learning, characterized in that the method comprises the following steps:

[0008] Step 1: Use source domain data to train the pre-trained classification model to obtain a source domain training classification model; cluster the feature vectors extracted from the source domain samples by the pre-trained classification model to obtain the cluster centers of each category in the source domain;

[0009] Step 2: Use the target domain data to train the source domain training classification model during testing, and calculate the training loss based on the final loss function;

[0010] 2.1) Perform data augmentation on target domain samples to generate augmented samples; target domain samples and corresponding augmented samples form positive sample pairs, and target domain samples and non-corresponding augmented samples form negative sample pairs. The source domain training classification model is optimized based on contrastive learning of sample pairs;

[0011] 2.2) Alignment of positive and negative category pairs;

[0012] The strong OOD score of the target domain sample is calculated according to formula (6), and the strong OOD score of the target domain sample is compared with the strong OOD discrimination threshold. If the strong OOD score is greater than the strong OOD discrimination threshold, the target domain sample is a strong OOD sample, otherwise it is a weak OOD sample.

[0013]

[0014] In the formula, Represents the target domain sample x i The strong OOD score, F′(x i ) represents the source domain training classification model F′(·) from the target domain sample x i The feature vector extracted from represents the cluster center of category k in the source domain, <·> represents the cosine similarity, It means to find the cluster center that is most similar to the category of the target domain sample from the cluster center of the source domain category. s Represents the set of cluster centers of source domain categories;

[0015] For weak OOD samples, we embed them near the cluster center of the category they belong to by generating pseudo labels, and calculate the negative log-likelihood loss L of the weak OOD category according to formula (9): wea_PC ;

[0016]

[0017] In the formula, F′(x i_wea ) indicates that the source domain training classification model is trained from weak OOD samples x i_wea The feature vector extracted from wea_a ,d wea_b They represent the cluster centers of weak OOD categories a and b respectively, δ is a hyperparameter, Pseudo-label representing the category, Y s Represents the source domain category set;

[0018] The same categories in the current batch and the source domain constitute positive category pairs, and different categories constitute negative category pairs. The NT-Xent loss L of the category pair is calculated according to formula (12): NT ;

[0019]

[0020] In the formula, Represents the similarity of the positive category pairs composed of category k in the current batch and the source domain, Represent the normalized cluster center vectors of category k in the current batch and source domain, respectively. represents the similarity between the negative category pairs consisting of category k in the current batch and category r in the source domain, represents the normalized cluster center vector of category r in the source domain, represents the similarity between the negative category pair consisting of category e in the current batch and category k in the source domain, represents the normalized cluster center vector of category e in the current batch, and α2 represents the hyperparameter;

[0021] 2.3) Alignment of strong OOD category cluster centers;

[0022] For strong OOD samples, the strong OOD score of the strong OOD sample is calculated according to formula (13); multiple cluster centers of strong OOD categories are created, and each strong OOD category corresponds to a strong OOD category discrimination threshold; the strong OOD score of the strong OOD sample is compared with each strong OOD category discrimination threshold. If the strong OOD score of the strong OOD sample is greater than a certain strong OOD category discrimination threshold, the strong OOD sample belongs to the strong OOD category; if the strong OOD score of the strong OOD sample is less than or equal to all existing strong OOD category discrimination thresholds, the strong OOD sample does not belong to the existing strong OOD category, and a new cluster center of the strong OOD category is created; after traversing all strong OOD samples, the cluster centers of all strong OOD categories are obtained;

[0023]

[0024] In the formula, Is a strong OOD sample x i-str Strong OOD score, d i_str Represents a strong OOD sample x i-str The cluster center of F′(x i_str ) indicates that the source domain training classification model is trained from strong OOD samples x i-str The feature vector extracted from str represents the set of cluster centers of strong OOD categories, Indicates finding the strong OOD sample x from the cluster centers of the source domain category and the strong OOD category i-str The cluster centers with the most similar categories;

[0025] According to the cluster center of the strong OOD category, the negative log-likelihood loss L of the strong OOD category is calculated by formula (14): str_PC ;

[0026]

[0027] Where, d str_c ,d str_d are the cluster centers of strong OOD categories c and d, respectively, and F′(x i_str ) indicates that the source domain training classification model is trained from strong OOD samples x i-str The feature vector extracted from strRepresents a set of strong OOD categories;

[0028] 2.4) KL divergence loss is used to measure the difference between the feature distribution of the source domain and the target domain. The KL divergence loss L is calculated according to formula (15): KLD ;

[0029]

[0030] Where D KL represents the KL divergence distribution, represents the source domain feature distribution, Represents the target domain feature distribution;

[0031] In summary, the final loss function is expressed as:

[0032] L cs =L NT +L wea_PC +L str_PC +L KLD (16)

[0033] Step 3: Calculate the cosine similarity between the feature vector extracted from the target domain sample by the source domain training classification model and the cluster center of each target domain category. The target domain category corresponding to the maximum cosine similarity is the predicted category label of the target domain sample.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. During training, data augmentation is used to generate enhanced samples, and then positive and negative sample pairs are obtained. The feature distance of the positive sample pairs is shortened through comparative learning of the sample pairs, while the feature distance of the negative sample pairs is expanded, ensuring that the model clusters similar samples in the feature space, so that the model can maintain classification stability when encountering domain transfer and samples of unknown categories.

[0036] 2. Use the cluster center of the source domain category as a reference and improve the model's adaptability and recognition ability for unknown categories through comparative learning of category cluster centers.

[0037] 3. The strong OOD discrimination threshold is automatically adjusted with the input of the test sample, helping the model to determine whether the sample is a weak OOD sample of a known category or a strong OOD sample of a completely unknown category, ensuring that the model remains robust in open world scenarios, effectively processing categories that have never been seen, and improving the classification accuracy of unknown categories.

[0038] 4. For weak OOD samples, by generating pseudo labels, their features are aligned to the vicinity of the cluster center of the known category, which improves the classification stability of the model in the target domain and effectively reduces misclassification. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is the overall flow chart of the present invention;

[0040] Figure 2 These are the classification results of different models on the CIFAR10C dataset. DETAILED DESCRIPTION

[0041] Specific embodiments are given below in conjunction with the accompanying drawings. The specific embodiments are only used to introduce the technical solutions of the present invention in detail and are not intended to limit the protection scope of the present application.

[0042] The present invention provides an open-world image classification method based on adaptive contrastive learning (hereinafter referred to as method, see Figures 1-2 ), including the following steps:

[0043] Step 1: Use source domain data to train the pre-trained classification model F(·) so that the model can accurately distinguish source domain categories and obtain the source domain training classification model F′(·); during the training process, the source domain samples are input into the pre-trained classification model to extract feature vectors, and the feature vectors of all source domain samples are clustered to obtain the cluster centers of each category. The cluster centers are expressed as:

[0044]

[0045] In the formula, represents the cluster center of category k in the source domain, M represents the number of source domain samples of category k, F(·) represents the pre-trained classification model, represents the i-th source domain sample of category k, Indicates that the source domain training classification model is obtained from the source domain samples The feature vector extracted from .

[0046] The pre-trained classification model refers to a pre-trained image classification model, which may be a ResNet series network.

[0047] Step 2: Use the target domain data to train the source domain training classification model F′(·) during testing, and calculate the training loss according to the final loss function;

[0048] 2.1) Perform data enhancement on target domain samples to generate enhanced samples; enhance the model’s feature extraction capability through comparative learning of sample pairs, so that the model can better identify and process unknown categories during the test phase; data enhancement uses random rotation, flipping and other means, and uses the NT-Xent loss of sample pairs to optimize the source domain training classification model to enhance the model’s adaptability to features, so that the model focuses on similar samples during feature extraction and improves the model’s adaptability to unknown categories.

[0049] Target domain sample x iAnd the corresponding enhanced sample x' i The target domain samples and the enhanced samples that do not correspond to them constitute positive sample pairs, and the target domain samples and the enhanced samples that do not correspond to them constitute negative sample pairs; the target domain samples and enhanced samples of the current batch are respectively input into the pre-trained classification model F(·) and the source domain training classification model F′(·) to extract feature vectors, and then these feature vectors are normalized by the L2 norm. The normalized feature vector is expressed as:

[0050]

[0051] In the formula, v i and v i ′ are target domain samples x i and the enhanced sample x' i The normalized eigenvector of F(x i ) and F(x i ′) are respectively the pre-trained classification model F(·) from the target domain sample x i and the enhanced sample x' i The feature vector extracted from i ) and F′(x′ i ) are respectively the source domain training classification model F′(·) from the target domain sample x i and the enhanced sample x' i The feature vector extracted from , B is the number of positive sample pairs in the current batch;

[0052] Based on the normalized feature vector v i and v i ′, calculate the similarity of the positive sample pair according to formula (3);

[0053]

[0054] In the formula, S(v i ,v i ′) pos is a positive sample pair x i and x' i similarity; is the normalized eigenvector v i and v i ′, the dot product similarity is used to measure the angular relationship between two feature vectors. The larger the value, the closer the two feature vectors are, that is, the target domain sample x i and the enhanced sample x' i The higher the similarity, the higher the temperature normalization factor.

[0055] The similarity calculation formula for negative sample pairs is as follows:

[0056]

[0057] In the formula, S(v i ,v j ′) neg is a negative sample pair x i and x' j The similarity of S(v i ′,v j ) neg is a negative sample pair x' i and x j The similarity of j , v′ j They are the target domain samples x j and the enhanced sample x' j The normalized eigenvector of ;

[0058] Based on the similarity of positive and negative sample pairs, the NT-Xent loss L of the sample pair is calculated according to formula (5): ps , optimize the source domain training classification model;

[0059]

[0060] Where α1 is a hyperparameter.

[0061] 2.2) Alignment of positive and negative category pairs;

[0062] The target domain data is divided into strong OOD and weak OOD data. The strong OOD score is used to distinguish the strong OOD samples in the target domain. The strong OOD score represents the similarity score between the target domain sample and the source domain sample. The higher the score, the lower the similarity between the target domain sample and the source domain sample, and the higher the possibility of being a strong OOD sample. The definition of the strong OOD score is as follows:

[0063]

[0064] In the formula, Represents the target domain sample x i Strong OOD score, Denotes the eigenvector F′(x i ) and cluster centers The cosine similarity of It means to find the cluster center that is most similar to the category of the target domain sample from the cluster center of the source domain category. s Represents the set of cluster centers of source domain categories;

[0065] The strong OOD score of the target domain sample is compared with the strong OOD discrimination threshold. If the strong OOD score is greater than the strong OOD discrimination threshold, the target domain sample is a strong OOD sample, otherwise it is a weak OOD sample. The strong OOD discrimination threshold τ is dynamically changed and gradually updated as the training process progresses during testing, so that the model can stably adapt to the feature distribution of different samples in the open world scenario. Therefore, in each batch, the optimal strong OOD discrimination threshold τ is optimized according to the following formula: * ;

[0066]

[0067]

[0068] Where N + 、N - Respectively represent the number of strong and weak OOD samples, is the target domain sample x j Strong OOD score; l(·) represents the indicator function, which is used to determine whether the condition is true. If true, it returns 1, otherwise it returns 0; similarly, if the condition If true, it returns 1, otherwise it returns 0; and Represent the total score of strong OOD samples and weak OOD samples, respectively. represents the variance of the strong OOD sample score, represents the variance of the weak OOD sample score; by minimizing the variance of the strong OOD and weak OOD sample scores through formula (9), the score distribution of strong OOD and weak OOD samples is made more separable, ensuring the optimal strong OOD discrimination threshold τ * Can effectively distinguish strong OOD and weak OOD samples;

[0069] For weak OOD samples, by generating pseudo labels, they are embedded near the cluster center of the category to which they belong and away from the cluster centers of other categories. The negative log-likelihood loss L of the weak OOD category is wea_PC The calculation formula is as follows:

[0070]

[0071] In the formula, F′(x i_wea ) represents the source domain training classification model F′(·) from the weak OOD sample x i_wea The feature vector extracted from wea_a ,d wea_b They represent the cluster centers of weak OOD categories a and b respectively, δ is a hyperparameter, Pseudo-label representing the category, Y s Represents the source domain category set;

[0072] In order to enhance the robustness of sample classification and simplify calculation, the cluster center of category k in the current batch is calculated according to formula (10):

[0073]

[0074] In the formula, They represent the i-th target domain sample and enhanced sample of category k in the current batch, respectively. They represent the source domain training classification model F′(·) from the target domain samples and enhanced samples The feature vector extracted from , K represents the number of target domain samples of category k in the current batch;

[0075] The L2 norm is used to normalize the cluster centers of the same category in the current batch and the source domain. The normalization formula is:

[0076]

[0077] In the formula, Represent the normalized cluster center vectors of category k in the current batch and the source domain respectively, and P represents the number of categories;

[0078] The similarity between the positive and negative category pairs is calculated based on the normalized cluster center vector, and the NT-Xent loss L of the category pair is further calculated according to formula (12): NT ;

[0079]

[0080] In the formula, Represents the similarity of the positive category pairs composed of category k in the current batch and the source domain, represents the similarity between the negative category pairs consisting of category k in the current batch and category r in the source domain, represents the normalized cluster center vector of category r in the source domain, represents the similarity between the negative category pair consisting of category e in the current batch and category k in the source domain, represents the normalized cluster center vector of category e in the current batch, and α2 represents the hyperparameter.

[0081] 2.3) Alignment of strong OOD category cluster centers;

[0082] The strong OOD score of a strong OOD sample must meet the following conditions:

[0083]

[0084] In the formula, Is a strong OOD sample xi-str Strong OOD score, Indicates finding the strong OOD sample x from the cluster centers of the source domain category and the strong OOD category i-str The cluster center with the most similar category, <F′(x i_str ),d i_str > represents a strongly OOD sample x i-str The eigenvector F′(x i_str ) and strong OOD sample x i-str The cluster center d i_str The cosine similarity between str Represents the set of cluster centers of strong OOD categories;

[0085] Create multiple cluster centers of strong OOD categories, each of which corresponds to a strong OOD category discrimination threshold with different values; compare the strong OOD score of the strong OOD sample with the existing strong OOD category discrimination thresholds to determine whether the strong OOD sample belongs to the existing strong OOD category, and then obtain the cluster center of the strong OOD category; if the strong OOD score of the strong OOD sample is greater than a certain strong OOD category discrimination threshold, the strong OOD sample belongs to the strong OOD category; if the strong OOD score of the strong OOD sample is less than or equal to all existing strong OOD category discrimination thresholds, the strong OOD sample does not belong to the existing strong OOD category, and a new cluster center of the strong OOD category is created for the judgment of the next strong OOD sample. In order to prevent the cluster center of the strong OOD category from growing infinitely, the cluster center set D of the strong OOD category is str Managed as a queue with fixed capacity, the oldest cluster centers of strong OOD categories are phased out as new cluster centers of strong OOD categories emerge.

[0086] Cluster centers based on strong OOD categories, negative log-likelihood loss L for strong OOD categories str_PC The calculation formula is as follows:

[0087]

[0088] Where, d str_c ,d str_d are the cluster centers of strong OOD categories c and d, respectively, and F′(x i_str ) represents the source domain training classification model F′(·) from the strong OOD sample x i-str The feature vector extracted from str Represents a set of strong OOD categories;

[0089] Self-training (ST) is susceptible to pseudo-labeling errors, a phenomenon known as confirmation bias. Confirmation bias can exacerbate over time and have a significant impact on model performance, especially in the presence of strong OOD samples in the target domain. Even models with lower confidence may mistakenly classify these samples as known categories, exacerbating confirmation bias. To reduce the risk of self-training failure, distribution alignment is used as a form of regularization for self-training, aiming to make the model's predictions closer to the actual distribution of the data to reduce the adverse effects of confirmation bias.

[0090] 2.4) Align the feature distribution of source domain and target domain;

[0091] To refine the clustering of target domain features, KL divergence loss L is used KLD Measures the difference between the feature distributions of the source domain and the target domain. The smaller the value, the closer the two distributions are. KL divergence loss L KLD The calculation formula is as follows:

[0092]

[0093] Where D KL represents the KL divergence distribution, Represents the source domain feature distribution, assuming that its mean is μ s , the covariance matrix is ​​Σ s Gaussian distribution of Represents the target domain feature distribution, assuming that its mean is μ t , the covariance matrix is ​​Σ t The target domain feature distribution is estimated using only strong OOD samples.

[0094] To sum up, the final loss function is expressed as:

[0095] L cs =L NT +L wea_PC +L str_PC +L KLD (16)

[0096] According to the final loss function, the training loss of the source domain training classification model F′(·) on the current batch is calculated and the model is optimized to update the model’s recognition ability for unknown categories, thereby improving the classification stability in open-world scenarios. After adaptive iterative optimization, the model has the ability to distinguish between known and unknown categories.

[0097] Step 3: Assign predicted category labels to target domain samples;

[0098] During the second step of testing and training, the feature vectors of each target domain sample extracted by the model are saved; the cosine similarity is used to quantify the distance between the target domain sample and the target domain category cluster center. The higher the cosine similarity, the closer the sample is to the target domain category in the feature space, that is, the sample is most likely to belong to the target domain category; the cosine similarity between the feature vector of the target domain sample and the cluster center of each target domain category is calculated, and the target domain category corresponding to the maximum cosine similarity is the predicted category label of the target domain sample, thus completing the image classification.

[0099] Example

[0100] This example selects CIFAR10 / CIFAR100 and ImageNet as source domain data, and CIFAR10C, CIFAR100C and ImageNetC as target domain data. In the target domain data, three datasets, Noise, MNIST and SVHN, are introduced as strong OOD samples. S Acc represents the accuracy on the source domain (i.e., weak OOD data). H Indicates the accuracy on the target domain, Acc N It represents the accuracy of strong OOD data. The test results of different methods on different target domain data are shown in Tables 1 to 3.

[0101] Table 1 Test results of different methods on CIFAR10C dataset (%)

[0102]

[0103] Table 2 Test results of different methods on CIFAR100C dataset (%)

[0104]

[0105]

[0106] Table 3 Test results of different methods on ImageNetC dataset (%)

[0107]

[0108] From the above test results, it can be seen that the existing model significantly improves the accuracy of strong OOD samples, but the accuracy of weak OOD samples is greatly reduced. This is because the strong OOD data interference causes the model to lose the ability to identify the source domain category. The method of the present invention has significantly improved the accuracy of both strong OOD and weak OOD samples. This is because the feature extraction ability of the model is enhanced through comparative learning of sample pairs and category cluster centers, preventing weak OOD samples from being mistakenly identified as strong OOD samples.

[0109] Figure 2 The figure shows the classification results of three models, TEST, OWTTT and OWDCL (model of the present invention), on the CIFAR10C dataset. Black represents the strong OOD category, and the other ten colors represent the ten categories in the CIFAR10 dataset. Compared with the TEST model, OWTTT shows higher classification accuracy, but the misclassification rate is also significantly improved. OWDCL shows better classification performance by further expanding the spatial separation between different categories, which shows that OWDCL has significant feature extraction capabilities and can effectively identify strong OOD samples during the test-time training (TTT) process.

[0110] The above model comes from:

[0111] [1]Alexander Bartler,Andre Bühler,Felix Wiewel,Mario and BinYang.2022.Mt3:Meta test-time training for self-supervised test-timeadaption.In International Conference on Artificial Intelligence andStatistics.PMLR,3080–3090.

[0112] [2]Sergey Ioffe,Christian Szegedy.Batch Normalization:AcceleratingDeep Network Training by Reducing Internal Covariate Shift[J],ICML'15:Proceedings of the 32nd International Conference on International Conferenceon Machine Learning-Volume 37,2015,448-456.

[0113] [3]Yuejiang L,Parth K,Bastien V D,Baptiste B,Taylor M,Alexandre A,etal.TTT++:when Does Self-Supervised Test-Time Training Fail or Thrive?[C],Conference on Neural Information Processing Systems,2021.

[0114] [4]Dequan Wang,Evan Shelhamer,Shaoteng Liu,Bruno Olshausen,and TrevorDarrell.2020.Tent:Fully test-time adaptation by entropy minimization.arXivpreprint arXiv:2006.10726(2020).

[0115] [5]Jian Liang,Dapeng Hu,and Jiashi Feng.2020.Do we really need toaccess the source data?source hypothesis transfer for unsupervised domainadaptation.In International conference on machine learning.PMLR,6028–6039.

[0116] [6]Yongyi Su,Xun Xu,and Kui Jia.2022.Revisiting realistic test-timetraining:Sequential inference and adaptation by anchored clustering.Advancesin Neural Information Processing Systems 35(2022),17543–17555.

[0117] [7] Yushu Li,

[0118] Any matters not described in the present invention are applicable to the prior art.

Claims

1. An open-world image classification method based on adaptive contrastive learning, characterized in that: The method comprises the following steps: Step 1: Use source domain data to train the pre-trained classification model to obtain a source domain training classification model; cluster the feature vectors extracted from the source domain samples by the pre-trained classification model to obtain the cluster centers of each category in the source domain; Step 2: Use the target domain data to train the source domain training classification model during testing, and calculate the training loss based on the final loss function; 2.1) Perform data augmentation on target domain samples to generate augmented samples; target domain samples and corresponding augmented samples form positive sample pairs, and target domain samples and non-corresponding augmented samples form negative sample pairs. The source domain training classification model is optimized based on contrastive learning of sample pairs; 2.2) Alignment of positive and negative category pairs; The strong OOD score of the target domain sample is calculated according to formula (6), and the strong OOD score of the target domain sample is compared with the strong OOD discrimination threshold. If the strong OOD score is greater than the strong OOD discrimination threshold, the target domain sample is a strong OOD sample, otherwise it is a weak OOD sample. In the formula, Represents the target domain sample x i The strong OOD score, F′(x i ) represents the source domain training classification model F′(·) from the target domain sample x i The feature vector extracted from represents the cluster center of category k in the source domain, <·> represents the cosine similarity, It means to find the cluster center that is most similar to the category of the target domain sample from the cluster center of the source domain category. s Represents the set of cluster centers of source domain categories; For weak OOD samples, we embed them near the cluster center of the category they belong to by generating pseudo labels, and calculate the negative log-likelihood loss L of the weak OOD category according to formula (9): wea_PC ; In the formula, F′(x i_wea ) indicates that the source domain training classification model is trained from weak OOD samples x i_wea The feature vector extracted from wea_a ,d wea_b They represent the cluster centers of weak OOD categories a and b respectively, δ is a hyperparameter, Pseudo-label representing the category, Y s Represents the source domain category set; The same categories in the current batch and the source domain constitute positive category pairs, and different categories constitute negative category pairs. The NT-Xent loss L of the category pair is calculated according to formula (12): NT ; In the formula, Represents the similarity of the positive category pairs composed of category k in the current batch and the source domain, Represent the normalized cluster center vectors of category k in the current batch and source domain, respectively. represents the similarity between the negative category pairs consisting of category k in the current batch and category r in the source domain, represents the normalized cluster center vector of category r in the source domain, represents the similarity between the negative category pairs composed of category e in the current batch and category k in the source domain, represents the normalized cluster center vector of category e in the current batch, and α2 represents the hyperparameter; 2.3) Alignment of strong OOD category cluster centers; For strong OOD samples, the strong OOD score of the strong OOD sample is calculated according to formula (13); multiple cluster centers of strong OOD categories are created, and each strong OOD category corresponds to a strong OOD category discrimination threshold; the strong OOD score of the strong OOD sample is compared with each strong OOD category discrimination threshold. If the strong OOD score of the strong OOD sample is greater than a certain strong OOD category discrimination threshold, the strong OOD sample belongs to the strong OOD category; if the strong OOD score of the strong OOD sample is less than or equal to all existing strong OOD category discrimination thresholds, the strong OOD sample does not belong to the existing strong OOD category, and a new cluster center of the strong OOD category is created; after traversing all strong OOD samples, the cluster centers of all strong OOD categories are obtained; In the formula, Is a strong OOD sample x i-str Strong OOD score, d i_str Represents a strong OOD sample x i-str The cluster center of F′(x i_str ) indicates that the source domain training classification model is trained from strong OOD samples x i-str The feature vector extracted from str represents the set of cluster centers of strong OOD categories, Indicates finding the strong OOD sample x from the cluster centers of the source domain category and the strong OOD category i-str The cluster centers with the most similar categories; According to the cluster center of the strong OOD category, the negative log-likelihood loss L of the strong OOD category is calculated by formula (14): str_PC ; Where, d str_c ,d str_d are the cluster centers of strong OOD categories c and d, respectively, and F′(x i_str ) indicates that the source domain training classification model is trained from strong OOD samples x i-str The feature vector extracted from str Represents a set of strong OOD categories; 2.4) KL divergence loss is used to measure the difference between the feature distribution of the source domain and the target domain. The KL divergence loss L is calculated according to formula (15): KLD ; L KLD =D KL (N(μ s ,S s )‖N(μ t ,S t )) (15) Where D KL represents the KL divergence distribution, N(μ s ,Σ s ) represents the source domain feature distribution, N(μ t ,Σ t ) represents the target domain feature distribution; In summary, the final loss function is expressed as: L cs =L NT +L wea_PC +L str_PC +L KLD (16) Step 3: Calculate the cosine similarity between the feature vector extracted from the target domain sample by the source domain training classification model and the cluster center of each target domain category. The target domain category corresponding to the maximum cosine similarity is the predicted category label of the target domain sample.

2. The open-world image classification method based on adaptive contrastive learning according to claim 1, characterized in that: In step 2.1), the target domain samples and enhanced samples of the current batch are respectively input into the pre-trained classification model and the source domain training classification model to extract feature vectors, and the feature vectors are normalized by formula (2); In the formula, v i and v′ i They are the target domain samples x i and the enhanced sample x' i The normalized eigenvector of F(x i ) and F(x′ i ) are respectively the pre-trained classification model from the target domain sample x i and the enhanced sample x' i The feature vector extracted from i ) and F′(x′ i ) are respectively the source domain training classification model from the target domain sample x i and the enhanced sample x' i The feature vector extracted from , B is the number of positive sample pairs in the current batch; The similarity of the positive sample pair is calculated according to formula (3): In the formula, S(v i ,v′ i ) pos is a positive sample pair x i and x' i Similarity, γ1 represents the temperature normalization factor; According to formula (4), the similarity of negative sample pairs is calculated: In the formula, S(v i ,v′ j ) neg is a negative sample pair x i and x' j The similarity of S(v′ i ,v j ) neg is a negative sample pair x' i and x j The similarity of j , v′ j They are the target domain samples x j and the enhanced sample x' j The normalized eigenvector of ; According to formula (5), the NT-Xent loss L of the sample pair is calculated ps : Where α1 is a hyperparameter.

3. The open-world image classification method based on adaptive contrastive learning according to claim 1 or 2, characterized in that: In each batch, the strong OOD discrimination threshold is optimized according to the following formula to obtain the optimal strong OOD discrimination threshold; Where N + 、N - Respectively represent the number of strong and weak OOD samples, is the target domain sample x j , τ represents the strong OOD score, l(·) represents the indicator function, which is used to determine whether the condition is met. If the condition is met, 1 is returned, otherwise 0 is returned.

4. The open-world image classification method based on adaptive contrastive learning according to claim 3, characterized in that: Cluster center of category k in the source domain It is expressed as: In the formula, M represents the number of source domain samples of category k, F(·) represents the pre-trained classification model, represents the i-th source domain sample of category k, Indicates that the source domain training classification model is obtained from the source domain samples The feature vector extracted from .

5. The open-world image classification method based on adaptive contrastive learning according to claim 1, characterized in that: Normalized cluster center vectors of category k in the current batch and source domain and It is expressed as: In the formula, represents the cluster center of category k in the current batch, P represents the number of categories, K represents the number of target domain samples of category k in the current batch, Respectively represent the source domain training classification model from the target domain samples and enhanced samples The feature vector extracted from .

Images