Method for adapting to open set image classification test based on class relation divergence

Abstract

The invention relates to an adaptation method for an open set image classification test based on class relation divergence, and the method comprises the following steps: selecting a public image data set, and selecting a general pre-training model to complete pre-training on the data set; selecting a target data set, dividing the target data set into a plurality of batches, and calculating the class relation divergence corresponding to each to-be-classified prediction image in each batch; presetting a class relation divergence threshold value, comparing the obtained class relation divergence corresponding to each to-be-classified image with the threshold value, and forming a known class sample subset by the to-be-classified images smaller than the threshold value; and then calculating a similarity value between each to-be-classified image and the category prototype of each known category, selecting the category corresponding to the highest similarity value as a prediction category of the to-be-classified image, and updating the original category prototype for prediction of the next batch. According to the method, a sample discrimination and adaptive constraint mechanism based on class relation divergence is introduced in a test stage, so that the method has a remarkable effect in an open set test environment.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an adaptive method for testing open set image classification based on class relation divergence. Background Technology

[0002] Deep learning models have been widely applied in computer vision tasks such as image classification and object recognition, and their performance largely depends on the consistency of the distribution between training data and real-world application data. However, in the actual deployment of models, the data received during the testing phase often comes from unknown or changing environments, resulting in significant differences in distribution from the source data during the training phase, thus leading to a decline in model performance. Since test data is usually not manually labeled, how to improve the model's adaptability during the testing phase under unsupervised conditions has become an important issue in current research and application.

[0003] To alleviate the aforementioned problems, Test-Time Adaptation (TTA) methods have been proposed and are gradually gaining attention. These methods typically update model parameters or intermediate statistics online using continuously arriving test data after model deployment, allowing the model to gradually adapt to changes in the data distribution of the target environment. Existing test-time adaptive methods generally aim to minimize prediction uncertainty, align feature distributions, or maintain model output stability, mitigating target domain distribution shifts without accessing source domain data or retraining the model.

[0004] Existing TTA methods mitigate the distribution offset problem between training and test data by updating model parameters or internal statistics online using continuously arriving test samples during the model deployment phase. These methods, assuming the test data class space remains consistent with the training space, can improve the model's adaptability to unknown environments to some extent.

[0005] However, the aforementioned adaptive methods during testing generally rely on the prediction results or statistical characteristics of the test samples as the basis for adaptive updates. When the test data contains unknown class samples that did not appear in the training phase, the model will still force predictions based on the existing class space and use these predictions in the subsequent adaptive update process. Since the distribution of unknown class samples in the feature space differs significantly from that of known classes, this indiscriminate adaptive update is prone to introducing erroneous gradients or statistical information, thus adversely affecting the model parameters.

[0006] Furthermore, existing test-time adaptive methods typically lack mechanisms for evaluating the reliability of test samples, making it difficult to determine whether a particular test sample is suitable for adaptive model updates. In open environments, when the proportion of unknown class samples is high or the class distribution changes drastically, the model will continue to perform adaptive operations, further amplifying the negative impact of erroneous updates. This lack of screening and constraints limits the usability and stability of existing test-time adaptive techniques in real-world, complex scenarios.

[0007] Therefore, existing technologies have the following objective drawbacks: they cannot effectively distinguish between known and unknown class samples during the testing phase, leading to erroneous adaptive updates in open-set testing environments, which in turn affects the overall model performance and system reliability. To address these shortcomings, there is an urgent need for a technical solution that can identify unknown class samples during the testing phase and constrain and optimize the adaptive process based on the identification results, thereby achieving more stable and effective test-time adaptation under open-world conditions. Summary of the Invention

[0008] To address the aforementioned problems in existing technologies, the technical problem this invention aims to solve is: how to achieve adaptive testing in complex target environments based on class relation divergence.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0010] A test-time adaptation method for open set image classification based on class relation divergence includes the following steps:

[0011] S1: Select a publicly available image dataset as the source domain dataset. ,in, express The i-th image sample, This represents the category label of the i-th image sample. express The total number of image samples in the dataset;

[0012] Select a general pre-trained model And a supervised cross-entropy loss function is constructed. right Pre-training to obtain trained and utilize calculate On The category prototypes of the known categories are denoted as . ;

[0013] S2: Select the dataset of test sample images to be classified as the target domain dataset. The The test samples in the document do not contain image category information. The image data of the test samples to be classified were divided into several batches;

[0014] S3: Select any batch and calculate the image of each test sample to be classified in that batch. Class relation divergence ,in Indicates the characteristics of the test samples. Indicates the distribution of target relations. This represents the class prototype most similar to the features of the current test sample image to be classified. Indicates the distribution of source relations;

[0015] Traverse all test sample images to be classified in this batch to obtain the class relationship divergence corresponding to each test sample image to be classified.

[0016] S4: Preset class relation divergence threshold. Compare each class relation divergence obtained in S3 with the class relation divergence threshold. Test sample images with divergence values ​​less than the threshold are classified as known class samples and form a subset of known class samples. ;

[0017] S5: From Select any test sample image to be classified ,in The corresponding test sample features are ;

[0018] calculate and Calculate the cosine similarity score between the prototypes of known categories, and then select the prototype with the highest cosine similarity score. The corresponding category as Category prediction results;

[0019] S6: Utilize renew The category prototype of the belonging category is updated using the following formula:

[0020]

[0021] in, Indicates the update magnitude. When the initial category prototype is ;

[0022] S7: Iterate through the corresponding batches For all test sample images to be classified, repeat steps S5-S6 to obtain the corresponding batch. The class prediction results of all test sample images to be classified, and the updated class prototype corresponding to each class, are used in the class prediction of the next batch of image data;

[0023] S8: Determine if completed The class prediction of all batches of test sample images to be classified is performed. If it is not completed, the process returns to step S3. Otherwise, the class prediction results of all known class samples are output, thus completing the image classification prediction work of this target domain image dataset.

[0024] Preferably, the calculation in S1 The steps for obtaining the category prototypes of several known categories are as follows:

[0025] Constructing a supervised cross-entropy loss function ,Will As input pair Training is performed using gradient descent and backpropagation to train the model. Training stops when the model converges or reaches the maximum number of iterations, yielding a pre-trained model. ; The calculation formula is as follows:

[0026]

[0027] in, Represents conditional probability;

[0028] Will As The input is used to obtain the sample features of each image and its corresponding category;

[0029] The features of samples belonging to the same category are aggregated, and then the category prototype of the corresponding category is calculated. The calculation formula is as follows:

[0030]

[0031] in, This represents the category prototype of the nth category. This represents the sample feature corresponding to the i-th image sample. This indicates an indicator function.

[0032] Preferably, the calculation in S3 Class relation divergence The steps are as follows:

[0033] S3-1: Take any test sample image enter From Corresponding test sample features ;

[0034] S3-2: Calculation Cosine similarity value between each category prototype Obtain the set of cosine similarity values ,in With the Each category prototype The formula for calculating the cosine similarity value between them is as follows:

[0035]

[0036] in, Describes the L2 norm of a vector;

[0037] Will All cosine similarity values ​​in the sample are normalized to obtain the result. Target Relationship Distribution The calculation formula is as follows:

[0038]

[0039] in, Indicates the first One category prototype;

[0040] S3-3: Based on Between each category prototype , will be the largest The category prototype corresponding to the value is used as Corresponding category prototype Using the method described in step S3-2, the following calculations were performed: Source relation distribution ;

[0041] S3-4: Jensen-Shannon divergence calculations regarding Class relation divergence The calculation formula is as follows:

[0042]

[0043] in, express The j-th component, express .

[0044] Compared with the prior art, the present invention has at least the following advantages:

[0045] This invention achieves significant technical results in open-set testing environments by introducing a sample discrimination and adaptive constraint mechanism based on class relation divergence during the testing phase. It proposes the metric "class relation divergence," calculating the class relation divergence for each test sample to quantify sample uncertainty, thereby effectively distinguishing between known and unknown class samples in the test dataset. By utilizing the prototype momentum update method of known class samples during testing, the model can quickly adapt to the test data distribution in a low-cost manner.

[0046] 1. Effectively suppressing erroneous adaptation caused by unknown category samples: During the testing phase, this invention distinguishes between known and unknown categories by constructing a class relation distribution between test samples and known category prototypes, and further calculating the class relation divergence of the samples. Since known category samples maintain a relatively stable class relation structure across different domains, their class relation divergence values ​​are small; while unknown category samples exhibit significant differences at the class relation level, their class relation divergence values ​​are large. Based on this difference, this invention can automatically identify unknown category samples during the testing phase and exclude them from the adaptive update process, thereby effectively avoiding erroneous adaptation caused by unknown category samples.

[0047] 2. Achieving stable test-time adaptation without accessing source domain data: This invention constructs a category prototype during the pre-training phase and uses the category prototype to approximate the source domain relationship distribution during the testing phase, avoiding the need for direct access to source domain data during the testing phase. Simultaneously, the category prototype gradually absorbs the target domain distribution features using momentum updates during the testing phase, enabling the model to complete the adaptation process relying solely on test data. Therefore, this invention achieves stable and effective test-time adaptation while meeting practical deployment constraints.

[0048] 3. Applicable to open-set testing environments, with stronger practical application value: Compared to test-time adaptive methods that only apply to closed-class spaces, this invention can simultaneously handle test data distribution shifts and the presence of unknown classes, making it suitable for more complex open-set testing environments. By achieving coordinated recognition of known classes and adaptive updates during the testing phase, this invention avoids dependence on prior information about the test data, improving the applicability and generalization ability of the method in real-world application scenarios, and better meeting the stability and reliability requirements of actual systems. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0050] Figure 2 This is a schematic diagram of class relation divergence in the method of the present invention.

[0051] Figure 3This paper analyzes the CRD index and its automatic threshold selection mechanism in this method.

[0052] Figure 4 This section analyzes the pseudo-label mechanism based on CRD guidance in this method.

[0053] Figure 5 This section describes the sensitivity analysis of the threshold parameter ρ in this method.

[0054] Figure 6 This is a schematic diagram illustrating the t-SNE visualization effect of the present invention. Detailed Implementation

[0055] The present invention will now be described in further detail.

[0056] See Figures 1-6 A test-time adaptation method for open set image classification based on class relation divergence includes the following steps:

[0057] S1: Select a publicly available image dataset as the source domain dataset. ,in, express The i-th image sample, This represents the category label of the i-th image sample. express The total number of image samples in the dataset;

[0058] Select a general pre-trained model This general pre-trained model uses a general ResNet-18 model pre-trained on ImageNet as the base model and constructs a supervised cross-entropy loss function. right Pre-training to obtain trained and utilize calculate On The category prototypes of the known categories are denoted as . ;

[0059] The calculation in S1 The steps for obtaining the category prototypes of several known categories are as follows:

[0060] Constructing a supervised cross-entropy loss function ,Will As input pair Training is performed using gradient descent and backpropagation to train the model. Training stops when the model converges or reaches the maximum number of iterations, yielding a pre-trained model. ; The calculation formula is as follows:

[0061]

[0062] in, Represents conditional probability;

[0063] Will As The input is used to obtain the sample features of each image and its corresponding category;

[0064] The features of samples belonging to the same category are aggregated, and then the category prototype of the corresponding category is calculated. The calculation formula is as follows:

[0065]

[0066] in, This represents the category prototype of the nth category. This represents the sample feature corresponding to the i-th image sample. This indicates an indicator function.

[0067] S2: Select the dataset of test sample images to be classified as the target domain dataset. The The test samples in the document do not contain image category information. The image data of the test samples to be classified were divided into several batches;

[0068] S3: Select any batch and calculate the image of each test sample to be classified in that batch. Class relation divergence ,in Indicates the characteristics of the test samples. Indicates the distribution of target relations. This represents the class prototype most similar to the features of the current test sample image to be classified. Indicates the distribution of source relations;

[0069] Traverse all test sample images to be classified in this batch to obtain the class relationship divergence corresponding to each test sample image to be classified.

[0070] The calculation in S3 Class relation divergence The steps are as follows:

[0071] S3-1: Take any test sample image enter From Corresponding test sample features ;

[0072] S3-2: Calculation Cosine similarity value between each category prototype Obtain the set of cosine similarity values ,in With the Each category prototype The formula for calculating the cosine similarity value between them is as follows:

[0073]

[0074] in, Describes the L2 norm of a vector;

[0075] Will All cosine similarity values ​​in the sample are normalized to obtain the result. Target Relationship Distribution The calculation formula is as follows:

[0076]

[0077] in, Indicates the first One category prototype;

[0078] S3-3: Based on Between each category prototype , will be the largest The category prototype corresponding to the value is used as Corresponding category prototype Using the method described in step S3-2, the following calculations were performed: Source relation distribution ;

[0079] S3-4: Jensen-Shannon divergence calculations regarding Class relation divergence The calculation formula is as follows:

[0080]

[0081] in, express The j-th component, express The Jensen-Shannon divergence is existing technology.

[0082] S4: Preset class relation divergence threshold. Compare each class relation divergence obtained in S3 with the class relation divergence threshold. Test sample images with divergence values ​​less than the threshold are classified as known class samples and form a subset of known class samples. The class relation divergence threshold used here can be set to a fixed value or a dynamically updated value. A fixed threshold can be used to initially divide the samples. During subsequent testing, the threshold is dynamically adjusted based on the class relation divergence statistics of known and unknown class samples in the processed batch. If a dynamic threshold update mechanism is introduced, the class relation divergence threshold can adapt to changes in the target domain data distribution, thereby improving the stability and accuracy of the division between known and unknown class samples.

[0083] S5: From Select any test sample image to be classified ,in The corresponding test sample features are ;

[0084] calculate and Calculate the cosine similarity score between the prototypes of known categories, and then select the prototype with the highest cosine similarity score. The corresponding category as Category prediction results;

[0085] S6: Utilize renew The category prototype of the belonging category is updated using the following formula:

[0086]

[0087] in, This indicates the update magnitude, typically set to 0.95. When the initial category prototype is ;

[0088] S7: Iterate through the corresponding batches For all test sample images to be classified, repeat steps S5-S6 to obtain the corresponding batch. The class prediction results of all test sample images to be classified, and the updated class prototype corresponding to each class, are used in the class prediction of the next batch of image data;

[0089] S8: Determine if completed The class prediction of all batches of test sample images to be classified is performed. If it is not completed, the process returns to step S3. Otherwise, the class prediction results of all known class samples are output, thus completing the image classification prediction work of this target domain image dataset.

[0090] For the test samples to be classified, the data is divided into several batches. All batches of data are input into the model framework in a streaming manner for prediction, and each batch of data is only classified and predicted once.

[0091] Experimental content and results

[0092] 1. Experimental setup

[0093] Dataset Introduction: To verify the effectiveness of the proposed method, experimental evaluations were conducted on several representative public benchmark datasets.

[0094] The PACS dataset consists of 9,991 images from four different domains: artistic images, cartoon images, real photographs, and sketches. The first four categories are classified as the known categories, and the remaining categories as the unknown categories, ordered alphabetically by their names.

[0095] The Office-Home dataset, derived from office and home environments, contains approximately 15,500 images across 65 categories, covering four domains: artistic images, clipart images, product images, and real-world scene images. In the experiment, the first 15 categories were designated as known categories, and the remaining categories were designated as unknown categories, following alphabetical order of their names.

[0096] The Office-31 dataset contains 31 categories from three domains: DSLR images, Amazon product images, and Webcam images. Ten categories that overlap with the Caltech-256 dataset are considered known categories, and the last 11 categories, in alphabetical order, are considered unknown categories.

[0097] The Digits dataset is a multi-domain handwritten digit dataset containing five subdomains: MNIST, MNIST-M, SVHN, USPS, and SYN. ​​In the experiments, MNIST was used as the source domain, the remaining subdomains as the target domains, and the first five digit categories were set as known categories.

[0098] The specific division of known and unknown categories for each dataset is summarized in Table 1.

[0099] Table 1 Tag Space Division

[0100]

[0101] Implementation details: During the source model training phase, only a single source domain is used for training. For each source domain, a source model with a memory is constructed, whereby the memory stores the feature prototypes corresponding to each category to characterize the category distribution characteristics.

[0102] The Adam optimizer was used during training, with a learning rate set to 5×10⁻5 The batch size was set to 32. For the PACS, Office-Home, and Office-31 datasets, a ResNet-18 network pre-trained on the ImageNet dataset was selected as the backbone network, and a batch normalization structure was introduced; for the Digits dataset, the LeNet network structure was used.

[0103] During the adaptive testing phase, the source model was adapted to each of the remaining target domains, and the experimental results were the average of those for each target domain. For the Office-31, PACS, and Digits datasets, since the number of known and unknown categories was relatively close, an automatic method was used to determine the CRD threshold. Specifically, the threshold was automatically calculated in each batch based on the average CRD of known and unknown samples. For the Office-Home dataset, since the number of unknown categories was significantly greater than the number of known categories, the threshold parameter ρ was set to −0.5 in the experiments. Experimental results show that this method is not sensitive to the selection of the threshold parameter, but when the difference between the number of known and unknown categories is large, using a specific threshold can achieve better results. All experiments were performed on an RTX-A4000 GPU based on the PyTorch framework.

[0104] Evaluation Metrics and Comparison Methods: The experiment used the H-score as the primary evaluation metric to comprehensively measure the recognition performance of known and unknown categories. The H-score was calculated as the harmonic mean of the accuracy rates for known and unknown categories, ensuring that the recognition performance of unknown categories was given equal importance in the overall evaluation. The experiments also reported the accuracy rates for known and unknown categories separately.

[0105] The experiment mainly compares the proposed method with the following representative methods:

[0106] One type is the Open Set Domain Adaptive (OSDA) method, which explicitly considers the existence of unknown categories in the target domain under the assumption that the source and target domain data can be accessed simultaneously.

[0107] One type is the Test-Time Adaptive (TTA) method, which adjusts the model during the testing phase, but usually assumes that the source and target domains have the same label space;

[0108] Another type is the Domain Generalization (DG) method, which improves the model's generalization ability during the training phase to cope with unknown distributions. Some of these methods further consider the impact of potential unknown categories.

[0109] Since the adaptive method itself cannot directly handle samples of unknown categories during testing, the output entropy was used as a threshold to identify unknown samples in the experiment. Different methods have some differences in experimental settings, which are detailed in the corresponding tables. Some methods also used test data for model adjustment during the testing phase.

[0110] 2. Comparison with the current best method

[0111] Experimental results on multiple benchmark datasets show that the proposed method achieves relatively stable performance improvements under different settings such as open set domain adaptation, test-time adaptation, and domain generalization.

[0112] Compared to existing methods, this proposed method demonstrates consistent performance superiority across various benchmark datasets, validating its effectiveness. Furthermore, for methods addressing only a single challenge, the proposed method achieves better results on all datasets, indicating the significant advantage of simultaneously tackling multiple challenges within a unified framework.

[0113] Furthermore, in comparison with domain generalization methods that consider potential unknown categories, the proposed method still shows strong competitiveness in the identification of unknown categories. This is because the method can adaptively adjust the target distribution during the testing phase, while traditional domain generalization methods usually cannot make corresponding updates during the testing phase.

[0114] Table 2 shows the performance of this method on multiple datasets. Accuracy for known classes Accuracy for unknown classes Refers to H-score.

[0115] Table 2 Main Experimental Results

[0116]

[0117] 3. Ablation experiment

[0118] To further analyze the role of each component module in the proposed method, a series of ablation experiments were conducted.

[0119] First, the CRD index and its automatic threshold selection mechanism were analyzed. For example... Figure 3As shown, from left to right, the tasks represent the Webcam to Amazon migration task from the Office-31 dataset, the Sketch to Art migration task from the PACS dataset, and the MNIST to USPS migration task from the Digits dataset. By analyzing the changes in CRD during testing, it can be observed that there is a significant difference in CRD values ​​between known and unknown class samples. Generally, the average CRD value for known class samples tends to be stable and negative, while unknown class samples show the opposite trend. This characteristic allows CRD to effectively distinguish between categories within a relatively wide threshold range. Although CRD values ​​may fluctuate in the early stages of testing, they gradually converge as the number of samples increases. Furthermore, due to differences in the degree of distribution shift across different tasks, the mean CRD values ​​for known and unknown categories will also differ.

[0120] Based on this characteristic, the designed automatic threshold generation process can obtain a reasonable threshold without manual parameter tuning, thereby achieving stable and clear separation of known and unknown samples during the testing process.

[0121] Secondly, the pseudo-tag mechanism based on CRD guidance was analyzed. For example... Figure 4 In this mechanism, only samples with low CRD values ​​are used to update the category prototype. From left to right, the examples represent the Webcam to Amazon migration task on the Office-31 dataset, the Sketch to Art migration task on the PACS dataset, and the MNIST to USPS migration task on the Digits dataset. Comparison of experimental results under different filtering ratios reveals that CRD-guided sample selection significantly improves the accuracy of pseudo-labels. In relatively simple tasks, pseudo-label accuracy can approach perfect accuracy; however, in more challenging scenarios, the selection mechanism can significantly improve pseudo-label accuracy from a low level. Without sample selection, erroneous pseudo-labels accumulate during testing, impacting model performance.

[0122] Regarding the selection of the threshold parameter ρ, such as Figure 5This paper analyzes the accuracy of known sample identification, the accuracy of unknown sample identification, and the overall H-score as a function of a threshold. From left to right, the examples represent the Webcam to Amazon migration task on the Office-31 dataset, the Sketch to Art migration task on the PACS dataset, and the MNIST to USPS migration task on the Digits dataset. As the threshold increases, more samples are classified into the known category, potentially including some unknown samples, leading to a decrease in the accuracy of the known category. Simultaneously, the accuracy of the unknown category improves due to the enhanced filtering mechanism. The overall H-score shows relatively small variation with the threshold, indicating strong robustness of the proposed method to the threshold parameter. Furthermore, this method automatically separates known and unknown samples, reducing the need for manual parameter tuning and facilitating practical applications.

[0123] Further quantitative results show that updating the prototype using only a subset of high-confidence samples during the testing phase can improve the H-score on multiple datasets; conversely, completely removing the screening mechanism leads to a decline in overall performance, thus validating the necessity of the CRD-guided screening strategy.

[0124] Table 3. Accuracy Analysis under Different Update Strategies

[0125]

[0126] 4. Visual Analysis

[0127] To visually analyze the changes in prototypes during testing, the t-SNE method was used to visualize the target domain features and category prototypes. The results show that after time-of-test adaptation, each category prototype shifted significantly from its initial state. This shift, while maintaining consistency with the corresponding known category samples, effectively increased the distance between the prototypes and the unknown category samples, thus validating the effectiveness of the proposed index in facilitating the discrimination between known and unknown samples and time-of-test adaptation.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A test-time adaptive method for open set image classification based on class relation divergence, characterized in that: Includes the following steps: S1: Select a publicly available image dataset as the source domain dataset. ,in, express The i-th image sample, This represents the category label of the i-th image sample. express The total number of image samples in the dataset; Select a general pre-trained model And a supervised cross-entropy loss function is constructed. right Pre-training to obtain trained and utilize calculate On The category prototypes of the known categories are denoted as . ; S2: Select the dataset of test sample images to be classified as the target domain dataset. The The test samples in the document do not contain image category information. The image data of the test samples to be classified were divided into several batches; S3: Select any batch and calculate the image of each test sample to be classified in that batch. Class relation divergence ,in Indicates the characteristics of the test samples. Indicates the distribution of target relations. This represents the class prototype most similar to the features of the current test sample image to be classified. Indicates the distribution of source relations; Traverse all test sample images to be classified in this batch to obtain the class relationship divergence corresponding to each test sample image to be classified. S4: Preset class relation divergence threshold. Compare each class relation divergence obtained in S3 with the class relation divergence threshold. Test sample images with divergence values ​​less than the threshold are classified as known class samples and form a subset of known class samples. ; S5: From Select any test sample image to be classified ,in The corresponding test sample features are ; calculate and Calculate the cosine similarity score between the prototypes of known categories, and then select the prototype with the highest cosine similarity score. The corresponding category as Category prediction results; S6: Utilize renew The category prototype of the belonging category is updated using the following formula: in, Indicates the update magnitude. When the initial category prototype is ; S7: Iterate through the corresponding batches For all test sample images to be classified, repeat steps S5-S6 to obtain the corresponding batch. The class prediction results of all test sample images to be classified, and the updated class prototype corresponding to each class, are used in the class prediction of the next batch of image data; S8: Determine if completed The class prediction of all batches of test sample images to be classified is performed. If it is not completed, the process returns to step S3. Otherwise, the class prediction results of all known class samples are output, thus completing the image classification prediction work of this target domain image dataset.

2. The test-time adaptation method for open set image classification based on class relation divergence as described in claim 1, characterized in that: The calculation in S1 The steps for obtaining the category prototypes of several known categories are as follows: Constructing a supervised cross-entropy loss function ,Will As input pair Training is performed using gradient descent and backpropagation to train the model. Training stops when the model converges or reaches the maximum number of iterations, yielding a pre-trained model. ; The calculation formula is as follows: in, Represents conditional probability; Will As The input is used to obtain the sample features of each image and its corresponding category; The features of samples belonging to the same category are aggregated, and then the category prototype of the corresponding category is calculated. The calculation formula is as follows: in, This represents the category prototype of the nth category. This represents the sample feature corresponding to the i-th image sample. This indicates an indicator function.

3. The test-time adaptation method for open set image classification based on class relation divergence as described in claim 2, characterized in that: The calculation in S3 Class relation divergence The steps are as follows: S3-1: Take any test sample image enter From Corresponding test sample features ; S3-2: Calculation Cosine similarity value between each category prototype Obtain the set of cosine similarity values ,in With the Each category prototype The formula for calculating the cosine similarity value between them is as follows: in, Describes the L2 norm of a vector; Will All cosine similarity values ​​in the sample are normalized to obtain the result. Target Relationship Distribution The calculation formula is as follows: in, Indicates the first One category prototype; S3-3: Based on Between each category prototype , will be the largest The category prototype corresponding to the value is used as Corresponding category prototype Using the method described in step S3-2, the following calculations were performed: Source relation distribution ; S3-4: Jensen-Shannon divergence calculations regarding Class relation divergence The calculation formula is as follows: in, express The j-th component, express .

Images