Self-adaption method during testing based on category balance and pseudo tag guidance

Abstract

The invention discloses a self-adaption method during testing based on category balance and pseudo tag guidance. The method comprises the following steps: step 1, constructing and dynamically updating a feature memory library based on category balance; 2, performing neighborhood retrieval and preliminary classification on the target domain sample through a KNN non-parameter classifier to obtain a non-parameter classification result; 3, performing global constraint and correction on a non-parameter classification result by introducing linear pseudo-label information obtained in a source domain supervision training stage, reducing interference of wrong pseudo-labels and local noise samples on a reasoning result, and reversely regulating and controlling an updating process of a memory bank according to a fusion reasoning result and a confidence coefficient thereof so as to obtain a non-parameter classification result; and dynamic balance of memory bank category distribution is realized. According to the method, the adaptation relation between the source domain knowledge and the target domain data is constructed, the cross-domain classification process is dynamically adjusted, and the problems that when an existing classification model faces the problems of domain offset and class imbalance, the classification performance is remarkably reduced, and the generalization ability is insufficient are effectively solved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision classification technology, specifically relating to a test-time adaptive method based on class balancing and pseudo-label guidance. Background Technology

[0002] Classification is one of the core tasks in computer vision, aiming to categorize data into predefined classes based on their characteristics. It is widely used in fields such as medical imaging, security monitoring, autonomous driving, industrial inspection, and intelligent recognition. In practical applications, models are typically trained on specific source domain datasets. However, when transferred to a new target domain dataset, various variations during data acquisition (such as differences in parameters between different brands of equipment, changes in lighting / angle in the shooting environment, regional / industry differences in data acquisition standards, and differences in the distribution of sample groups) can lead to significant domain shifts between the source and target domains.

[0003] Domain shift can render a model's feature extraction capabilities ineffective, making it unable to accurately adapt to the distribution characteristics of the target domain data, thus leading to a decrease in classification accuracy. Furthermore, data in various scenarios often suffer from severe class imbalance (e.g., in medical data, benign samples far outnumber malignant samples; in industrial inspection, the proportion of qualified products far exceeds that of defective products; in security monitoring, normal scene samples outnumber abnormal scene samples, etc.). Traditional classification models tend to favor the majority class in such situations, resulting in excessively low accuracy in identifying the minority class, making it difficult to meet the needs of practical applications.

[0004] Test-time adaptation (TTA) is a key solution to the domain offset problem, aiming to adapt the model to the target domain through adaptive adjustments during the testing phase without requiring additional labeled data for the target domain. Non-parametric classifiers have gained widespread attention in test-time adaptation tasks due to their low computational cost and the fact that they do not require retraining model parameters, effectively avoiding the risk of overfitting caused by parameter adjustments.

[0005] The paper "AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation" proposes a test-time adaptation method based on a non-parametric classifier. During the training phase, this method constructs a source domain memory to store feature-label pairs. During the testing phase, for each target sample, a K-Nearest Neighbor (KNN) classifier is used to retrieve the most similar sample from the memory bank, and classification is performed based on feature similarity. This achieves online domain adaptation without backpropagation, resulting in good performance on natural image tasks.

[0006] However, the shortcomings of this method are: first, it does not make full use of the classifier capabilities learned by the model during the source domain supervised training phase, relying only on the feature similarity matching of the KNN classifier, resulting in insufficient utilization of source domain knowledge; second, it does not consider the class imbalance problem that is common in real-world scenarios. When the majority class samples account for too high a proportion in the memory bank, the classification results will be biased towards the majority class, which will seriously affect the recognition accuracy of the minority class and reduce the practical value of the model.

[0007] While existing test-time adaptive classification techniques have alleviated the domain offset problem to some extent, they still face the following key challenges: Insufficient utilization of source domain knowledge: Existing nonparametric adaptive methods rely solely on feature similarity for classification, without combining the classifier output learned by the model during supervised training. This fails to fully exploit the effective supervisory information in the source domain data, resulting in limited accuracy in cross-domain classification. Poor adaptability to class imbalance: In the face of class imbalance data that is common in various scenarios, existing methods have not designed a targeted balancing mechanism. The excessive accumulation of majority class samples in the memory bank will lead to the classification results being biased towards the majority class, and the recognition rate of minority class samples is too low, which cannot meet the needs of accurate recognition of each class in practical applications. Limited generalization ability: Existing methods show a significant decline in classification performance in scenarios with complex domain shifts (such as multi-source domains and large distribution differences) and severe class imbalance, making it difficult to adapt to the actual application needs of different fields. Summary of the Invention

[0008] To overcome the shortcomings of the existing technologies, the present invention aims to provide a test-time adaptive method (PLATA) based on class balancing and pseudo-label guidance. By using pseudo-label guidance and a memory sample balancing strategy, the method constructs an adaptation relationship between source domain knowledge and target domain data, dynamically adjusts the cross-domain classification process, and effectively solves the problems of significant decline in classification performance and insufficient generalization ability of existing classification models when facing domain shift (such as differences in data acquisition equipment, changes in environmental conditions, differences in data distribution, and different acquisition specifications) and class imbalance.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A test-time adaptive method based on class balancing and pseudo-label guidance includes the following steps; Step 1: Construct and dynamically update a class-balanced feature memory to provide a foundation for the KNN nonparametric classification in Step 2; The feature memory is defined as: a reference data set that adopts a fixed-length queue structure and consists of several "L2-normalized feature vectors + category labels" to maintain a balanced category distribution. Its mathematical expression includes feature matrices and label vectors.

[0010] As the core reference for KNN nonparametric classification, its class balance characteristic ensures the accuracy of neighborhood retrieval. The dynamic update process continuously adapts to the target domain distribution through reverse regulation guided by pseudo-labels, reduces noise interference in cross-domain classification, and improves the adaptability and inference accuracy of classification tasks.

[0011] Step 2: Based on the feature memory, the target domain samples are searched and preliminarily classified using the KNN nonparametric classifier to obtain nonparametric classification results, providing local structural information for the fusion inference in Step 3; Step 3: By introducing the linear pseudo-label information obtained in the source domain supervised training stage, the nonparametric classification results in Step 2 are globally constrained and corrected, effectively reducing the interference of erroneous pseudo-labels and local noise samples on the inference results. Based on the fused inference results and their confidence, the update process of the memory bank in Step 1 is adjusted in reverse, thereby achieving a dynamic balance of the category distribution of the memory bank.

[0012] Ultimately, the model achieves stable adaptation to the target domain data distribution shift through the fusion of nonparametric inference and source domain discriminative information without requiring backpropagation and target domain labels during the training phase. This effectively alleviates the class imbalance problem in cross-domain scenarios, thereby improving the model's prediction accuracy and robustness during the testing phase.

[0013] Steps 1 through 3 described above are not independent or simply parallel, but rather constitute an interdependent and mutually constraining test-time adaptive closed-loop system. Through these interactions, the three steps form a continuously evolving adaptive closed loop, enabling the model to gradually adapt to changes in the target domain distribution and suppress decision bias caused by class imbalance during the testing phase.

[0014] In the update mechanism of step 1, when the number of samples in a certain category reaches the preset upper limit, new samples are no longer written unconditionally. Instead, the earliest written sample from the target domain in that category is replaced first. This ensures the upper limit of the number of categories while introducing more representative recent samples, thus avoiding long-term solidification of the memory distribution.

[0015] Step 1 specifically involves: Step 1): Initialization: Before starting the target domain test, the feature memory needs to be initialized using the source domain data; The feature memory adopts a fixed-length queue structure, consisting of several "feature vectors + labels"; each feature vector is L2 normalized to ensure the stability of cosine similarity calculation. Step 2): Queue update mechanism: During the testing phase, the target domain data is tested step by step in mini-batches, and the feature memory is updated once each batch of testing is completed. In step 1): The memory storage method is as follows: Feature matrix: memory∈R^{Q×D}; Label vector: memory_label∈R^{Q}.

[0016] Q: Memory length, which represents the total number of samples currently stored in the feature memory, can be a fixed value or change dynamically during the testing phase; D: Feature dimension, representing the dimension of the feature vector output by the feature extraction network; ,in This represents the feature vector corresponding to the i-th sample; ,in Representation and eigenvectors The corresponding category label.

[0017] The feature matrix and the label vector together constitute an adaptive feature memory for KNN nonparametric classification and testing. During the initialization phase, the model that has completed supervised training in the source domain is used to perform forward inference on the training domain data.

[0018] Specifically, the source domain samples are sequentially input into the feature extraction network to extract the corresponding feature vectors, and the feature vectors are normalized to eliminate the influence of feature scale differences between different samples; at the same time, the classifier trained in the model is used to predict the source domain samples to obtain the true class label or supervision label corresponding to each sample. Subsequently, the normalized feature vectors and their corresponding category labels are stored as a set of "feature-label" pairs in a preset order in the feature memory, gradually constructing the initial feature matrix and label vectors. When the memory capacity reaches the preset length limit, writing stops, thus forming the initial memory structure used for KNN nonparametric classification and adaptive inference in the testing phase; After the above initialization is completed, the feature memory is maintained and continuously used as the basic reference set in the testing phase. After entering the testing phase, the memory no longer remains in a static state, but is updated in a controlled manner through a subsequent queue update mechanism without destroying the initial semantic structure, so as to gradually introduce target domain distribution information and enhance cross-domain adaptive capabilities.

[0019] The initialization phase and the queue update phase together constitute the complete lifecycle management mechanism of the feature memory. The initialization phase is used to build a stable memory base with source domain discrimination semantics, while the queue update phase is used to introduce target domain distribution information and perform controlled evolution of the memory content during the testing process. The two phases are interconnected and complementary.

[0020] Step 2) specifically refers to: To avoid low-quality pseudo-label samples from polluting the feature memory, this invention performs queue update judgment on the target domain samples sequentially within each test mini-batch; Specifically, for each input sample in the target domain, prediction is performed using both a KNN nonparametric classifier based on a feature memory and a linear classifier trained under supervision in the source domain, and the maximum class prediction probability corresponding to the two classifiers is calculated. and ; Compare the predicted confidence level with a preset confidence threshold. and A comparison is made if and only if both of the following conditions are met: The target domain sample is determined to be a highly reliable sample, and its corresponding feature vector and predicted class label are allowed to enter the feature memory. If any confidence level does not reach the threshold requirement, the sample is discarded and will not participate in the memory update.

[0021] For samples that meet the write conditions, before performing the write operation, the number of samples of that category already stored in the current memory is first counted according to their predicted category. If the number does not exceed the maximum capacity max_per_class preset for that category, the normalized feature vector and predicted label of the sample are written to the memory in queue order. If the number of samples of that category has reached the maximum capacity, the write operation is not performed directly, but the storage queue corresponding to that category is scheduled and controlled according to the preset update strategy.

[0022] The update strategy includes: suspending the writing of the current sample; or replacing existing samples in the category that have been stored for a long time and have low confidence, thereby introducing more representative feature information while keeping the total number of samples in the category unchanged.

[0023] Through the aforementioned queue update mechanism, the feature memory only introduces target domain samples with high prediction consistency and controlled class distribution during the testing phase. This effectively suppresses the interference of noisy samples and majority class samples on the memory structure while gradually adapting to the target domain distribution, ensuring the stability and reliability of the subsequent KNN nonparametric classification process.

[0024] Step 2 specifically involves: Step 2 performs nonparametric classification inference based on the feature memory built and dynamically maintained in Step 1, which is used to determine the category of the target domain samples during the testing phase. Specifically, for the target domain samples to be classified, the corresponding feature vectors are first extracted using a feature extraction network. The feature vectors are then subjected to L2 normalization to ensure that their similarity calculations have a consistent metric with those of the feature vectors already stored in the feature memory.

[0025] The feature matrix of the feature memory maintained in step 1 is: Its i-th row vector This represents the feature vector corresponding to the i-th sample; the corresponding label vector is: where the i-th element Representation and eigenvectors The corresponding category labels. Each feature vector in the feature matrix has been normalized before being written into the memory. During the test inference process, the feature vector z to be classified is calculated and compared with the feature matrix in the feature memory. The cosine similarity between each row vector yields a similarity vector: Subsequently, the similarity vectors are sorted according to the similarity scores, and the k most similar sample indices are selected to form a nearest neighbor set. Read the corresponding category label from the nearest neighbor set. The similarity scores of nearest neighbor samples belonging to the same category are summed to obtain scores for each category: in, This is an indicator function; it takes the value 1 when the category label of the i-th nearest neighbor sample is equal to category c, and otherwise takes the value 0. Based on the above calculations, an unnormalized category score vector is obtained. Where C represents the total number of categories. The score vector is determined entirely by the feature memory and similarity calculation results constructed in step 1, without relying on any learnable parameters. It is directly used as a non-parametric classification output and is used for fusion inference with the output results of other classifiers (step 3) without the need for additional training or parameter update processes.

[0026] Step 3 specifically involves: This invention proposes a linear pseudo-label-guided fusion inference method for stable and reliable cross-domain classification of target domain samples during the testing phase without backpropagation or access to source domain data. This fully integrates the global discriminant information obtained during the source domain supervised training phase with the local structural information of the target domain samples in the feature space.

[0027] In this step, the linear pseudo-labels are generated by a linear classifier obtained during the source domain supervised training phase. This linear classifier, serving as the classification head of the feature extraction network, is trained under the supervision of the true labels of the source domain samples. Its network parameters are frozen during the testing phase and used only for forward inference on the target domain samples. For the input target domain samples... The linear classifier outputs the corresponding class prediction probability distribution. And the category with the highest probability is used as the linear pseudo-label. At the same time, the maximum probability value is used as the prediction confidence level to characterize the degree of consistency between the target domain sample and the source domain discrimination boundary; Meanwhile, based on the feature memory library constructed and dynamically updated in step 1, the nonparametric KNN classification method described in step 2 is used to determine the neighborhood of the target domain samples.

[0028] Specifically, based on the target sample features and the feature matrix stored in the feature memory. Similarity calculations are performed to obtain the unnormalized class score vector of KNN based on local feature similarity. The score vector is determined entirely by the feature memory and similarity calculation results constructed in step 1; when confidence comparison or fusion with linear classifier results is required, the score vector is... Normalization is performed to obtain the corresponding KNN class probability distribution. The category with the highest probability is used as the pseudo-label for KNN classification. The KNN classification results mentioned above mainly reflect the local consistency structure of the target sample in the feature space of the target domain, but they are easily affected by noisy samples and class imbalance in cross-domain scenarios. When using a smaller neighborhood size While the KNN classifier can maintain sensitivity to the local discriminative structure of the target sample, a small number of outlier neighbors may significantly interfere with the voting results. Conversely, if a large neighborhood size is directly used, the KNN classification results will be more biased towards reflecting the overall distribution characteristics of the memory bank, easily leading to oversmoothing and weakening the discriminative ability near the class boundaries. Based on these trade-offs, this invention introduces linear pseudo-labels as consistency constraint signals during the fusion inference stage to guide the nonparametric decision-making process.

[0029] Specifically, when the linear classifier and the KNN classifier predict the same category for the target domain samples, that is, when the condition is met... The target sample is determined to be a high-confidence sample, and the supervision weights are defined. KNN voting weights The predicted probabilities of the two classifiers are weighted and fused to obtain the fused prediction result: Among them, weight and Used to balance the contribution ratio of source domain supervision information and target domain local structure information in the final prediction; When the predictions of the two classifiers are inconsistent, i.e.: This invention does not directly use linear pseudo-labels as the final decision basis, but instead re-evaluates nonparametric decisions by adaptively expanding the KNN neighborhood range.

[0030] Specifically, the number of neighbors will be changed from Expand to Recalculate the KNN classification probability distribution based on a larger neighborhood. Furthermore, the confidence scores of KNN predictions at the original and expanded neighborhood sizes are compared based on the maximum class probability value, and the prediction with higher confidence is selected as the final output. Through the above mechanism, linear pseudo-labels do not directly replace nonparametric classification results, but indirectly guide the selection of KNN classification results through consistency constraints and confidence screening. This enables the nonparametric inference process in the testing phase to fully inherit the discriminative information learned in the source domain supervised training phase while maintaining the ability to adapt to the feature distribution of the target domain. Thus, stable and robust classification of cross-domain samples can be achieved without additional training or backpropagation.

[0031] The beneficial effects of this invention are: This invention introduces a linear classifier obtained during the source domain supervised training phase into the fusion inference process during the testing phase. Utilizing the global discriminative boundary information learned by this classifier under the supervision of real labels, the KNN nonparametric classification results based on feature similarity are guided and constrained. Specifically, in step 3, the linear classifier performs forward inference on the target domain samples under parameter-frozen conditions, generating linear pseudo-labels with confidence assessments to characterize the discriminative consistency of the target samples in the source domain discriminative space.

[0032] By comparing the linear pseudo-labels with the KNN classification results based on the feature memory, this invention can effectively identify KNN misclassifications caused by local noise samples or unstable neighborhood structures. When the two classification results are consistent, it indicates that the target sample simultaneously satisfies the global discrimination boundary and local feature similarity constraints. In this case, by weighted fusion of the linear classifier output and the KNN inference results, stable global discrimination information can be introduced while maintaining local discrimination capability. When the two classification results are inconsistent, the direct use of linear pseudo-labels or KNN results with fixed neighborhoods is avoided. Instead, a neighborhood adaptive re-evaluation mechanism is triggered to alleviate the sensitivity of KNN to local noise samples and the problem of excessive smoothing of decisions under large neighborhoods by adjusting the neighborhood size.

[0033] Therefore, this invention effectively corrects the KNN nonparametric classification process by using linear pseudo-label guidance, so that it neither relies solely on local feature similarity nor introduces additional model updates. This improves the stability, accuracy, and robustness of classification decisions in the cross-domain testing phase without the need for backpropagation.

[0034] A memory bank sample balancing strategy is designed to mitigate the impact of class imbalance on classification results, improve the recognition accuracy of minority class samples, and ensure balanced classification performance across all classes. This is specifically reflected in the feature memory bank construction and update mechanism in step 1 and the fusion inference and sample selection process in step 3. In the memory bank writing stage of step 1, this invention sets maximum storage capacity constraints for different classes, i.e., setting a maximum sample quantity limit for each class. When the number of samples of a certain class in the memory bank reaches a preset threshold, newly arriving samples of the same class will no longer be written or will replace existing samples according to preset rules, thus preventing the memory bank from being continuously occupied by a few high-frequency classes, leading to a severe imbalance in class distribution. In step 3, the fusion inference results are used not only to output the final classification results but also to drive the memory bank update decision. Through the consistency constraint between linear pseudo-labels and KNN inference results, samples with consistent predictions and high confidence are preferentially written into the storage queue of the corresponding class, so that the quantity and quality of samples of different classes in the memory bank are both under dual control. Therefore, the memory bank can gradually form a more balanced class distribution structure during dynamic updates.

[0035] It retains the core advantage of adaptive nonparametric training during testing of nonparametric methods, without the need for additional labeled data and parameter fine-tuning during the testing phase. While slightly increasing the static parameters of the model, it avoids a significant increase in computational overhead, achieves efficient adaptation to different domain offset scenarios, and expands the application scope of classification models. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation

[0037] The present invention will now be described in further detail with reference to the accompanying drawings.

[0038] This invention addresses the shortcomings of existing adaptive classification methods during testing in terms of accuracy and class balance under cross-domain distribution shifts, such as... Figure 1 As shown, this paper proposes a test-time adaptive method based on class balancing and pseudo-label guidance. The input is an unlabeled or poorly labeled target domain image, and the core task is cross-domain image classification (focusing on the class determination problem in the target domain scene). The implementation logic of its adaptive mechanism is as follows: After model training, the inference phase achieves adaptive optimization through two core steps: First, dynamically updating the feature memory to continuously store feature information of the target domain image, providing real-time data support for cross-domain feature matching; second, using a linear classifier to generate reliable pseudo-labels for the target domain image, using these pseudo-labels as supervision signals to guide the neighborhood voting classification process of KNN, avoiding similarity measurement bias caused by domain shift in KNN, and ultimately achieving deep fusion of feature memory information and supervision information, allowing the model to adaptively adapt to the feature distribution characteristics of the target domain. Through the above adaptive design, this method significantly improves the model's core performance in the target domain: First, it enhances domain adaptability, effectively alleviating the "domain shift" problem in cross-domain scenes; second, it improves classification accuracy, significantly increasing the class determination accuracy of the target domain image; and third, it ensures prediction stability, reducing the impact of noise interference or distribution fluctuations on the classification results, making the model output more consistent.

[0039] The following detailed description, with reference to specific embodiments, illustrates the test-time adaptive method based on class balancing and linear pseudo-label guidance proposed in this invention. This embodiment uses the commonly used cross-domain image classification dataset PACS as an example, where images of different artistic styles or shooting conditions are treated as different domains to simulate data distribution shifts caused by changes in the acquisition environment in real-world applications.

[0040] Step 1: Feature memory based on class balance: 1) Initialization In this embodiment, the feature extraction network and its linear classifier are first trained in a supervised manner on the source domain data. Taking the PACS dataset as an example, this dataset consists of data from four different domains, and the "P", "A", and "C" domains are selected as the source domains. The ResNet50, the most commonly used model in this field, is selected as the base model, and the ResNet50 feature extraction network and its linear classifier are jointly trained, enabling the model to learn the discriminative feature representations shared between different source domains. After training, the parameters of the feature extraction network and the linear classifier are frozen to avoid parameter drift during the testing phase. Subsequently, the trained and parameter-frozen model is used to perform forward inference on the source domain samples, sequentially extracting the feature vector corresponding to each source domain sample. The feature vectors are then L2 normalized to eliminate the impact of feature scale differences on subsequent similarity calculations. Simultaneously, the true class label corresponding to each sample is obtained. The normalized feature vectors and their corresponding class labels are treated as a "feature-label" pair and written into the feature memory in a preset order, thereby constructing the initial feature matrix and label vector.

[0041] By constructing a feature memory bank using source domain supervised training samples during the initialization phase, a stable and reliable initial feature distribution foundation is provided for nonparametric KNN inference during the testing phase.

[0042] 2) Queue update mechanism After entering the testing phase, the domains not involved in training are used as target domains for testing. Taking the PACS dataset as an example, the "S" domain is selected as the target domain, and the test samples are input for inference in batches of 8 samples each. After each test batch is completed, the confidence and consistency of the target domain samples are judged based on the fusion inference results described in step 3.

[0043] A sample is considered a high-confidence sample and added to the feature memory as a candidate for writing only if it simultaneously meets the dual-confidence screening criteria of both the linear classifier and the KNN nonparametric classifier. Before performing the write operation, the number of samples of the predicted class currently stored in the feature memory is counted based on the sample's predicted class, and a maximum capacity threshold max_per_class is set for each class.

[0044] When the number of samples of a certain category in the memory bank does not exceed the threshold, the normalized feature vector and predicted label of the sample are written into the memory bank in queue order; when the number of samples of the category reaches or exceeds the threshold, the earliest non-source domain sample written in the category is deleted to free up storage space, thereby ensuring that the newly introduced target domain features can gradually update the memory bank structure.

[0045] This queue update mechanism avoids low-quality samples from contaminating the feature memory through dual-confidence screening, and prevents excessive accumulation of samples from a single class by limiting the maximum capacity by class and prioritizing the elimination of old non-source domain samples, thereby mitigating the impact of class imbalance on KNN inference results. This allows the feature memory to gradually absorb feature information from target domain samples while maintaining capacity and class balance, achieving adaptive behavior during testing.

[0046] Step 2: KNN nonparametric classifier: For each target domain sample input during the testing phase, a feature extraction network with frozen parameters is used to extract its feature representation, and the feature vector is then L2 normalized. Subsequently, the normalized target domain feature vector is compared with the feature matrix in the feature memory constructed and dynamically updated in step 1 using cosine similarity calculation to measure the similarity between the target sample and each sample in the memory in the feature space. Based on the similarity calculation results, the samples in the memory are sorted according to their similarity, and the sample with the highest similarity is selected. Each sample is used as the nearest neighbor set of the target sample, where the number of neighbors is... For preset parameters (e.g.) Based on the category labels corresponding to the nearest neighbor samples, the similarity of neighbor samples belonging to the same category is accumulated to obtain an unnormalized category score vector. The score vector is determined entirely by the feature memory and similarity calculation results constructed in step 1, without introducing any learnable parameters or relying on the backpropagation process. In the subsequent fusion inference stage, the score vector is only modified when confidence comparisons or fusion with linear classifier results are required. Normalization is performed. By introducing a nonparametric KNN classifier based on a feature memory, the classification decision during the testing phase can directly reflect the local neighborhood structure of the target domain samples in the current feature space, thereby enhancing the model's ability to perceive changes in the distribution of the target domain.

[0047] Step 3: Fusion Inference Guided by Linear Pseudo-Labels: For the score vector provided in step 2 Normalization yields The category with the highest probability is used as the KNN classification result. While performing step 2, the linear classifier obtained during the source domain supervised training phase is used to perform forward inference on the sample to obtain a linear probability distribution. and linear pseudo-labels The maximum probability value is used as the prediction confidence of the linear classifier to characterize the consistency between the target sample and the source domain discrimination boundary.

[0048] Subsequently, the consistency between the linear classifier prediction results and the KNN classification results obtained in step 2 based on the feature memory is judged. and Inconsistency indicates a high probability of erroneous pseudo-labels and local noise. In this case, instead of directly using linear pseudo-labels as the final decision-making basis, a neighborhood adaptive re-evaluation mechanism is triggered. The number of neighbors used in step 2 is adjusted from... Expand to The new class probability distribution is obtained by recalculating. By adaptively expanding the KNN neighborhood, the nonparametric decision is re-evaluated, allowing the classification results to reference more the overall distribution information in the feature space, thereby effectively reducing the interference of erroneous pseudo-labels and local noise samples on the inference results; when and When consistent, direct weighted fusion yields the fusion inference result. Subsequently, the KNN outputs at the original and expanded neighborhood sizes are compared based on the confidence level of the largest class probability value. The one with the higher prediction confidence is selected as the final classification result for the target domain sample. The overall summary is as follows: The final fusion inference result is not only used to output the category labels of the target domain samples, but also serves as an important basis for the queue update mechanism and sample writing criteria in subsequent step 1. Step 4: Experiment: Dataset: The PACS dataset is a classic dataset for cross-domain visual learning, containing four different style domains: Photo, Artpainting, Cartoon, and Sketch. Each domain includes seven categories (such as dog, horse, guitar, etc.). This dataset is used to evaluate the ability of models to generalize to unseen target domains after training in multiple source domains. The different domains differ significantly in color, texture, and shape, making it suitable for testing the robustness of cross-domain classification and test-time adaptive methods.

[0049] Experimental results: To verify the effectiveness of the pseudo-label-guided test-time adaptive classification method (PLATA) described in this invention, experiments were conducted on the PACS dataset. The PACS dataset contains four domains: ArtPainting (A), Cartoon (C), Photo (P), and Sketch (S). Comparison methods included the basic ResNet50 and ViT-B16 models, as well as combinations of these models with the existing test-time adaptive method AdaNPC. Each method was trained and tested on the four domains, and the classification accuracy (%) was recorded, along with the average performance (Avg.).

[0050] Table 1 Performance evaluation of PLATA in PACS Overall performance improvement: Compared with the base model, the average accuracy is significantly improved after adding the PLATA method. For example, the average accuracy of ResNet50 increased from 83.46% to 84.84%, and Vit-B16 increased from 77.47% to 80.36%, indicating that PLATA can effectively utilize source domain supervision information to guide the KNN classifier during testing, thereby improving cross-domain classification performance.

[0051] Cross-domain consistency: Among the four domains, the PLATA method shows significant improvement in low-performance domains (such as Sketch). For example, the accuracy of ResNet50+PLATA in the S domain increased from 69.17% to 70.05%, and that of Vit-B16+PLATA in the S domain increased from 50.06% to 53.61%, indicating that the method can still maintain high generalization ability even with large domain shifts.

[0052] Compared with existing methods: PLATA shows an improvement in average performance compared with the AdaNPC method, indicating that the KNN fusion strategy guided by linear pseudo-labels can more fully mine source domain supervision information, improve the recognition rate of minority class samples, and maintain the computational advantages of nonparametric methods.

[0053] Applicability to different network architectures: PLATA is applicable to both convolutional networks (ResNet50) and Transformers (Vit-B16), demonstrating the method's versatility and scalability.

[0054] Experimental results verify that the PLATA method described in this invention can achieve cross-domain adaptive classification by guiding the KNN classifier with pseudo-labels without increasing significant computational overhead, effectively improving the model's average classification accuracy and generalization ability.

[0055] The test-time adaptive method based on class balance and linear pseudo-label guidance is suitable for cross-domain classification and recognition tasks with significant distribution differences, and is especially suitable for practical application scenarios where source domain data cannot be obtained during the testing phase and backpropagation or model retraining is not allowed.

[0056] In practical applications, the method can be deployed in image, video, or other high-dimensional signal analysis systems based on deep neural networks. After completing one-time supervised training on source domain data, the model directly receives data input from the target domain during the actual operation phase. For each target domain sample, the system first extracts feature representations using a feature extraction network and generates linear pseudo-labels using a linear classifier obtained through supervised training in the source domain. Simultaneously, nonparametric KNN inference is performed based on a dynamically maintained feature memory to obtain class scores reflecting the local structural information of the target domain. Subsequently, through a fusion inference mechanism guided by linear pseudo-labels, the final classification result is output without updating the model parameters, and high-confidence samples are selected for memory updates, thereby achieving continuous adaptation to the target domain distribution.

[0057] The described method can be widely applied to natural image classification, cross-device or cross-scene visual recognition, remote sensing image analysis, industrial inspection, and medical image-assisted analysis, and is particularly suitable for scenarios where data distribution changes due to different acquisition devices, imaging conditions, or environmental factors. By introducing a nonparametric memory mechanism and a linear pseudo-label guidance strategy during the testing phase, the method can improve the model's generalization ability and robustness in unknown domains or new environments while ensuring system stability and real-time performance.

[0058] Furthermore, since the method does not require access to source domain data, additional annotation information, or a backpropagation process, it is particularly suitable for deployment in edge devices or online inference systems with high requirements for privacy, security, or limited computing resources, and has good engineering feasibility and application promotion value.

[0059] The present invention includes a step of selectively updating the memory bank based on high-confidence samples and class balance constraints during the testing phase, enabling the KNN classifier to adapt to the target domain distribution in real time.

[0060] This memory update mechanism ensures the accumulation and efficient utilization of cross-domain features without gradient fine-tuning.

[0061] This invention utilizes the output of a linear classifier trained in the source domain to generate pseudo-labels, thereby providing supervised guidance for the KNN nonparametric classifier.

[0062] By fusing consistent samples through a pseudo-label weighting mechanism and dynamically selecting KNN outputs for inconsistent samples based on confidence, cross-domain classification accuracy is improved.

[0063] This invention combines a feature memory update mechanism with pseudo-label guidance from a linear classifier to achieve a test-time adaptive classification framework that requires no additional labeled data and no parameter fine-tuning.

[0064] This joint strategy can significantly improve the model's generalization ability in multi-source domains, class imbalance, and distribution shift scenarios.

Claims

1. A test-time adaptive method based on class balancing and pseudo-label guidance, characterized in that, Includes the following steps; Step 1: Build and dynamically update a class-balanced feature memory; Step 2: Based on the feature memory, perform neighborhood retrieval and preliminary classification of the target domain samples using the KNN nonparametric classifier to obtain the nonparametric classification results; Step 3: By introducing the linear pseudo-label information obtained in the source domain supervised training stage, the nonparametric classification results in Step 2 are globally constrained and corrected to reduce the interference of erroneous pseudo-labels and local noise samples on the inference results. Based on the fused inference results and their confidence, the update process of the feature memory bank in Step 1 is adjusted in reverse to achieve dynamic balance of the category distribution of the memory bank.

2. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 1, characterized in that, In the update mechanism of step 1, when the number of samples in a certain category reaches the preset upper limit, new samples are no longer written unconditionally. Instead, the earliest written sample from the target domain in that category is replaced. While ensuring the upper limit of the number of categories, more representative recent samples are introduced to avoid long-term solidification of the memory distribution.

3. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 2, characterized in that, Step 1 specifically involves: Step 1): Before starting the target domain test, initialize the feature memory using the source domain data; The feature memory adopts a fixed-length queue structure, consisting of several "feature vectors + labels"; Each feature vector is L2 normalized to ensure the stability of cosine similarity calculation; Step 2): During the testing phase, the target domain data is tested step by step in mini-batch. The feature memory is updated once each batch is completed.

4. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 3, characterized in that, In step 1): The memory storage method is as follows: Feature matrix: memory∈R^{Q×D}; Label vector: memory_label∈R^{Q}; Q: Memory length, which represents the total number of samples currently stored in the feature memory, can be a fixed value or change dynamically during the testing phase; D: Feature dimension, representing the dimension of the feature vector output by the feature extraction network; ,in This represents the feature vector corresponding to the i-th sample; ,in Representation and eigenvectors Corresponding category tags; The feature matrix and label vector together constitute an adaptive feature memory for KNN nonparametric classification and testing.

5. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 4, characterized in that, During the initialization phase, the model that has completed supervised training in the source domain is used to perform forward inference on the training domain data; Specifically, the source domain samples are sequentially input into the feature extraction network to extract the corresponding feature vectors, and the feature vectors are normalized to eliminate the influence of feature scale differences between different samples; at the same time, the classifier trained in the model is used to predict the source domain samples to obtain the true class label or supervision label corresponding to each sample. Subsequently, the normalized feature vector and its corresponding category label are stored in the feature memory bank as a set of "feature-label" pairs in a preset order to gradually build the initial feature matrix and label vector; when the memory bank capacity reaches the preset length limit, the writing stops, thus forming the initial memory bank structure for KNN nonparametric classification and adaptive inference in the testing phase. After the above initialization construction is completed, the feature memory is maintained and continuously used as the basic reference set in the testing phase; Once the testing phase begins, the memory no longer remains static. Instead, through a subsequent queue update mechanism, the content stored in the memory is updated in a controlled manner without disrupting the initial semantic structure. This gradually introduces target domain distribution information and enhances cross-domain adaptive capabilities.

6. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 5, characterized in that, Step 2) specifically refers to: Within each test mini-batch, queue update judgments are performed sequentially on the target domain samples. For each input sample in the target domain, prediction is performed using both a KNN nonparametric classifier based on a feature memory and a linear classifier trained under supervision in the source domain. The maximum class prediction probability for each classifier is then calculated. and ; Compare the predicted confidence level with a preset confidence threshold. and A comparison is made if and only if both of the following conditions are met: The target domain sample is determined to be a highly reliable sample, and its corresponding feature vector and predicted class label are allowed to enter the feature memory. If any confidence level fails to meet the threshold requirement, the sample is discarded and will not be included in the memory update. For samples that meet the write conditions, before performing the write operation, the number of samples of that category already stored in the current memory bank is first counted according to their predicted category. If the number does not exceed the maximum capacity max_per_class preset for that category, the normalized feature vector and predicted label of the sample are written to the memory bank in queue order. If the number of samples of that category has reached the maximum capacity, the write operation is not performed directly, but the storage queue corresponding to that category is scheduled and controlled according to the preset update strategy. The update strategy includes: Pause the writing of the current sample; Alternatively, existing samples in the category that have been stored for a long time and have low confidence can be replaced, thereby introducing more representative feature information while keeping the total number of samples in the category unchanged.

7. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 6, characterized in that, Step 2 specifically involves: Step 2 is based on the feature memory library built and dynamically maintained in step 1 to perform nonparametric classification reasoning, which is used to determine the category of the target domain sample during the testing phase. For the target domain samples to be classified, the corresponding feature vectors are first extracted using a feature extraction network. The feature vectors are then subjected to L2 normalization to ensure that their similarity calculations have a consistent metric with those of the feature vectors already stored in the feature memory.

8. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 7, characterized in that, Specifically: The feature matrix of the feature memory maintained in step 1 is: Its i-th row vector This represents the feature vector corresponding to the i-th sample; the corresponding label vector is: where the i-th element Representation and eigenvectors The corresponding category labels; each feature vector in the feature matrix has been normalized when written into the memory; During the test inference process, the feature vector z to be classified is calculated and compared with the feature matrix in the feature memory. The cosine similarity between each row vector yields a similarity vector: Subsequently, the similarity vectors are sorted according to the similarity scores, and the k most similar sample indices are selected to form a nearest neighbor set. Read the corresponding category label from the nearest neighbor set. The similarity scores of nearest neighbor samples belonging to the same category are summed to obtain scores for each category: in, This is an indicator function; it takes the value 1 when the category label of the i-th nearest neighbor sample is equal to category c, and otherwise takes the value 0. Based on the above calculations, an unnormalized category score vector is obtained: Where C represents the total number of categories; the score vector is determined entirely by the feature memory and similarity calculation results constructed in step 1, without relying on any learnable parameters, and is directly used as a non-parametric classification output.

9. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 8, characterized in that, Step 3 specifically involves: A linear pseudo-label-guided fusion inference method is constructed to perform stable and reliable cross-domain classification of target domain samples during the testing phase without backpropagation or access to source domain data. This fully integrates the global discriminant information obtained during the source domain supervised training phase with the local structural information of target domain samples in the feature space. Linear pseudo-labels are generated by a linear classifier obtained during the source domain supervised training phase. This linear classifier, serving as the classification head of the feature extraction network, is trained under the supervision of the true labels of the source domain samples. Its network parameters are frozen during the testing phase and used only for forward inference on the target domain samples. For the input target domain samples... The linear classifier outputs the corresponding class prediction probability distribution. And the category with the highest probability is used as the linear pseudo-label. At the same time, the maximum probability value is used as the prediction confidence level to characterize the degree of consistency between the target domain sample and the source domain discrimination boundary; Meanwhile, based on the feature memory library constructed and dynamically updated in step 1, the nonparametric KNN classification method described in step 2 is used to determine the neighborhood of the target domain samples.

10. The test-time adaptive method based on class balancing and pseudo-label guidance according to claim 9, characterized in that, Based on the target sample features and the feature matrix stored in the feature memory. Similarity calculations are performed to obtain the unnormalized class score vector of KNN based on local feature similarity. The score vector is determined entirely by the feature memory and similarity calculation results constructed in step 1; when confidence comparison or fusion with linear classifier results is required, the score vector is... Normalization is performed to obtain the corresponding KNN class probability distribution. The category with the highest probability is used as the pseudo-label for KNN classification. The KNN classification results mainly reflect the local consistency structure of the target sample in the feature space of the target domain, but they are easily affected by noisy samples and class imbalance in cross-domain scenarios. When using a smaller neighborhood size At the same time, the KNN classifier can maintain its sensitivity to the local discriminative structure of the target sample, but a small number of abnormal neighbors may have a significant impact on the voting results. If a large neighborhood size is used directly, the KNN classification results will be more biased towards reflecting the overall distribution characteristics of the memory bank, which can easily lead to over-smoothing and thus weaken the discriminative ability near the class boundary. Based on the above trade-offs, a linear pseudo-label is introduced as a consistency constraint signal in the fusion inference stage to guide the nonparametric decision-making process. When the linear classifier and the KNN classifier predict the same class for the target domain samples, that is, when the following condition is met: The target sample is determined to be a high-confidence sample, and the supervision weights are defined. KNN voting weights The predicted probabilities of the two classifiers are weighted and fused to obtain the fused prediction result: Among them, weight and Used to balance the contribution ratio of source domain supervision information and target domain local structure information in the final prediction; When the predictions of the two classifiers are inconsistent, i.e., satisfying Nonparametric decisions are re-evaluated by adaptively expanding the neighborhood of the KNN; Change the number of neighbors from Expand to Recalculate the KNN classification probability distribution based on a larger neighborhood. Furthermore, the confidence scores of KNN predictions at the original and expanded neighborhood sizes are compared based on the maximum class probability value, and the prediction with higher confidence is selected as the final output. By indirectly guiding the selection of KNN classification results through consistency constraints and confidence screening, the nonparametric inference process in the testing phase can fully inherit the discriminative information learned in the source domain supervised training phase while maintaining the ability to adapt to the feature distribution of the target domain. Thus, stable and robust classification of cross-domain samples can be achieved without additional training or backpropagation.

Images