Image classification method and system based on continual test-time adaptation

Abstract

The present invention relates to the technical field of computer vision, and in particular to an image classification method and system based on continual test-time adaptation. In the present invention, by using class centroids corresponding to different image classes as nodes, and using distances between the class centroids as edges, a dynamically updated inter-class topological graph is constructed, thereby ensuring the stability of the inter-class topological structure; the logarithm of the average Gaussian potential over all node pairs in the inter-class topological graph is used as an inter-class uniformity loss, thereby preventing inter-class topology collapse; on the basis of distribution of the different image classes, a batch imbalance topology weighting mechanism is proposed to adapt to distribution imbalance of image classes in batches, thus obtaining an inter-hierarchical uniformity loss; an intra-class compactness loss is introduced to enhance the compactness of features; and a topological consistency loss, a symmetric cross-entropy loss and a feature alignment loss are used to jointly construct a total loss function, and, by means of online iterative optimization of model parameters, adaptation under continual domain shifts is achieved. The present invention improves the image classification accuracy of models in dynamic target domains.

Description

An image classification method and system based on continuous test time adaptation Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an image classification method and system based on continuous test time adaptation. Background Technology

[0002] In recent years, Test-Time Adaptation (TTA) has been extensively studied. This technique, used in unsupervised settings, aims to adapt a model trained in the source domain to a completely new target domain during inference without accessing the source image data. The emergence of TTA has made it possible to apply models to scenarios with different image data distributions, greatly expanding the applicability of models and demonstrating potential application value in numerous fields. For example, in autonomous driving, images captured by vehicle sensors are affected by various factors such as weather, lighting conditions, and traffic conditions. With the help of TTA, autonomous driving systems can automatically adjust model parameters in real time based on images captured by sensors in complex environments, ensuring that the vehicle can accurately identify key image information such as roads, pedestrians, and traffic signals, thereby guaranteeing driving safety.

[0003] With continuous technological advancements, Continuous Test-Time Adaptation (CTTA), as an online continuous form of test-time adaptation, has gained widespread attention. CTTA focuses on continuously adjusting the model, enabling it to adapt to the ever-changing image data distribution in the target domain, without relying on the source image data. This characteristic allows the model to continuously adapt to new image environments in numerous real-world unsupervised scenarios. Currently, mainstream CTTA methods largely rely on pseudo-labeling techniques, among which methods based on the Mean Teacher (MT) structure are highly effective. For images in the target domain, the MT generates pseudo-labels, and by optimizing the consistency loss between the pseudo-labels and model predictions, the model is continuously driven to adapt to the target domain.

[0004] However, current models utilizing continuous test-time adaptive methods often experience a gradual decline in performance when processing target domain image data. This is primarily due to two key factors. Firstly, the balance of inter-class features is easily disrupted. Because the distribution of target domain image data differs significantly from that of source domain image data, the original balance of inter-class features in the source domain is broken. For example, source domain image data may exhibit specific distribution patterns in features such as color and shape, providing clear distinctions between different image categories. However, when faced with target domain image data, changes in shooting environment and image style disrupt the relative relationships of inter-class features. Feature combinations that were previously effective in distinguishing different image categories no longer apply in the target domain, making it difficult for the model to accurately classify images. Secondly, target domain image data is constantly evolving. As the model continuously processes target domain image data, the uniformity of feature distribution deteriorates. Data that should have been tightly clustered and easily identified as belonging to the same category becomes scattered, blurring classification boundaries. The model becomes increasingly unable to accurately distinguish image samples from different categories. The continuous accumulation of errors within the model severely impairs image classification performance. Incorrect pseudo-labels mislead model parameter updates, further exacerbating performance degradation and causing insufficient classification accuracy when faced with constantly changing target domain image data. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the defects of the model using the continuous test time adaptive method in processing target domain image data. The difference in data distribution between the target domain and the source domain causes the balance between inter-class features to be broken, and the uniformity of feature distribution deteriorates, resulting in blurred classification boundaries. This leads to insufficient classification accuracy of the model when facing constantly changing target domain image data.

[0006] To address the aforementioned technical problems, this invention provides an image classification method based on continuous test time adaptation, comprising the following steps:

[0007] Obtain the image datasets corresponding to T time points within the target region. Based on the image datasets at time t, obtain the features and predicted image category of each image output by the student model at time t; where t = 1, 2, ..., T.

[0008] Based on the features of each image at time t and the predicted image category, calculate the initial class centroids corresponding to different image categories at time t;

[0009] Based on the initial centroids of different image categories at time t and the centroids of different image categories at time t-1, the centroids of different image categories at time t are obtained through the exponential moving average algorithm.

[0010] Using the centroids of different image categories at time t as nodes and the distances between the centroids of different image categories at time t as edges, construct the inter-class topology graph at time t.

[0011] The logarithm of the average Gaussian potential of all nodes in the inter-class topological graph at time t is used as the inter-class uniformity loss at time t.

[0012] Based on the distances between all nodes in the inter-class topology graph at time t to their corresponding features, construct the intra-class compactness loss at time t.

[0013] Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t, a total loss function at time t is constructed.

[0014] The parameters of the student model at time t are updated using the total loss function at time t. Based on the parameters of the student model at time t, the parameters of the teacher model at time t are updated. After training is completed at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

[0015] Preferably, the logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t is used as the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is:

[0016] in, Let |V be the inter-class uniformity loss at time t. (t) | represents the number of nodes in the inter-class topology graph at time t, where T is a set parameter, and V (t) Let v be the set of nodes in the inter-class topology graph at time t. i Let i and v be nodes in the inter-class topology graph at time t. j Let i be node j in the inter-class topology graph at time t, i be the index of the first node, j be the index of the second node, and w be the index of the second node. ij Let be the edge between node i and node j in the inter-class topology graph at time t.

[0017] Preferably, the intra-class compactness loss at time t is constructed based on the distances between all nodes in the inter-class topology graph at time t to their corresponding features. The formula for the intra-class compactness loss at time t is:

[0018] in, Let be the intra-class compactness loss at time t. Let v be the class centroid of image class k in the inter-class topological graph at time t. k edge set, V (t) Let v be the set of nodes in the inter-class topology graph at time t. k Let be the class centroid of image class k in the inter-class topological graph at time t. Let T be the desired value, and z be the set parameter. α Let z be the α-th feature corresponding to image class k in the inter-class topological graph at time t. β Let |z| be the β-th feature corresponding to image class k in the inter-class topological graph at time t. α -z β | 2 For z α With z β The square of the distance between them To calculate the expected value of the squared distances between all possible feature pairs extracted from image class k in the inter-class topology graph at time t, where k is the image class index.

[0019] Preferably, the total loss function at time t further includes: the inter-stratum uniformity loss at time t, and the process of constructing the inter-stratum uniformity loss at time t includes:

[0020] Based on the distribution of different image categories in the image dataset at time t, the initial weighting terms of each edge in the inter-class topological graph at time t are obtained;

[0021] Based on the initial weighted terms of each edge in the inter-class topology graph at time t and the weighted terms of each edge in the inter-class topology graph at time t-1, the weighted terms of each edge in the inter-class topology graph at time t are obtained by using the exponential moving average algorithm.

[0022] By weighting the edges of the inter-class topology graph at time t, the inter-class uniformity loss at time t is adjusted to obtain the inter-hierarchical uniformity loss at time t.

[0023] Based on the inter-level uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t, a total loss function at time t is constructed.

[0024] Preferably, the initial weighting terms for each edge in the inter-class topological graph at time t, based on the distribution of different image categories in the image dataset at time t, include:

[0025] Based on the distribution of each image category in the image dataset at time t, the weight coefficient of each image category in the topology at time t is calculated using the following formula:

[0026] Based on the weight coefficient of each image category in the topology at time t, calculate the initial weighting terms of each edge in the inter-class topology graph at time t;

[0027] in, Let be the weight coefficient of image category k in the topology at time t. K represents the number of images belonging to image category k in the image dataset at time t. (t)Let be the number of image categories in the image dataset corresponding to time t, k be the image category index, and ∈ be a very small constant used to avoid a denominator of zero. Let be the initial weighting term of the edge between node i and node j in the inter-class topology graph at time t, where i is the index of the first node and j is the index of the second node. Let be the weight coefficient of the image class corresponding to node i in the inter-class topology graph at time t, within the topological structure. Let be the weight coefficient of the image class corresponding to node j in the inter-class topology graph at time t.

[0028] Preferably, the inter-class uniformity loss at time t is adjusted by weighting the edges of the inter-class topology graph at time t to obtain the inter-hierarchical uniformity loss at time t. The formula for the inter-hierarchical uniformity loss at time t is:

[0029] in, V represents the loss of inter-stratum homogeneity at time t, where T is a set parameter. (t) Let v be the set of nodes in the inter-class topology graph at time t. i Let i and v be nodes in the inter-class topology graph at time t. j Let i be node j in the inter-class topology graph at time t, i be the index of the first node, j be the index of the second node, and w be the index of the second node. ij Let be the edge between node i and node j in the inter-class topology graph at time t. Let be the weighted term of the edge between node i and node j in the inter-class topology graph at time t.

[0030] Preferably, the total loss function at time t is constructed based on the inter-level uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss. The total loss function at time t is:

[0031] in, Let be the total loss function at time t. The symmetric cross-entropy loss at time t, Let be the feature alignment loss at time t. The loss represents the uniformity between different levels at time t. Let λ be the intra-class compactness loss at time t, λ1 be the feature alignment weight, and λ2 be the topological stability weight.

[0032] Preferably, the image dataset corresponding to time t is augmented to obtain the augmented image dataset corresponding to time t.

[0033] Input the augmented image dataset corresponding to time t into the feature extractor of the student model to obtain the random augmented features of each image at time t;

[0034] Based on the random augmentation features of each image at time t and the predicted image category, the initial class centroids corresponding to different image categories at time t are calculated.

[0035] Preferably, the initial class centroids corresponding to different image classes at time t are calculated based on the random augmentation features and predicted image class of each image at time t, using the following formula:

[0036] in, This represents the initial class centroid corresponding to image class k at time t. This represents the index of the maximum value in the output of the a-th image at time t after processing by the student model. Aug(·) indicates data augmentation. Let represent the a-th image in the image dataset corresponding to time t, argmax(·) be the argmax function, I(·) be the indicator function, and B be the image in the dataset corresponding to time t. (t) This represents the number of images in the image dataset at time t. This represents the set of images belonging to image category k in the image dataset at time t. This represents the number of images belonging to image category k in the image dataset at time t. Indicates judgment Is it equal to k? If it is, then... Not equal to This represents the features extracted by the student model from the augmented data at time t. Let be the feature extractor for the student model at time t, where a is the image index and k is the image category index.

[0037] The present invention also provides an image classification system based on continuous test time adaptation, comprising:

[0038] The data acquisition module is used to acquire image datasets corresponding to T time points within the target area. Based on the image datasets corresponding to time point t, it acquires the features and predicted image category of each image output by the student model at time point t; where t = 1, 2…T.

[0039] The initial class centroid calculation module is used to calculate the initial class centroid corresponding to different image categories at time t based on the features of each image at time t and the predicted image category;

[0040] The class centroid calculation module is used to obtain the class centroids corresponding to different image categories at time t based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1, using the exponential moving average algorithm.

[0041] The inter-class topology graph construction module is used to construct the inter-class topology graph at time t by taking the centroids of different image categories at time t as nodes and the distances between the centroids of different image categories at time t as edges.

[0042] The inter-class uniformity loss construction module is used to take the logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t as the inter-class uniformity loss at time t.

[0043] The intra-class compactness loss construction module is used to construct the intra-class compactness loss at time t based on the distances between all nodes in the inter-class topology graph and their corresponding features at time t.

[0044] The total loss function construction module is used to construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t.

[0045] The parameter update module is used to update the parameters of the student model at time t using the total loss function at time t. Based on the parameters of the student model at time t, the parameters of the teacher model at time t are updated. After training is completed at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

[0046] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0047] This invention discloses an image classification method and system based on continuous test time adaptation. Considering that the target domain image data continuously and dynamically changes over time in continuous test time adaptation, this invention updates the current-time class centroids using an exponential moving average, based on the class centroids corresponding to different image categories at the previous time step and the initial class centroids corresponding to different image categories at the current time step. This effectively tracks the dynamic changing trend of image data, retaining the feature information of different image categories in historical data while continuously optimizing the judgment of features for each image category. By using the class centroids corresponding to different image categories as nodes and the distances between class centroids as edges, an inter-class topology graph reflecting the geometric relationships between classes is constructed. The inter-class topology graph is updated according to the datasets at different times, ensuring the structural stability of the feature space and avoiding erroneous judgments caused by confusion in inter-class features. Unlike traditional inter-class uniformity loss methods, which often rely on simple distances or single similarity metrics and fail to comprehensively reflect the overall uniformity and topological structure of inter-class distribution, this invention uses a dynamically changing inter-class topological graph. It employs the average Gaussian potential of all node pairs within the graph as the inter-class uniformity loss, minimizing the Gaussian potential between class centroids to ensure a uniform distribution of centroids. This reorganizes the previously chaotic inter-class feature relationships, enabling the model to maintain a consistent topological structure when facing dynamically changing target domains. Furthermore, this invention introduces intra-class compactness loss. Traditional intra-class loss methods often focus only on simple distance metrics between features, neglecting the overall structure and dynamic changes in feature distribution. This invention utilizes topological information such as class centroids and their edge sets to calculate the expected distance between intra-class feature pairs. Exponential and logarithmic transformations further strengthen the constraint on intra-class compactness. This intra-class compactness loss effectively addresses feature distribution issues arising from the dynamic evolution of target domain image data. By optimizing feature distribution compactness, it indirectly supports inter-class topological stability, enhances feature compactness, and avoids blurred classification boundaries caused by overly dispersed features. This invention designs inter-class uniformity loss and intra-class compactness loss from the perspectives of inter-class and intra-class, respectively. By optimizing the model parameters online iteratively, the model can effectively avoid erroneous pseudo-labels and biased updates caused by inter-class topological instability and poor feature distribution when facing constantly changing target domain data, thus significantly improving the classification accuracy of the model when facing constantly changing target domain image data.

[0048] This invention addresses the common challenges of continuous test-time adaptive methods, such as limited image data quantity and distribution differences between target and source domain image data. It utilizes the characteristics of randomly augmented data to calculate initial class centroids. The diverse features of the augmented data, compared to directly using the original data, result in more stable and representative initial class centroids, laying the foundation for calculating the inter-class topology graph at the current time. This allows the model to effectively reduce the impact of distribution shift and improve the classification accuracy of target domain images when processing target domain image data.

[0049] This invention takes into account that the image categories and number of images may differ in different batches of data. This imbalance can gradually destroy the inter-class topology, thereby affecting the model's prediction performance. Based on the distribution of different image categories in the image dataset, the class centroid weights in the inter-class uniformity loss are dynamically adjusted to optimize the inter-class distance. This ensures that the stability of the topological structure can still be maintained even when the image category distribution is uneven. Furthermore, the hierarchical uniformity loss is obtained. Through the hierarchical uniformity loss, the model's adaptability to the differences in the distribution of image category data is improved, significantly improving the accuracy and stability of the model for classifying images in the target domain. Attached Figure Description

[0050] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0051] Figure 1 is a flowchart of the steps of an image classification method based on continuous test time adaptation according to the present invention.

[0052] Figure 2 is a structural diagram of an image classification method based on continuous test time adaptation according to the present invention.

[0053] Figure 3 is a schematic diagram comparing the topological stability of the present invention with other methods.

[0054] Figure 4 is a schematic diagram comparing the dimensionality reduction distribution of the model output features after using the present invention with other methods.

[0055] Figure 5 is a schematic diagram comparing the distribution of the model output features on a two-dimensional unit hypersphere after using the present invention with other methods. Detailed Implementation

[0056] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0057] In practice, training and testing data for models often come from different distributions, resulting in a domain transfer problem. For example, training data might be images taken during the day, while testing data might be images taken at night. Traditional domain adaptation methods typically require training with target domain data before testing. However, this is impractical in many real-world applications because target domain data may only become available at test time and may be constantly changing. Therefore, the Test-Time Adaptation (TTA) method was introduced.

[0058] With the advancement of technological research, the Continuous Test Time Adaptive (CTTA) method has been proposed. However, in the process of processing target domain image data, due to the significant differences in the distribution between target domain image data and source domain image data, the original inter-class feature balance in the source domain is disrupted, leading to confusion in the relative relationships between inter-class features. Feature combinations that were originally effective in distinguishing different image categories are no longer applicable in the target domain, making it difficult for the model to accurately determine the image category. Furthermore, target domain image data is constantly evolving. As the model continues to process target domain image data, the uniformity of feature distribution deteriorates. Data that should have been tightly clustered in one place and easily identified as belonging to the same class by the model now becomes scattered, and the classification boundaries become blurred, making it increasingly difficult for the model to accurately distinguish image samples of different image categories.

[0059] Therefore, this invention proposes an image classification method based on continuous test time adaptation.

[0060] Referring to Figure 1, this embodiment provides an image classification method based on continuous test time adaptation, including the following steps:

[0061] This invention addresses existing source data. The pre-trained model is parameterized, where, For the dataset in the existing source data, for With the corresponding label set, after proper training, the goal of continuous test-time adaptation is to respond to the constantly changing target domain during the testing phase. Without accessing any source data.

[0062] As shown in Figure 2, Figure 2 is a structural diagram of an image classification method based on continuous test time adaptation according to the present invention.

[0063] Step S1: Obtain the image datasets corresponding to T time points within the target area. Based on the image datasets at time t, obtain the features and predicted image category of each image output by the student model at time t; where t = 1, 2…T.

[0064] As shown in Figure 3, which is a comparative diagram of the present invention and other methods in terms of topological stability, the present invention aims to achieve the goal of adaptive continuous testing time by dynamically maintaining the inter-class topological relationship. Its core lies in using class centroids as proxies for image categories and class centroid distances as proxies for inter-class topological distances, thereby constructing an inter-class topological structure.

[0065] First, define the class topology graph. in, v represents the set of centroids of different image categories. KThis represents the class centroid corresponding to the Kth image category, where K is the number of image categories. For example, in an image classification task, if we want to distinguish between images of cats and dogs, each category has its own corresponding class centroid. The class centroid represents the typical location of this category of images in the feature space. The edge set represents the neighborhood relationship between the centroids of these classes, where each edge e ij Each is assigned a weight ω ij w ij The class centroid v used to represent image category i i and the class centroid v of image category j j The Euclidean distance (L2 distance) between them describes the relative positional relationship of different image categories in the feature space by constructing the topological relationship between the centroids of the classes.

[0066] In traditional machine learning and computer vision, calculating class centroids is a fundamental and crucial operation, commonly used in tasks such as classification and clustering. When the K-means clustering algorithm is applied to image data, for each clustered image category, the centroid representation of that image category is obtained by directly averaging the pixel values ​​of images belonging to that category in the RGB color channel dimensions. This method can simply and quickly summarize the features of image categories in scenarios with static datasets and relatively stable features. Existing technologies obtain the class centroid corresponding to each image category by calculating the mean of all original data features in each image category. However, in the field of continuous test time adaptation, the model needs to be able to adapt to new data when faced with dynamically changing data. Therefore, relying solely on the original features to calculate the class centroid of each image category cannot comprehensively and accurately capture the true distribution of image categories at different times.

[0067] This invention constructs a dynamically updated inter-class topology graph based on class centroids. It calculates class centroids using data-augmented features and updates them through an exponential moving average, ensuring the stability of the inter-class topology. The specific construction process of the class topology graph is shown in steps S2-S4.

[0068] Step S2: Based on the features and predicted category of each image at time t, calculate the initial class centroids corresponding to different image categories at time t;

[0069] In this embodiment, preferably, the image dataset corresponding to time t is augmented to obtain the augmented image dataset corresponding to time t.

[0070] Input the augmented image dataset corresponding to time t into the feature extractor of the student model to obtain the random augmented features of each image at time t;

[0071] Based on the random augmentation features of each image at time t, the initial class centroids corresponding to different image categories at time t are calculated.

[0072] In this embodiment, preferably, the initial class centroids corresponding to different image categories at time t are calculated based on the random enhancement features of each image at time t and the predicted image category of each image at time t. The calculation formula is as follows:

[0073] in, This represents the initial class centroid corresponding to image class k at time t. This represents the index of the maximum value in the output of the a-th image at time t after processing by the student model. Aug(·) indicates data augmentation. Let represent the a-th image in the image dataset corresponding to time t, argmax(·) be the argmax function, I(·) be the indicator function, and B be the image in the dataset corresponding to time t. (t) This represents the number of images in the image dataset at time t. This represents the set of images belonging to image category k in the image dataset at time t. This represents the number of images belonging to image category k in the image dataset at time t. Indicates judgment Is it equal to k? If it is, then... Not equal to This represents the features extracted by the student model from the augmented data at time t. Let be the feature extractor for the student model at time t, where 'a' is the image index and 'k' is the image category index. Let t be the student model at time t.

[0074] in, This represents the index of the maximum value in the output of the a-th image at time t after processing by the student model. For example, if the student model is targeting... The output result is [0.2, 0.5, 0.3].

[0075] At time t, this invention uses the random augmentation features of the current batch to improve the reliability of the class centroid, which can eliminate the instability of the class centroid caused by finite samples or distribution shift, thereby producing more stable and reliable pseudo-labels.

[0076] Step S3: Based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1, the class centroids corresponding to different image categories at time t are obtained through the exponential moving average algorithm;

[0077] In this embodiment, preferably, the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1 are used to obtain the class centroids corresponding to different image categories at time t through an exponential moving average algorithm, and the formula is as follows:

[0078] in, Let k be the centroid of the image class k at time t. Let the centroid of image class k at time t-1 be . γ represents the initial centroid of image category k at time t, γ is a weighting parameter that controls the influence of previous batch centroids, and k is the image category index.

[0079] In scenarios with varying batch sizes and imbalanced data distributions, the exponential moving average method, which fuses the update of class centroid information from batches before and after processing, enables the model to effectively maintain a stable inter-class topology. This avoids topological disorder caused by data imbalance or batch differences. This strategy ensures stable inter-class topology across different batch sizes and imbalanced data distributions, even in situations with scarce images or single-image (B) datasets. t =1) In online scenarios, the random enhancement features of the current image are used as a proxy for class centroid updates to ensure that the updates of class centroids are still based on evidence, maintain the dynamic updates and improvement of the topology map, and effectively track the dynamic changes of image data. While retaining the feature information of different image categories in historical data, the judgment of each image category feature is continuously optimized.

[0080] By performing specific computational processes on image datasets corresponding to different times, the instability of class centroids caused by limited images or distribution shifts can be effectively eliminated. The data augmentation operation Aug(.) is used to increase data diversity, thereby reducing the impact of limited images or distribution shifts on class centroid calculation, laying the foundation for obtaining relatively reliable class centroids. Reliable class centroids will be used to generate stable pseudo-labels in the future.

[0081] Step S4: Using the centroids of different image categories at time t as nodes and the distances between the centroids of different image categories at time t as edges, construct the inter-class topology graph at time t.

[0082] In each batch processing step, the inter-class topology covers all image categories, and the class topology graph is continuously updated throughout the testing process. Specifically, this invention will update the class topology graph... Initialized as an empty image, it will be iteratively populated and updated with class centroids as new batches are processed. The class centroid updates depend not only on the initial class centroids corresponding to different image categories in the current batch of data. Furthermore, it includes class centroids corresponding to different image categories from the previous batch. This led to the acquisition of the class centroids corresponding to different image categories in the current batch of data after the updates.

[0083] Step S5: Take the logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t as the inter-class uniformity loss at time t.

[0084] In traditional graph neural networks, the inter-class uniformity loss uses a simple method based on node connection density. It measures inter-class uniformity by calculating the ratio of the number of direct connections between nodes in different image categories to the total number of nodes. This method is relatively intuitive but has significant drawbacks. It only considers the direct connections between nodes, ignoring the connection strength and the relative positions of nodes within the network space. In the field of manifold learning for image dimensionality reduction, to achieve a uniform distribution of data from different image categories in the low-dimensional space, an inter-class uniformity loss based on Euclidean distance between data points is often used. For example, minimizing the average Euclidean distance between data points from different image categories in the low-dimensional space achieves inter-class uniformity. However, this method does not consider the complex nonlinear relationships between data points. In image data, the features of different image categories often exhibit complex nonlinear distributions. Simple Euclidean distance cannot accurately capture these nonlinear features, making it difficult to achieve true inter-class uniformity when processing complex image data.

[0085] In continuous test-time adaptation, the target domain image data has unique dynamic characteristics, the data distribution changes continuously over time, and there are significant differences between the source and target domain data. Traditional inter-class uniformity loss methods in graph neural networks cannot adapt to the complex feature evolution of image data in continuous test-time adaptation because they do not consider the dynamic changes and inter-domain differences of the data. In manifold learning, the inter-class uniformity loss based on Euclidean distance cannot be effectively adjusted to meet the model's accurate requirements for inter-class uniformity in the face of the nonlinear dynamic changes of image data in continuous test-time adaptation.

[0086] Therefore, this invention proposes a unique method for inter-class uniformity loss. In this embodiment, preferably, the logarithm of the average Gaussian potential of all nodes in the inter-class topological graph at time t is used as the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is:

[0087] in, Let |V be the inter-class uniformity loss at time t. (t) | represents the number of nodes in the inter-class topology graph at time t, and T is a set parameter. In this embodiment, T is set to 2. (t) Let v be the set of nodes in the inter-class topology graph at time t. i Let i and v be nodes in the inter-class topology graph at time t.j Let i be node j in the inter-class topology graph at time t, i be the index of the first node, j be the index of the second node, and w be the index of the second node. ij Let be the edge between node i and node j in the inter-class topology graph at time t.

[0088] To address the problem of topological instability, this invention separates the centroids of different image categories by using inter-class uniformity loss, minimizes the Gaussian potential energy between class centroids to maintain inter-class topological stability, reduces the propagation of inter-class errors, maintains stable inter-class distances, prevents inter-class topological collapse, and thus maintains a coherent and consistent topological structure.

[0089] Step S6: Based on the distances between all nodes in the inter-class topology graph at time t to their corresponding features, construct the intra-class compactness loss at time t;

[0090] The stability of inter-class topology depends on inter-class distribution and feature concentration. Overly concentrated features can ignore subclass variations and noise, while scattered features hinder stable intra-class learning. Traditional intra-class compactness loss typically considers either intra-class compactness or inter-class discriminability separately. For example, the common center loss only calculates the distance between sample features and fixed class centers to bring similar samples closer to their class centers. Center loss is usually based on source domain data to determine class centers during training. However, when applied to the target domain, due to changes in data distribution, the original class centers may no longer be applicable to the target domain data. Furthermore, in the continuous test-time adaptive domain, target domain image data often changes over time, easily generating outliers. Since center loss only considers the distance between samples and class centers, outliers or noisy data can significantly affect the calculation of class centers, thus affecting the calculation of the entire intra-class compactness loss, misleading model training, and reducing the model's generalization ability and classification accuracy.

[0091] Therefore, in order to further maintain a stable inter-class topology, this invention further compresses and unifies features, introducing an intra-class compactness loss for V. (t) The feature pairs within the class are paired and homogenized, taking into account the mathematical expectation of the square of the distance between all possible extracted feature pairs, in order to achieve a more compact and uniform inter-class feature distribution.

[0092] In this embodiment, preferably, the intra-class compactness loss at time t is constructed based on the distances between all nodes in the inter-class topology graph at time t to their corresponding features. The formula for the intra-class compactness loss at time t is as follows:

[0093] in, Let be the intra-class compactness loss at time t. Let v be the class centroid of image class k in the inter-class topological graph at time t. k edge set, V(t) Let v be the set of nodes in the inter-class topology graph at time t. k Let be the class centroid of image class k in the inter-class topological graph at time t. Let T be the desired value, and z be the set parameter. α Let z be the α-th feature corresponding to image class k in the inter-class topological graph at time t. β Let |z| be the β-th feature corresponding to image class k in the inter-class topological graph at time t. α -z β | 2 For z α With z β The square of the distance between them To calculate the expected value of the squared distances between all possible feature pairs extracted from image class k in the inter-class topology graph at time t, where k is the image class index.

[0094] By introducing intra-class compactness loss at time t, the compactness of feature distribution is optimized to indirectly support inter-class topological stability, enhance feature compactness, and avoid the blurring of classification boundaries caused by overly dispersed features.

[0095] This invention addresses dynamic domain adaptation scenarios by considering both inter-class and feature aspects separately. Without this feature separation and instead treating all features uniformly, the inter-class topology is highly likely to be distorted. This is because, under uniform processing, class boundaries tend to become blurred, reducing the discriminative power between different image categories. This is particularly detrimental in dynamic environments where class distributions change over time, leading to a significant decline in model performance.

[0096] This invention effectively avoids the aforementioned problems by separating inter-class and feature processing for targeted handling. Regarding inter-class relationships, a dynamically updated inter-class topology graph is constructed. This stable inter-class topological relationship accurately represents the relative positions of different image categories in the feature space, preventing the inter-class topological structure from being distorted due to feature confusion. This allows the model to clearly grasp inter-class boundaries when facing dynamically changing class distributions. At the intra-class level, an intra-class compactness loss is introduced to enhance feature compactness and prevent overly dispersed feature distributions from blurring classification boundaries. In this way, this invention can better adapt to the complex and ever-changing situations in dynamic domains, significantly improving the model's robustness and processing accuracy in dynamic environments, ensuring stable and improved model performance.

[0097] Step S7: Construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss.

[0098] At time t, the teacher model generates pseudo-labels to aid the student model's learning process. This invention calculates the symmetric cross-entropy loss (SCE) between the student model's output and the pseudo-label's output. The formula for the symmetric cross-entropy loss at time t is:

[0099] in, Let K be the symmetric cross-entropy loss at time t, and k be the image category index. t Let be the number of image categories in the image dataset at time t. Let be the probability predicted by the teacher model at time t that the input x belongs to image category k. Let t be the probability that the student model predicts the input x as belonging to image category k at time t.

[0100] Furthermore, while maintaining feature consistency, the feature distributions between positive pairs are aligned, and the formula for the feature alignment loss at time t is:

[0101] in, Let be the feature alignment loss at time t. This indicates that at time t, the positive sample pair (x, y) follows a distribution p. pos (t) The expected operation below, The square of the L2 norm. Let p be the feature extractor for the student model at time t, where x and y represent features obtained from the input after undergoing different forms of data augmentation. pos (t) This represents the distribution of positive sample pairs (x, y) at time t, that is, the probability distribution of these positive sample pairs appearing at time t.

[0102] Step S8: Update the parameters of the student model at time t using the total loss function at time t. Based on the parameters of the student model at time t, update the parameters of the teacher model at time t. After completing the training at time T, use the teacher model at time T with updated parameters as the target image classification model to classify the target domain image.

[0103] Different batches of data often cover different image categories, and the number of samples in these categories varies significantly. In unlabeled data environments, this imbalance in sample size progressively disrupts the inter-class topology. While the feature distribution may appear uniform on the surface, it actually exhibits a high degree of imbalance, which undoubtedly negatively impacts the model's predictive performance. This invention innovatively integrates the inter-class topological distance with the true class distribution information output by the model. By dynamically adjusting the class centroid weights based on the image category distribution within a batch, the inter-class distance is optimized. Even in complex situations with uneven image category distribution, the stability of the inter-class topology can be effectively ensured, thereby significantly improving the model's performance in practical applications. The specific steps are as follows:

[0104] In this embodiment, preferably, the total loss function at time t further includes: the inter-hierarchical uniformity loss at time t, and the process of constructing the inter-hierarchical uniformity loss at time t includes:

[0105] Based on the distribution of different image categories in the image dataset at time t, the initial weighting terms of each edge in the inter-class topological graph at time t are obtained;

[0106] In this embodiment, preferably, the initial weighting terms for each edge in the inter-class topological graph at time t, based on the distribution of different image categories in the image dataset at time t, include:

[0107] Based on the distribution of each image category in the image dataset at time t, the weight coefficient of each image category in the topology at time t is calculated using the following formula:

[0108] The batch imbalanced topology weighting matrix is ​​composed of the weight coefficients of all image categories in the topology at time t. It can effectively reflect the relative importance of different image categories in the topology at time t, as well as the weight of each edge in the inter-class topological relationship. This provides a basis for subsequent weighted adjustment of the uniformity loss between hierarchical levels, in order to cope with the impact of the imbalance of image category sample size in different batches on model performance.

[0109] Based on the weight coefficient of each image category in the topology at time t, calculate the initial weighting terms of each edge in the inter-class topology graph at time t;

[0110] in, Let be the weight coefficient of image category k in the topology at time t. K represents the number of images belonging to image category k in the image dataset at time t. (t) Let be the number of image categories in the image dataset corresponding to time t, k be the image category index, and ∈ be a very small constant used to avoid a denominator of zero. Let be the initial weighting term of the edge between node i and node j in the inter-class topology graph at time t, where i is the index of the first node and j is the index of the second node. Let be the weight coefficient of the image class corresponding to node i in the inter-class topology graph at time t, within the topological structure. Let be the weight coefficient of the image class corresponding to node j in the inter-class topology graph at time t.

[0111] Based on the initial weighted terms of each edge in the inter-class topology graph at time t and the weighted terms of each edge in the inter-class topology graph at time t-1, the weighted terms of each edge in the inter-class topology graph at time t are obtained by using the exponential moving average algorithm.

[0112] In this embodiment, preferably, the weighted terms of each edge in the inter-class topology graph at time t and the weighted terms of each edge in the inter-class topology graph at time t-1 are obtained by using an exponential moving average algorithm, and the formula is as follows:

[0113] in, Let ω be the weighted term of the edge between node i and node j in the inter-class topology graph at time t, where ω is the weight parameter. Let be the weighted term of the edge between node i and node j in the inter-class topology graph at time t-1. Let be the initial weighting term of the edge between node i and node j in the inter-class topology graph at time t.

[0114] During model training, the data distribution may change over time or with different batches of data. The weighted matrix, composed of the weighted terms of each edge in the inter-class topology graph at time t, is used to adjust the importance of each image category or connection relationship in the model, and has a key impact on model performance. By using exponential moving average to calculate the weighted terms of each edge in the inter-class topology graph at time t, the latest weights reflected by the current data are integrated, while retaining the weighted features corresponding to the past data distribution. This allows the weighted matrix to take into account both the continuity and variability of the data distribution over time, preventing the model from deviating excessively from the previously learned useful information due to the local features of the current data, thereby improving the stability and adaptability of the model under different data distribution conditions.

[0115] By weighting the edges of the inter-class topology graph at time t, the inter-class uniformity loss at time t is adjusted to obtain the inter-hierarchical uniformity loss at time t.

[0116] In this embodiment, preferably, the inter-class uniformity loss at time t is adjusted by weighting the edges of each edge in the inter-class topology graph at time t to obtain the inter-hierarchical uniformity loss at time t. The formula for the inter-hierarchical uniformity loss at time t is:

[0117] in, V represents the loss of inter-stratum homogeneity at time t, where T is a set parameter. (t) Let v be the set of nodes in the inter-class topology graph at time t. i Let i and v be nodes in the inter-class topology graph at time t. j Let i be node j in the inter-class topology graph at time t, i be the index of the first node, j be the index of the second node, and w be the index of the second node. ij Let be the edge between node i and node j in the inter-class topology graph at time t. Let be the weighted term of the edge between node i and node j in the inter-class topology graph at time t.

[0118] This invention applies gradient stopping operation to the initial weighting term of the edge between node i and node j in the inter-class topological graph at time t. This ensures that the model remains stable during training, allowing the loss between hierarchical uniformity at time t to take into account the differences in importance of relationships between different image categories. This more accurately reflects the uniformity of feature distribution between hierarchical levels, thereby optimizing the training process and improving model performance.

[0119] Based on the inter-level uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t, a total loss function at time t is constructed.

[0120] In this embodiment, specifically, the total loss function at time t is constructed based on the inter-level uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss. The total loss function at time t is:

[0121] in, Let be the total loss function at time t. The symmetric cross-entropy loss at time t, Let be the feature alignment loss at time t. The loss represents the uniformity between different levels at time t. Let λ be the intra-class compactness loss at time t, λ1 be the feature alignment weight, and λ2 be the topological stability weight.

[0122] This invention obtains the topological consistency loss at time t by integrating the inter-level uniformity loss and the intra-class compactness loss at time t.

[0123] As shown in Figure 4, Figure 4 is a schematic diagram comparing the dimensionality reduction distribution of the model output features after using the present invention with other methods. The upper left area of ​​Figure 4 shows the dimensionality reduction distribution of the model output features after processing the source domain data. The upper right area of ​​Figure 4 shows the dimensionality reduction distribution of the model output features after using the CoTTA method and adding Gaussian noise to the data. The lower left area of ​​Figure 4 shows the dimensionality reduction distribution of the model output features after using the RMT method and adding Gaussian noise to the data. The lower right area of ​​Figure 4 shows the dimensionality reduction distribution of the model output features after using the continuous test time adaptive image classification method (TCA) proposed in this invention and adding Gaussian noise to the data.

[0124] As shown in Figure 4, the model using the present invention for test time adaptation has a more concentrated and compact distribution of output features compared to models using other methods. The boundaries between different image categories are clearer, and a more obvious and stable inter-class topology is formed in the feature space. This indicates that the present invention can more effectively handle noise interference in the data, enabling the model to better adapt to changes in the target domain during the testing phase, reducing the possibility of inter-class error propagation, and thus improving the performance and robustness of the model in dynamic environments.

[0125] Figure 5 shows a comparison of the distribution of output features of the model using the present invention on a two-dimensional unit hypersphere with other methods. Figure 5 is divided into two rows. The first row shows the distribution of output features on a two-dimensional unit hypersphere using the CoTTA method, with the addition of three different types of noise: Gaussian noise, snowflake noise, and JPEG noise. The second row shows the distribution of output features on a two-dimensional unit hypersphere using the continuously test-time adaptive image classification method (TCA) proposed in this invention, again with the addition of Gaussian noise, snowflake noise, and JPEG noise.

[0126] As shown in Figure 5, compared to the model using the CoTTA method, the model using the method of this invention (TCA) exhibits a more significant advantage in the distribution of output features on the two-dimensional unit hypersphere. When facing different types of noise interference, the output feature distribution of the model using the method of this invention (TCA) is more concentrated, with a higher degree of clustering among similar feature points. This indicates that the invention effectively enhances the compactness of features, making the model more accurate in recognizing features of each image category. Simultaneously, the separation between features of different image categories is more significant, clearly defining the regions of each image category in the feature space. This means that the dynamically updated inter-class topology graph constructed by this invention effectively stabilizes the inter-class topology structure, reduces inter-class confusion caused by noise interference, and greatly improves the model's adaptability and robustness in complex noise environments. It ensures that the model can maintain good performance during test-time adaptation. Compared with other methods, this invention significantly maintains stable inter-class topological relationships and achieves more accurate classification performance.

[0127] This invention is based on online iterative dynamic model parameter optimization and adaptive updating. Through an online iterative optimization strategy, each batch of data sequentially performs class centroid calculation, topology graph update, loss optimization, and parameter update, ensuring that the model can adapt to continuous domain changes in real time and maintain the stability of feature distribution. After online iterative optimization, the model can output stable classification prediction image categories, significantly reducing the classification error rate, improving the robustness and processing accuracy of the model in dynamic environments, ensuring that the autonomous driving system can accurately identify key information such as roads, pedestrians, and traffic signals, and significantly improving the safety and reliability of autonomous driving.

[0128] Based on Embodiment 1, this Embodiment 2 utilizes an image classification method based on continuous test time adaptation proposed in this invention to evaluate three benchmark dataset test tasks in image processing, demonstrating the experimental results of this invention in the field of image processing.

[0129] The three benchmark datasets used for testing include CIFAR-10-C, CIFAR-100-C, and ImageNet-C. These testing tasks aim to evaluate the robustness of machine learning models to corruption and perturbation in the input data. CIFAR-10-C extends the CIFAR-10 dataset, comprising 32 × 32 color images from 10 classes. It includes 15 different corruptions, each at 5 severity levels, applied to the CIFAR-10 test images, resulting in a total of 10,000 images. CIFAR-100-C extends the CIFAR-100 dataset, containing 32 × 32 color images from 100 classes. It includes 15 different corruptions, each at 5 severity levels, applied to the CIFAR-100 test images, resulting in a total of 10,000 images. ImageNet-C extends the ImageNet dataset, which contains over 14 million images across more than 20,000 image categories. ImageNet-C includes 15 different corruptions, each with 5 severity levels. These corruptions are applied to validation images on ImageNet. This invention strictly adheres to the CTTA settings and does not access the source data. All comparative experimental models, including CoTTA, TENT, AdaContrast, and DSS, are evaluated online based on the maximum corruption severity level across all datasets. Model predictions are generated first before being adapted to the current test stream. This invention uses standard pre-trained WideResNet, ResNeXt-29, and ResNet-50 as source models for CIFAR10-C, CIFAR100-C, and ImageNet-C.

[0130] By utilizing adaptive thresholding for pseudo-label filtering, this invention successfully reduced the average error on CIFAR100-C and ImageNet-C from 32.5% to 29.7% and from 66.8% to 59.3%, respectively, compared to CoTTA. To further investigate the effectiveness of this invention on baselines, its adaptation performance on 10 different sequences was also evaluated on ImageNet-C. Compared to CoTTA, the average error reduction was 3% for more than 10 different sequences, indicating that the method of this invention is more robust to the order of the target structural domain sequences, as shown in Table 1. Table 1 illustrates the experimental results of this invention and different methods on the CIFAR10-C dataset.

[0131] Table 1

[0132] As shown in Table 2, Table 2 illustrates the experimental results of the present invention and different methods on the CIFAR100-C dataset.

[0133] Table 2

[0134] As shown in Table 3, Table 3 illustrates the experimental results of the present invention and different methods on the ImageNet-C dataset.

[0135] Table 3

[0136] Example 2 utilizes the online domain variation continuous learning method proposed in this invention to evaluate the test tasks on three benchmark datasets in the field of image processing: CIFAR-10-C, CIFAR-100-C, and ImageNet-C. Experimental results show that, in comparison experiments with various other methods (such as TENT, Ada, DSS, CoTTA, etc.), the overall performance of this invention on the three datasets is also superior, demonstrating the excellent robustness of this method in dealing with corrupted and interfered input data, and effectively improving the performance of machine learning models in image processing tasks.

[0137] This third embodiment provides an image classification system based on continuous test time adaptation, including:

[0138] The data acquisition module is used to acquire image datasets corresponding to T time points within the target area. Based on the image datasets corresponding to time point t, it acquires the features and predicted image category of each image output by the student model at time point t; where t = 1, 2…T.

[0139] The initial class centroid calculation module is used to calculate the initial class centroid corresponding to different image categories at time t based on the features of each image at time t and the predicted image category;

[0140] The class centroid calculation module is used to obtain the class centroids corresponding to different image categories at time t based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1, using the exponential moving average algorithm.

[0141] The inter-class topology graph construction module is used to construct the inter-class topology graph at time t by taking the centroids of different image categories at time t as nodes and the distances between the centroids of different image categories at time t as edges.

[0142] The inter-class uniformity loss construction module is used to take the logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t as the inter-class uniformity loss at time t.

[0143] The intra-class compactness loss construction module is used to construct the intra-class compactness loss at time t based on the distances between all nodes in the inter-class topology graph and their corresponding features at time t.

[0144] The total loss function construction module is used to construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t.

[0145] The parameter update module is used to update the parameters of the student model at time t using the total loss function at time t. Based on the parameters of the student model at time t, the parameters of the teacher model at time t are updated. After training is completed at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

[0146] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0148] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0149] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0150] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An image classification method based on continuous test time adaptation, characterized in that, Includes the following steps: Obtain the image datasets corresponding to T time points within the target region. Based on the image datasets at time t, obtain the features and predicted image category of each image output by the student model at time t; where t = 1, 2, ..., T. Based on the features of each image at time t and the predicted image category, calculate the initial class centroids corresponding to different image categories at time t; Based on the initial centroids of different image categories at time t and the centroids of different image categories at time t-1, the centroids of different image categories at time t are obtained through the exponential moving average algorithm. Using the centroids of different image categories at time t as nodes and the distances between the centroids of different image categories at time t as edges, construct the inter-class topology graph at time t. The logarithm of the average Gaussian potential of all nodes in the inter-class topological graph at time t is used as the inter-class uniformity loss at time t. Based on the distances between all nodes in the inter-class topology graph at time t to their corresponding features, construct the intra-class compactness loss at time t. Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t, a total loss function at time t is constructed. The parameters of the student model at time t are updated using the total loss function at time t. Based on the parameters of the student model at time t, the parameters of the teacher model at time t are updated. After training is completed at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

2. The image classification method based on continuous test time adaptation according to claim 1, characterized in that, The logarithm of the average Gaussian potential of all nodes in the inter-class topological graph at time t is used as the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is: in, Let |V be the inter-class uniformity loss at time t. (t) | represents the number of nodes in the inter-class topology graph at time t, where T is a set parameter, and V (t) Let v be the set of nodes in the inter-class topology graph at time t. i Let i and v be nodes in the inter-class topology graph at time t. j Let i be node j in the inter-class topology graph at time t, i be the index of the first node, j be the index of the second node, and w be the index of the second node. ij Let be the edge between node i and node j in the inter-class topology graph at time t.

3. The image classification method based on continuous test time adaptation according to claim 1, characterized in that, The intra-class compactness loss at time t is constructed by calculating the distances between all nodes in the inter-class topology graph at time t to their corresponding features. The formula for the intra-class compactness loss at time t is as follows: in, Let be the intra-class compactness loss at time t. Let v be the class centroid of image class k in the inter-class topological graph at time t. k edge set, V (t) Let v be the set of nodes in the inter-class topology graph at time t. k Let be the class centroid of image class k in the inter-class topological graph at time t. Let T be the desired value, and z be the set parameter. α Let z be the α-th feature corresponding to image class k in the inter-class topological graph at time t. β Let |z| be the β-th feature corresponding to image class k in the inter-class topological graph at time t. α -z β | 2 For z α With z β The square of the distance between them To calculate the expected value of the squared distances between all possible feature pairs extracted from image class k in the inter-class topology graph at time t, where k is the image class index.

4. The image classification method based on continuous test time adaptation according to claim 1, characterized in that, The total loss function at time t also includes: the inter-hierarchical uniformity loss at time t. The construction process of the inter-hierarchical uniformity loss at time t includes: Based on the distribution of different image categories in the image dataset at time t, the initial weighting terms of each edge in the inter-class topological graph at time t are obtained; Based on the initial weighted terms of each edge in the inter-class topology graph at time t and the weighted terms of each edge in the inter-class topology graph at time t-1, the weighted terms of each edge in the inter-class topology graph at time t are obtained by using the exponential moving average algorithm. By weighting the edges of the inter-class topology graph at time t, the inter-class uniformity loss at time t is adjusted to obtain the inter-hierarchical uniformity loss at time t. Based on the inter-level uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t, a total loss function at time t is constructed.

5. The image classification method based on continuous test time adaptation according to claim 4, characterized in that, The distribution of different image categories in the image dataset at time t is used to obtain the initial weighting terms for each edge in the inter-class topological graph at time t, including: Based on the distribution of each image category in the image dataset at time t, the weight coefficient of each image category in the topology at time t is calculated using the following formula: Based on the weight coefficient of each image category in the topology at time t, calculate the initial weighting terms of each edge in the inter-class topology graph at time t; in, Let be the weight coefficient of image category k in the topology at time t. K represents the number of images belonging to image category k in the image dataset at time t. (t) Let be the number of image categories in the image dataset corresponding to time t, k be the image category index, and ε be a very small constant used to avoid a zero denominator. Let be the initial weighting term of the edge between node i and node j in the inter-class topology graph at time t, where i is the index of the first node and j is the index of the second node. Let be the weight coefficient of the image class corresponding to node i in the inter-class topology graph at time t, within the topological structure. Let be the weight coefficient of the image class corresponding to node j in the inter-class topology graph at time t.

6. The image classification method based on continuous test time adaptation according to claim 4, characterized in that, The inter-class uniformity loss at time t is adjusted by weighting the edges of each edge in the inter-class topology graph at time t, resulting in the inter-hierarchical uniformity loss at time t. The formula for the inter-hierarchical uniformity loss at time t is as follows: in, V represents the loss of inter-stratum homogeneity at time t, where T is a set parameter. (t) Let v be the set of nodes in the inter-class topology graph at time t. i Let i and v be nodes in the inter-class topology graph at time t. j Let i be node j in the inter-class topology graph at time t, i be the index of the first node, j be the index of the second node, and w be the index of the second node. ij Let be the edge between node i and node j in the inter-class topology graph at time t. Let be the weighted term of the edge between node i and node j in the inter-class topology graph at time t.

7. The image classification method based on continuous test time adaptation according to claim 4, characterized in that, The total loss function at time t is constructed based on the inter-level uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss. The total loss function at time t is: in, Let be the total loss function at time t. The symmetric cross-entropy loss at time t, Let be the feature alignment loss at time t. The loss represents the uniformity between different levels at time t. Let λ be the intra-class compactness loss at time t, λ1 be the feature alignment weight, and λ2 be the topological stability weight.

8. The image classification method based on continuous test time adaptation according to claim 1, characterized in that, After performing data augmentation on the image dataset corresponding to time t, we obtain the augmented image dataset corresponding to time t. Input the augmented image dataset corresponding to time t into the feature extractor of the student model to obtain the random augmented features of each image at time t; Based on the random augmentation features of each image at time t and the predicted image category, the initial class centroids corresponding to different image categories at time t are calculated.

9. The image classification method based on continuous test time adaptation according to claim 8, characterized in that, Based on the random enhancement features and predicted image category of each image at time t, the initial class centroids corresponding to different image categories at time t are calculated using the following formula: in, This represents the initial class centroid corresponding to image class k at time t. This represents the index of the maximum value in the output of the a-th image at time t after processing by the student model. Aug(·) indicates data augmentation. Let denot be the a-th image in the image dataset corresponding to time t, argmax(·) be the argmax function, I(·) be the indicator function, and B(·) be the index function. (t) This represents the number of images in the image dataset at time t. This represents the set of images belonging to image category k in the image dataset at time t. This represents the number of images belonging to image category k in the image dataset at time t. Indicates judgment Is it equal to k? If it is, then... Not equal to This represents the features extracted by the student model from the augmented data at time t. Let be the feature extractor for the student model at time t, where a is the image index and k is the image category index.

10. An image classification system based on continuous test time adaptation, characterized in that, include: The data acquisition module is used to acquire image datasets corresponding to T time points within the target area. Based on the image datasets corresponding to time point t, it acquires the features and predicted image category of each image output by the student model at time point t; where t = 1, 2…T. The initial class centroid calculation module is used to calculate the initial class centroid corresponding to different image categories at time t based on the features of each image at time t and the predicted image category; The class centroid calculation module is used to obtain the class centroids corresponding to different image categories at time t based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1, using the exponential moving average algorithm. The inter-class topology graph construction module is used to construct the inter-class topology graph at time t by taking the centroids of different image categories at time t as nodes and the distances between the centroids of different image categories at time t as edges. The inter-class uniformity loss construction module is used to take the logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t as the inter-class uniformity loss at time t. The intra-class compactness loss construction module is used to construct the intra-class compactness loss at time t based on the distances between all nodes in the inter-class topology graph and their corresponding features at time t. The total loss function construction module is used to construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross-entropy loss, and feature alignment loss at time t. The parameter update module is used to update the parameters of the student model at time t using the total loss function at time t. Based on the parameters of the student model at time t, the parameters of the teacher model at time t are updated. After training is completed at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

Images