Adaptive image classification method and system based on continuous test time

Abstract

The invention relates to the technical field of computer vision, in particular to an adaptive image classification method and system based on continuous test time. According to the method, the class centroids corresponding to different image classes serve as nodes, the distances between the class centroids serve as edges, the dynamically updated inter-class topological graph is constructed, the stability of an inter-class topological structure is ensured, the logarithm of average Gaussian potential of all nodes in the inter-class topological graph serves as inter-class uniformity loss, inter-class topological collapse is prevented, and the method has the advantages of being high in robustness and high in robustness. According to the distribution of different image categories, a batch imbalance topological weighting mechanism is provided to adapt to the distribution imbalance of the image categories in the batch, and the inter-level uniformity loss is obtained; intra-class compactness loss is introduced, and the compactness of the features is enhanced; topological consistency loss, symmetric cross entropy loss and feature alignment loss jointly construct a total loss function, and model parameters are optimized through online iteration to realize self-adaption under continuous domain change. According to the invention, the image classification precision of the model in the dynamic target domain is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image classification method and system based on continuous test time adaptation. Background Art

[0002] In recent years, test-time adaptation (TTA) has been extensively studied. This unsupervised technique aims to adapt a model trained in a source domain to a new target domain during inference without requiring access to the source image data. The emergence of TTA opens up the possibility of applying models to scenarios with varying image data distributions, greatly expanding the model's scope of application and demonstrating potential application value in numerous fields. For example, in autonomous driving, images captured by vehicle sensors are affected by a variety of factors, including weather, lighting conditions, and traffic conditions. With TTA, autonomous driving systems can automatically adjust model parameters in real time based on images captured by sensors in complex environments, ensuring the vehicle can accurately identify key image information such as roads, pedestrians, and traffic signals, thereby ensuring driving safety.

[0003] With the continuous advancement of technology, continuous test-time adaptation (CTTA), as an online continuous form of test-time adaptation, has begun to attract widespread attention. CTTA focuses on continuously adjusting the model so that it can adapt to the ever-changing distribution of image data in the target domain, without relying on the source image data. This feature allows the model to continuously adapt to new image environments in many practical unsupervised scenarios. Currently, mainstream continuous test-time adaptation methods rely heavily on pseudo-labeling technology, among which methods based on the mean teacher (MT) structure are particularly effective. For images in the target domain, the mean teacher generates pseudo-labels and, by optimizing the consistency loss between the pseudo-labels and the model predictions, drives the model to continuously adapt to the target domain.

[0004] However, current models using continuous test-time adaptive methods often experience a gradual decline in performance when processing target domain image data. This is primarily due to two key factors. First, the balance of inter-class features is easily disrupted. Due to significant differences in the distribution of target domain image data and source domain image data, the original inter-class feature balance in the source domain is disrupted. For example, source domain image data may exhibit specific distribution patterns in feature dimensions such as color and shape, with clear distinctions between different image categories based on these features. However, when faced with target domain image data, variations in shooting environment and image style can disrupt the relative relationships between inter-class features. Feature combinations that previously effectively distinguished different image categories no longer apply in the target domain, making it difficult for the model to accurately determine image categories. Second, target domain image data evolves dynamically. As the model continues to process target domain image data, the uniformity of the feature distribution deteriorates. Data that was once tightly clustered and easily identified as belonging to the same class becomes dispersed, blurring the classification boundaries and making it increasingly difficult for the model to accurately distinguish between image samples of different classes. This leads to the accumulation of errors within the model, severely impairing image classification performance. Incorrect pseudo-labels mislead the model parameter update, further exacerbating the performance degradation and causing the model to have insufficient classification accuracy when faced with constantly changing target domain image data. Summary of the Invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the defects that when a model using a continuous test time adaptive method processes target domain image data, the difference in data distribution between the target domain and the source domain causes the balance of features between classes to be broken, and the uniformity of feature distribution deteriorates, making the classification boundaries blurred, resulting in insufficient classification accuracy of the model when facing constantly changing target domain image data.

[0006] To solve the above technical problems, the present invention provides an image classification method based on continuous test time adaptation, comprising the following steps:

[0007] Obtain image datasets corresponding to T time periods within the target area. Based on the image dataset corresponding to time period t, obtain the features and predicted image category of each image output by the student model at time period t; where t = 1, 2…T;

[0008] Based on the features of each image at time t and the predicted image category, calculate the initial class centroid corresponding to different image categories at time t;

[0009] Based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1, the class centroids corresponding to different image categories at time t are obtained through the exponential moving average algorithm;

[0010] The centroids of different image categories at time t are used as nodes, and the distances between the centroids of different image categories at time t are used as edges to construct an inter-class topology graph at time t.

[0011] The logarithm of the average Gaussian potential of all node pairs in the inter-class topology graph at time t is used as the inter-class uniformity loss at time t;

[0012] Based on the distances between all pairs of corresponding features in the inter-class topology graph at time t, the intra-class compactness loss at time t is constructed;

[0013] Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t, the total loss function at time t is constructed;

[0014] Through the total loss function at time t, the parameters of the student model at time t are updated. Based on the parameters of the student model at time t, the parameters of the teacher model at time t are updated. After completing the training at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

[0015] Preferably, the logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t is used as the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is:

[0016]

[0017] in, is the inter-class uniformity loss at time t, |V (t) | is the number of nodes in the inter-class topology graph at time t, T is the set parameter, V (t) is the node set of the inter-class topology graph at time t, v i is the node i, v in the inter-class topology graph at time t j is the node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, and w ij is the edge between node i and node j in the inter-class topology graph at time t.

[0018] Preferably, the distances between all nodes and corresponding features in the inter-class topology graph at time t are used to construct the intra-class compactness loss at time t. The formula of the intra-class compactness loss at time t is:

[0019]

[0020] in, is the intra-class compactness loss at time t, is the centroid v of image category k in the inter-class topology graph at time t k The edge set of V (t)is the node set of the inter-class topology graph at time t, v k is the centroid of image category k in the inter-class topology graph at time t, is the expectation, T is the setting parameter, z α is the αth feature corresponding to image category k in the inter-class topology graph at time t, z β is the βth feature corresponding to image category k in the inter-class topology graph at time t, |z α -z β | 2 For z α With z β The square of the distance between To find the mathematical expectation of the square of the distance of all possible feature pairs extracted in the image category k in the inter-class topology graph at time t, k is the image category index.

[0021] Preferably, the total loss function at time t further includes: inter-class uniformity loss at time t, and the construction process of the inter-class uniformity loss at time t includes:

[0022] Based on the distribution of different image categories in the image dataset corresponding to time t, the initial weighted items of each edge in the inter-class topological graph at time t are obtained;

[0023] Based on the initial weighted items of each edge in the inter-class topology graph at time t and the weighted items of each edge in the inter-class topology graph at time t-1, the weighted items of each edge in the inter-class topology graph at time t are obtained by the exponential moving average algorithm;

[0024] By weighting the edges in the inter-class topology graph at time t, the inter-class uniformity loss at time t is weightedly adjusted to obtain the inter-class uniformity loss at time t;

[0025] Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t, the total loss function at time t is constructed.

[0026] Preferably, obtaining the initial weighted items of each edge in the inter-class topology graph at time t based on the distribution of different image categories in the image data set corresponding to time t includes:

[0027] Based on the distribution of each image category in the image dataset corresponding to time t, the weight coefficient of each image category in the topological structure at time t is calculated. The formula is:

[0028]

[0029] Based on the weight coefficient of each image category in the topological structure at time t, calculate the initial weighted terms of each edge in the inter-class topological graph at time t;

[0030]

[0031] in, is the weight coefficient of image category k in the topological structure at time t, K represents the number of images belonging to image category k in the image dataset corresponding to time t. (t) is the number of image categories in the image dataset corresponding to time t, k is the image category index, ∈ is a very small constant used to avoid the denominator being zero, is the initial weighted item of the edge between node i and node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, is the weight coefficient of the image category corresponding to node i in the inter-class topology graph at time t in the topological structure, is the weight coefficient of the image category corresponding to node j in the inter-class topology graph at time t in the topological structure.

[0032] Preferably, the inter-class uniformity loss at time t is weightedly adjusted by the weighted terms of each edge in the inter-class topology graph at time t to obtain the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is:

[0033]

[0034] in, is the uniformity loss between layers at time t, T is the setting parameter, V (t) is the node set of the inter-class topology graph at time t, v i is the node i, v in the inter-class topology graph at time t j is the node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, and w ij is the edge between node i and node j in the inter-class topology graph at time t, is the weighted term of the edge between node i and node j in the inter-class topology graph at time t.

[0035] Preferably, the total loss function at time t is constructed based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t. The total loss function at time t is:

[0036]

[0037] in, is the total loss function at time t, is the symmetric cross entropy loss at time t, is the feature alignment loss at time t, is the uniformity loss between classes at time t, is the intra-class compactness loss at time t, λ1 is the feature alignment weight, and λ2 is the topological stability weight.

[0038] Preferably, after performing data enhancement on the image dataset corresponding to time t, an enhanced image dataset corresponding to time t is obtained;

[0039] Input the enhanced image dataset corresponding to time t into the feature extractor of the student model to obtain the random enhanced features of each image at time t;

[0040] Based on the randomly enhanced features and predicted image categories of each image at time t, the initial class centroids corresponding to different image categories at time t are calculated.

[0041] Preferably, the initial class centroids corresponding to different image categories at time t are calculated based on the random enhancement features of each image at time t and the predicted image category, and the calculation formula is:

[0042]

[0043] in, represents the initial class centroid corresponding to the image category k at time t, It represents the index corresponding to the maximum value in the output result after the a-th image at time t is processed by the student model. Aug(·) represents data enhancement. represents the ath image in the image dataset corresponding to time t, argmax(·) is the argmax function, I(·) represents the indicator function, B (t) Indicates the number of images in the image dataset corresponding to time t, represents the set of images belonging to image category k in the image dataset corresponding to time t, represents the number of images belonging to image category k in the image dataset corresponding to time t, Express judgment Is it equal to k? If so, then If not equal to Represents the features extracted by the student model for the enhanced data at time t, is the feature extractor of the student model at time t, a is the image index, and k is the image category index.

[0044] The present invention also provides an image classification system based on continuous test time adaptation, comprising:

[0045] The data acquisition module is used to obtain image datasets corresponding to T time periods in the target area, and based on the image dataset corresponding to time period t, obtain the features and predicted image category of each image output by the student model at time period t; where t = 1, 2…T;

[0046] The initial class centroid calculation module is used to calculate the initial class centroid corresponding to different image categories at time t based on the features of each image at time t and the predicted image category;

[0047] The class centroid calculation module is used to obtain the class centroids corresponding to different image categories at time t based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1 by using the exponential moving average algorithm;

[0048] The inter-class topology graph construction module is used to construct the inter-class topology graph at time t by taking the class centroids corresponding to different image categories at time t as nodes and the distances between the class centroids corresponding to different image categories at time t as edges;

[0049] The inter-class uniformity loss construction module is used to take the logarithm of the average Gaussian potential of all node pairs in the inter-class topology graph at time t as the inter-class uniformity loss at time t;

[0050] The intra-class compactness loss construction module is used to construct the intra-class compactness loss at time t based on the distances between all node pairs corresponding to features in the inter-class topology graph at time t;

[0051] The total loss function construction module is used to construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss;

[0052] The parameter update module is used to update the parameters of the student model at time t through the total loss function at time t, and update the parameters of the teacher model at time t based on the parameters of the student model at time t. After completing the training at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

[0053] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0054] The present invention describes an image classification method and system based on continuous test time adaptation. Taking into account that target domain image data will continue to dynamically change over time during continuous test time adaptation, the present invention uses the class centroids corresponding to different image categories at the previous moment and the initial class centroids corresponding to different image categories at the current moment, updating the class centroids at the current moment using an exponential moving average. This method effectively tracks the dynamic changing trends of image data, while retaining the feature information of different image categories in historical data, and continuously optimizes the judgment of the features of each image category. The class centroids corresponding to different image categories are used as nodes, and the distances between class centroids are used as edges. An inter-class topology graph reflecting the geometric relationships between classes is constructed, and this inter-class topology graph is updated based on the data sets corresponding to different moments. This ensures the structural stability of the feature space and avoids erroneous judgments caused by confusion between inter-class features. Different from traditional inter-class uniformity loss, which is mostly based on simple distance or single similarity index and cannot fully reflect the overall distribution uniformity and topological structure between classes, the present invention is based on a dynamically changing inter-class topological graph, and uses the average Gaussian potential of all node pairs in the inter-class topological graph as the inter-class uniformity loss, minimizing the Gaussian potential between class centroids, ensuring the uniform distribution of centroids, and reorganizing the originally chaotic relative relationship between class features, so that the model can maintain a consistent topological structure when facing a dynamically changing target domain. The present invention also introduces an intra-class compactness loss. Since traditional intra-class loss methods often only focus on simple distance metrics between features and do not consider the overall structure and dynamic changes of feature distribution, the present invention uses topological information such as class centroids and their edge sets to perform expected calculations on the distances between intra-class feature pairs, and further strengthens the constraints on intra-class compactness through exponential and logarithmic transformations. The intra-class compactness loss of the present invention can effectively deal with the feature distribution problems caused by the dynamic evolution of target domain image data, indirectly support inter-class topological stability by optimizing the compactness of feature distribution, enhance the compactness of features, and avoid blurred classification boundaries caused by excessive feature dispersion. From the perspectives of between classes and within classes, the present invention designs inter-class uniformity loss and intra-class compactness loss respectively. By optimizing the model parameters through online iteration, the model can effectively avoid erroneous pseudo-labels and bias updates caused by unstable inter-class topology and poor feature distribution when facing constantly changing target domain data, significantly improving the classification accuracy of the model when facing constantly changing target domain image data.

[0055] Since the continuous test time adaptive method often faces the problem of limited image data and distribution differences between the target domain and the source domain image data, the present invention uses the characteristics of the randomly enhanced data to calculate the initial class centroid. The characteristics of the randomly enhanced data are diverse. Compared with directly using the original data, the calculated initial class centroid is more stable and representative, laying the foundation for calculating the inter-class topology map at the current moment, so that the model can effectively reduce the impact of distribution offset when processing target domain image data and improve the classification accuracy of target domain images.

[0056] The present invention takes into account that the image categories and number of images in different batches of data may be different. This imbalance will gradually destroy the inter-class topology, thereby affecting the prediction performance of the model. According to the distribution of different image categories in the image dataset, the class centroid weights in the inter-class uniformity loss are dynamically adjusted, and the inter-class distance is optimized to ensure that the stability of the topological structure can be maintained when the image category distribution is uneven. The inter-class uniformity loss is further obtained. Through the inter-class uniformity loss, the model's adaptability to the distribution differences of data of each image category is improved, and the accuracy and stability of the model in classifying target domain images are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0058] Figure 1 The present invention is a flowchart of the steps of an image classification method based on continuous test time adaptation.

[0059] Figure 2 This is a structural diagram of an image classification method based on continuous test time adaptation of the present invention.

[0060] Figure 3 This is a schematic diagram comparing the topological stability of the present invention with other methods.

[0061] Figure 4 It is a schematic diagram of the comparison results of the dimensionality reduction distribution of the output features of the model after using the present invention and other methods.

[0062] Figure 5 It is a schematic diagram showing the comparison results of the distribution of the output features of the model after using the present invention on a two-dimensional unit hypersphere and other methods. DETAILED DESCRIPTION

[0063] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0064] In practice, model training and test data often come from different distributions, resulting in a domain shift problem. For example, the training data might be images taken during the day, while the test data might be images taken at night. Traditional domain adaptation methods typically require adaptive training using data from the target domain before testing. However, this is impractical in many real-world applications because the target domain data may only appear during testing and may be constantly changing. Therefore, test-time adaptation (TTA) methods were introduced.

[0065] As technological research advances, the Continuous Test Time Adaptation (CTTA) method has been proposed. However, when processing target domain image data, due to significant differences in the distribution of target and source domain image data, the original balance of inter-class features in the source domain is disrupted, leading to confusion in the relative relationships between inter-class features. Feature combinations that were previously effective in distinguishing different image categories are no longer applicable in the target domain, making it difficult for the model to accurately determine image categories. Furthermore, target domain image data is constantly evolving. As the model continues to process target domain image data, the uniformity of feature distribution deteriorates. Data that should have been tightly clustered and easily identified as belonging to the same class becomes dispersed, blurring the classification boundaries and making it increasingly difficult for the model to accurately distinguish between image samples of different image categories.

[0066] To this end, the present invention proposes an image classification method based on continuous test time adaptation.

[0067] Reference Figure 1 As shown, this embodiment provides an image classification method based on continuous test time adaptation, including the following steps:

[0068] The present invention uses existing source data The pre-trained model is parameterized, where For datasets in existing source data, for The corresponding label set, after a good training, the goal of continuous test time adaptation is to respond to the changing target domain during the test phase. Without accessing any source data.

[0069] like Figure 2 As shown, Figure 2 This is a structural diagram of an image classification method based on continuous test time adaptation of the present invention.

[0070] Step S1: Obtain image datasets corresponding to T time periods within the target area, and based on the image dataset corresponding to time period t, obtain the features and predicted image category of each image output by the student model at time period t; where t = 1, 2…T;

[0071] like Figure 3 As shown, Figure 3 This figure compares the topological stability of our method with other methods. Our method aims to achieve continuous test time adaptation by dynamically maintaining inter-class topological relationships. Its core approach is to use class centroids as proxies for image categories and class centroid distances as proxies for inter-class topological distances, thereby constructing inter-class topological structures.

[0072] First, define the class topology in, Represents the set of centroids of different image categories, v K Represents the class centroid corresponding to the K-th image category, where K is the number of image categories. For example, in an image classification task, if we want to distinguish between two categories of images, cats and dogs, each category of images has its own corresponding class centroid, which represents the typical position of this category of images in the feature space. The edge set representing the neighborhood relationship between these class centroids, each edge e ij Assign a weight ω ij , w ij The class centroid v used to represent the image category i i and the class centroid v of image category j j The Euclidean distance (L2 distance) between them is used to describe the relative position relationship of different image categories in the feature space by constructing the topological relationship between the class centroids.

[0073] In traditional machine learning and computer vision, calculating class centroids is a basic and critical operation, often used in tasks such as classification and clustering. When the K-means clustering algorithm is applied to image data, for each clustered image category, the pixel values of the images belonging to that image category in color channel dimensions such as RGB are directly averaged to obtain the centroid representation of the image category. This method can simply and quickly summarize the features of image categories in scenarios with static data sets and relatively stable features. The existing technology calculates the mean of all raw data features in each image category to obtain the class centroid corresponding to each image category. However, in the field of continuous test time adaptation, the model needs to be able to adapt to new data when facing dynamically changing data. Therefore, relying solely on raw features to calculate the class centroid of each image category cannot fully and accurately capture the true distribution of image categories at different times.

[0074] The present invention constructs a dynamically updated inter-class topology map based on class centroids, calculates class centroids using data-enhanced features, and updates class centroids using exponential moving averages to ensure the stability of the inter-class topology. The specific construction process of the class topology map is shown in steps S2-S4.

[0075] Step S2: Based on the features and predicted categories of each image at time t, calculate the initial class centroids corresponding to different image categories at time t;

[0076] In this embodiment, preferably, after performing data enhancement on the image dataset corresponding to time t, an enhanced image dataset corresponding to time t is obtained;

[0077] Input the enhanced image dataset corresponding to time t into the feature extractor of the student model to obtain the random enhanced features of each image at time t;

[0078] Based on the randomly enhanced features of each image at time t, the initial class centroids corresponding to different image categories at time t are calculated.

[0079] In this embodiment, preferably, the initial class centroids corresponding to different image categories at time t are calculated based on the random enhancement features of each image at time t and the predicted image category of each image at time t, and the calculation formula is:

[0080]

[0081]

[0082] in, represents the initial class centroid corresponding to the image category k at time t, It represents the index corresponding to the maximum value in the output result after the a-th image at time t is processed by the student model. Aug(·) represents data enhancement. represents the ath image in the image dataset corresponding to time t, argmax(·) is the argmax function, I(·) represents the indicator function, B (t) Indicates the number of images in the image dataset corresponding to time t, represents the set of images belonging to image category k in the image dataset corresponding to time t, represents the number of images belonging to image category k in the image dataset corresponding to time t, Express judgment Is it equal to k? If so, then If not equal to Represents the features extracted by the student model for the enhanced data at time t, is the feature extractor of the student model at time t, a is the image index, k is the image category index, is the student model at time t.

[0083] in, Indicates the index corresponding to the maximum value in the output result after the a-th image is processed by the student model at time t. For example, if the student model is for The output result is [0.2, 0.5, 0.3], then

[0084] At time t, the present invention uses the random enhancement features of the current batch to improve the reliability of the class centroid, which can eliminate the instability of the class centroid caused by limited samples or distribution shift, thereby generating more stable and reliable pseudo labels.

[0085] Step S3: Based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1, the class centroids corresponding to different image categories at time t are obtained by using an exponential moving average algorithm;

[0086] In this embodiment, preferably, the class centroids corresponding to different image categories at time t are obtained by using an exponential moving average algorithm based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1. The formula is:

[0087]

[0088] in, is the centroid of the image category k at time t, is the centroid of the image category k at time t-1, represents the initial class centroid corresponding to the image category k at time t, γ is a weighting parameter that controls the influence of the centroid of the previous batch, and k is the image category index.

[0089] In the context of different batch sizes and unbalanced data distribution, the update method of the centroid information of the batches before and after the exponential moving average fusion is used to enable the model to effectively maintain a stable inter-class topology structure, avoiding the disorder of the topology structure caused by data imbalance or batch differences. This strategy ensures that the inter-class topology is stable in different batch sizes and unbalanced data distribution, and is effective in image scarcity or even single image (B t =1), the random enhancement features of the current image are used as a proxy for updating the class centroid, ensuring that the update of the class centroid is still based on a basis, maintaining the dynamic update and improvement of the topology map, and effectively tracking the dynamic change trend of the image data. While retaining the feature information of different image categories in the historical data, the judgment of the features of each image category is continuously optimized.

[0090] By executing a specific calculation process on the image data sets corresponding to different moments, we can effectively eliminate the problem of unstable class centroids caused by limited images or distribution offsets. We use the data augmentation operation Aug(.) to increase data diversity, thereby reducing the impact of limited images or distribution offsets on class centroid calculations, laying the foundation for obtaining relatively reliable class centroids, which will be used to subsequently generate stable pseudo-labels.

[0091] Step S4: The class centroids corresponding to different image categories at time t are used as nodes, and the distances between the class centroids corresponding to different image categories at time t are used as edges to construct an inter-class topology graph at time t;

[0092] In each batch processing process, the inter-class topology covers all image categories, and the class topology map is continuously updated during the entire test process. Specifically, the present invention converts the class topology map Initialized as an empty graph, it will be iteratively filled and updated with the class centroids as new batches are processed. The class centroid updates depend not only on the initial class centroids corresponding to different image categories in the current batch data, but also on the image class centroids of the image class. It also contains the class centroids corresponding to different image categories from the previous batch Then the corresponding class centroids of different image categories in the current batch data are obtained after updating.

[0093] Step S5: The logarithm of the average Gaussian potential of all node pairs in the inter-class topology graph at time t is used as the inter-class uniformity loss at time t;

[0094] In traditional graph neural networks, inter-class uniformity loss uses a simple approach based on node connection density. This approach measures inter-class uniformity by calculating the ratio of the number of direct connections between nodes of different image categories to the total number of nodes. While this approach is relatively intuitive, it has significant flaws. It only considers direct connections between nodes, ignoring the strength of these connections and the relative positions of nodes in the network. In the related field of manifold learning for image dimensionality reduction, an inter-class uniformity loss based on the Euclidean distance between data points is often used to ensure a uniform distribution of data from different image categories in low-dimensional space. For example, this approach minimizes the average Euclidean distance between data points of different image categories in low-dimensional space to achieve uniform distribution between classes. However, this approach fails to account for the complex nonlinear relationships between data points. In image data, features of different image categories often exhibit complex nonlinear distributions. Simple Euclidean distance cannot accurately capture these nonlinear characteristics, making it difficult to achieve a truly uniform distribution between classes when processing complex image data.

[0095] In continuous test time adaptation, the target domain image data has unique dynamic change characteristics, the data distribution changes continuously over time, and there are significant differences between the source domain and target domain data. The inter-class uniformity loss method in traditional graph neural networks cannot adapt to the complex feature evolution of image data in continuous test time adaptation because it does not consider the dynamic changes of data and inter-domain differences. The inter-class uniformity loss based on Euclidean distance in manifold learning cannot effectively adjust to meet the model's accurate requirements for inter-class uniformity in the face of nonlinear dynamic changes in image data in continuous test time adaptation.

[0096] Therefore, the present invention proposes a unique inter-class uniformity loss method. In this embodiment, preferably, the logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t is used as the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is:

[0097]

[0098] in, is the inter-class uniformity loss at time t, is the number of nodes in the inter-class topology graph at time t, T is a setting parameter. In this embodiment, T is set to 2, V (t)is the node set of the inter-class topology graph at time t, v i is the node i, v in the inter-class topology graph at time t j is the node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, and w ij is the edge between node i and node j in the inter-class topology graph at time t.

[0099] To address the problem of topological instability, the present invention separates the centroids of different image categories through inter-class uniformity loss, minimizes the Gaussian potential energy between class centroids to maintain inter-class topological stability, reduces the propagation of inter-class errors, maintains a stable inter-class distance, prevents inter-class topological collapse, and thus maintains a coherent topological structure.

[0100] Step S6: Based on the distances between all node pairs corresponding to features in the inter-class topology graph at time t, construct the intra-class compactness loss at time t;

[0101] The stability of inter-class topology depends on the inter-class distribution and feature concentration. Overly concentrated features can ignore sub-class variations and noise, while dispersed features hinder stable intra-class learning. Traditional intra-class compactness losses typically consider either intra-class compactness or inter-class discrimination separately. For example, the common center loss only calculates the distance between sample features and a fixed class center to bring similar samples closer to the class center. During training, the center loss typically determines the class center based on source domain data. When applied to the target domain, due to changes in data distribution, the original class center may no longer be applicable to the target domain data. Furthermore, in the field of continuous test-time adaptation, target domain image data often changes over time and is prone to outliers. The center loss only considers the distance between samples and class centers. When outliers or noisy data appear, these data points will have a significant impact on the calculation of the class center, which in turn affects the calculation of the entire intra-class compactness loss, thereby misleading model training and reducing the model's generalization ability and classification accuracy.

[0102] Therefore, in order to further maintain a stable inter-class topology, the present invention further compresses and unifies features, introduces intra-class compactness loss, and (t) The feature pairs within the class are pairwise homogenized, taking into account the mathematical expectation of the square of the distances of all possible feature pairs to achieve a more compact and uniform distribution of inter-class features.

[0103] In this embodiment, preferably, the distances between all nodes and corresponding features in the inter-class topology graph at time t are used to construct the intra-class compactness loss at time t. The formula for the intra-class compactness loss at time t is:

[0104]

[0105] in, is the intra-class compactness loss at time t, is the centroid v of image category k in the inter-class topology graph at time t k The edge set of V (t) is the node set of the inter-class topology graph at time t, v k is the centroid of image category k in the inter-class topology graph at time t, is the expectation, T is the setting parameter, z α is the αth feature corresponding to image category k in the inter-class topology graph at time t, z β is the βth feature corresponding to image category k in the inter-class topology graph at time t, |z α -z β | 2 For z α With z β The square of the distance between To find the mathematical expectation of the square of the distance of all possible feature pairs extracted in the image category k in the inter-class topology graph at time t, k is the image category index.

[0106] By introducing the intra-class compactness loss at time t, the compactness of feature distribution is optimized to indirectly support inter-class topological stability, enhance the compactness of features, and avoid blurred classification boundaries caused by excessive feature dispersion.

[0107] The present invention addresses dynamic domain adaptation scenarios by considering both inter-class and feature-based approaches separately. If this feature separation is not performed and all features are treated uniformly, the inter-class topology is likely to be distorted. This is because, in a uniform treatment, the inter-class boundaries tend to blur, reducing the ability to distinguish between different image categories. This is particularly unfavorable in dynamic environments where class distributions constantly change over time, leading to a significant decrease in model performance.

[0108] The present invention can effectively avoid the above problems by separating classes and features for targeted processing. In terms of inter-class, a dynamically updated inter-class topological map is constructed to accurately characterize the relative positions of different image categories in the feature space with a stable inter-class topological relationship, avoiding the distortion of the inter-class topological structure due to feature confusion, so that the model can clearly grasp the inter-class boundaries when facing dynamically changing class distributions. At the intra-class level, intra-class compactness loss is introduced to enhance the compactness of features and prevent the feature distribution from being too dispersed, resulting in blurred classification boundaries. In this way, the present invention can better adapt to the complex and changing situations in dynamic domain adaptation scenarios, significantly improve the robustness and processing accuracy of the model in dynamic environments, and ensure the stability and improvement of model performance.

[0109] Step S7: Construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t;

[0110] At time t, the teacher model generates pseudo labels to help the learning process of the student model. The present invention calculates the symmetric cross entropy loss (SCE) between the output of the student model and the pseudo labels. The formula of the symmetric cross entropy loss at time t is:

[0111]

[0112] in, is the symmetric cross entropy loss at time t, k is the image category index, K t is the number of image categories of the image dataset corresponding to time t, is the predicted probability of the teacher model that the input x belongs to the image category k at time t, is the predicted probability of the student model at time t that the input x belongs to image category k.

[0113] Furthermore, while maintaining feature consistency, the feature distribution between aligns, and the formula for feature alignment loss at time t is:

[0114]

[0115] in, is the feature alignment loss at time t, Indicates that at time t, the positive sample pair (x, y) follows the distribution p pos (t) The expected operation under is the square of the L2 norm, is the feature extractor of the student model at time t, x and y represent the features obtained from the input subjected to different forms of data augmentation, and p pos (t) Represents the distribution of positive sample pairs (x, y) at time t, that is, the probability distribution of these positive sample pairs appearing at time t.

[0116] Step S8: Update the parameters of the student model at time t through the total loss function at time t. Based on the parameters of the student model at time t, update the parameters of the teacher model at time t. After completing the training at time T, use the teacher model at time T with updated parameters as the target image classification model to classify the target domain image.

[0117] Different batches of data often cover different image categories, and the number of samples contained in these image categories varies significantly. In an unlabeled data environment, this imbalance in sample size will gradually damage the inter-class topological structure. On the surface, the feature distribution seems to be uniform, but in fact there is a high degree of imbalance inside it, which will undoubtedly have a negative impact on the predictive performance of the model. The present invention innovatively integrates the inter-class topological distance with the true class distribution information output by the model, and optimizes the inter-class distance by dynamically adjusting the class centroid weights based on the image category distribution within the batch. Even in complex situations where the image category distribution is uneven, the stability of the inter-class topological structure can be effectively ensured, thereby significantly improving the performance of the model in practical applications. The specific steps are as follows:

[0118] In this embodiment, preferably, the total loss function at time t further includes: inter-layer uniformity loss at time t, and the construction process of the inter-layer uniformity loss at time t includes:

[0119] Based on the distribution of different image categories in the image dataset corresponding to time t, the initial weighted items of each edge in the inter-class topological graph at time t are obtained;

[0120] In this embodiment, preferably, obtaining the initial weighted item of each edge in the inter-class topology graph at time t based on the distribution of different image categories in the image dataset corresponding to time t includes:

[0121] Based on the distribution of each image category in the image dataset corresponding to time t, the weight coefficient of each image category in the topological structure at time t is calculated. The formula is:

[0122]

[0123] The batch imbalanced topology weighted matrix is composed of the weight coefficients of all image categories in the topological structure at time t. It can effectively reflect the relative importance of different image categories in the topological structure at time t, as well as the weights of each edge in the inter-class topological relationship. It provides a basis for the subsequent weighted adjustment of the inter-class uniformity loss to cope with the impact of the imbalanced sample size of image categories in different batches on model performance.

[0124] Based on the weight coefficient of each image category in the topological structure at time t, calculate the initial weighted terms of each edge in the inter-class topological graph at time t;

[0125]

[0126] in, is the weight coefficient of image category k in the topological structure at time t, K represents the number of images belonging to image category k in the image dataset corresponding to time t. (t)is the number of image categories in the image dataset corresponding to time t, k is the image category index, ∈ is a very small constant used to avoid the denominator being zero, is the initial weighted item of the edge between node i and node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, is the weight coefficient of the image category corresponding to node i in the inter-class topology graph at time t in the topological structure, is the weight coefficient of the image category corresponding to node j in the inter-class topology graph at time t in the topological structure.

[0127] Based on the initial weighted items of each edge in the inter-class topology graph at time t and the weighted items of each edge in the inter-class topology graph at time t-1, the weighted items of each edge in the inter-class topology graph at time t are obtained by the exponential moving average algorithm;

[0128] In this embodiment, preferably, the initial weighted item of each edge in the inter-class topology graph at time t and the weighted item of each edge in the inter-class topology graph at time t-1 are used to obtain the weighted item of each edge in the inter-class topology graph at time t by an exponential moving average algorithm, and the formula is:

[0129]

[0130] in, is the weighted term of the edge between node i and node j in the inter-class topology graph at time t, ω is the weight parameter, is the weighted term of the edge between node i and node j in the inter-class topology graph at time t-1, is the initial weighted term of the edge between node i and node j in the inter-class topology graph at time t.

[0131] During model training, the data distribution may change over time or across different batches of data. The weighted matrix, composed of the weighted terms of each edge in the inter-class topology graph at time t, is used to adjust the importance of each image category or connection relationship in the model and has a critical impact on model performance. By calculating the weighted terms of each edge in the inter-class topology graph at time t using an exponential moving average, we not only incorporate the latest weights reflected by the current data, but also retain the weighted features corresponding to past data distributions. This allows the weighted matrix to account for both the continuity and variability of data distribution over time, preventing the model from deviating excessively from previously learned useful information due to local features of the current data, thereby improving the model's stability and adaptability under different data distribution conditions.

[0132] By weighting the edges in the inter-class topology graph at time t, the inter-class uniformity loss at time t is weightedly adjusted to obtain the inter-class uniformity loss at time t;

[0133] In this embodiment, preferably, the inter-class uniformity loss at time t is weightedly adjusted by the weighted terms of each edge in the inter-class topology graph at time t to obtain the inter-layer uniformity loss at time t. The inter-layer uniformity loss formula at time t is:

[0134]

[0135] in, is the uniformity loss between layers at time t, T is the setting parameter, V (t) is the node set of the inter-class topology graph at time t, v i is the node i, v in the inter-class topology graph at time t j is the node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, and w ij is the edge between node i and node j in the inter-class topology graph at time t, is the weighted term of the edge between node i and node j in the inter-class topology graph at time t.

[0136] The present invention applies the gradient stop operation to the initial weighted term of the edge between node i and node j in the inter-class topology graph at time t It keeps it stable during model training, allowing the inter-class uniformity loss at time t to take into account the importance differences between the relationships between different image categories, thereby more accurately reflecting the uniformity of the model's feature distribution between classes, thereby optimizing the model training process and improving model performance.

[0137] Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t, the total loss function at time t is constructed;

[0138] In this embodiment, specifically, the total loss function at time t is constructed based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t. The total loss function at time t is:

[0139]

[0140] in, is the total loss function at time t, is the symmetric cross entropy loss at time t, is the feature alignment loss at time t, is the uniformity loss between classes at time t, is the intra-class compactness loss at time t, λ1 is the feature alignment weight, and λ2 is the topological stability weight.

[0141] The present invention obtains the topological consistency loss at time t by integrating the inter-class uniformity loss at time t and the intra-class compactness loss at time t.

[0142] like Figure 4 As shown, Figure 4 This is a schematic diagram showing the comparison of the dimensionality reduction distribution of the model output features after using the present invention and other methods. Figure 4 The upper left corner shows the dimensionality reduction distribution of the model output features after the source domain data is processed. Figure 4 The upper right corner shows the dimensionality reduction distribution of the model output features after adding Gaussian noise to the data using the Continuous Test Time Adaptation (CTTA) method. Figure 4 The lower left corner shows the dimensionality reduction distribution of the model output features after using the RMT method and adding Gaussian noise to the data. Figure 4 The lower right corner area shows the dimensionality reduction distribution of the model output features after using the image classification method based on continuous test time adaptation (TCA) proposed by the present invention and adding Gaussian noise to the data.

[0143] Depend on Figure 4 It can be seen that the distribution of output features of the model using the present invention for test time adaptation is more concentrated and compact than that of the model using other methods, the boundaries between different image categories are clearer, and a more obvious and stable inter-class topological structure is formed in the feature space. This shows that the present invention can more effectively handle noise interference in the data, enable the model to better adapt to changes in the target domain during the test phase, reduce the possibility of inter-class error propagation, and thus improve the performance and robustness of the model in dynamic environments.

[0144] like Figure 5 As shown, Figure 5 This is a schematic diagram showing the comparison of the distribution of the model output features on a two-dimensional unit hypersphere using the present invention and other methods. Figure 5 In the figure, it is divided into two rows. The first row shows the distribution of the output features of the model using the continuous test time adaptation (CTTA) method on the two-dimensional unit hypersphere when three different types of noise, namely Gaussian noise, snowflake noise, and JPEG noise, are added respectively. The second row shows the distribution of the output features of the model using the image classification method based on continuous test time adaptation (TCA) proposed by the present invention, also after adding Gaussian noise, snowflake noise, and JPEG noise respectively.

[0145] Depend on Figure 5It can be seen that compared with the model using the continuous test time adaptation (CTTA) method, the model using the method of the present invention (TCA) shows a more obvious advantage in the distribution of output features on the two-dimensional unit hypersphere. When facing different types of noise interference, the output feature distribution of the model of the method of the present invention (TCA) is more concentrated, and the degree of aggregation between similar feature points is higher, which shows that the present invention can effectively enhance the compactness of features, making the model more accurate in identifying features of each image category. At the same time, the separation between features of different image categories is more significant, and the area of each image category in the feature space is clearly defined. This means that the dynamically updated inter-class topology map constructed by the present invention effectively stabilizes the inter-class topology structure, reduces inter-class confusion caused by noise interference, and greatly improves the adaptability and robustness of the model in complex noise environments, ensuring that the model can maintain good performance during the test time adaptation process. Compared with other methods, the present invention significantly maintains a stable inter-class topological relationship and obtains more accurate classification performance.

[0146] The present invention is based on online iterative dynamic model parameter optimization and adaptive update. Through the online iterative optimization strategy, each batch of data performs class centroid calculation, topology map update, loss optimization and parameter update in sequence, ensuring that the model can adapt to continuous domain changes in real time and maintain the stability of feature distribution. After online iterative optimization, the model can output stable classification prediction image categories, significantly reduce the classification error rate, improve the robustness and processing accuracy of the model in dynamic environments, ensure that the autonomous driving system can accurately identify key information such as roads, pedestrians, traffic signals, etc., and significantly improve the safety and reliability of autonomous driving.

[0147] Based on Example 1, this Example 2 utilizes an image classification method based on continuous test time adaptation proposed by the present invention to evaluate three benchmark dataset test tasks in image processing, demonstrating the experimental results of the present invention in the field of image processing.

[0148] The three benchmark datasets tested include CIFAR-10-C, CIFAR-100-C, and ImageNet-C. These tasks are designed to evaluate the robustness of machine learning models to corruption and interference in input data. CIFAR10-C extends the CIFAR-10 dataset and consists of 32×32 color images from 10 classes. It includes 15 different types of corruption, each with five severity levels, applied to CIFAR-10 test images, for a total of 10,000 images. CIFAR100-C extends the CIFAR-100 dataset and consists of 32×32 color images from 100 classes. It includes 15 different types of corruption, each with five severity levels, applied to CIFAR-100 test images, for a total of 10,000 images. ImageNet-C extends the ImageNet dataset, which contains over 14 million images across more than 20,000 image categories. ImageNet-C includes 15 different corruptions, each with 5 severity levels. These corruptions are applied to the validation images of ImageNet. The present invention strictly adheres to the CTTA setting and does not access the source data. All comparative experimental models, including CoTTA, TENT, AdaContrast and DSS, are evaluated online based on the five levels of maximum corruption severity across all datasets. Model predictions are first generated before adapting to the current test stream. The present invention uses standard pre-trained WideResNet, ResNeXt-29 and ResNet-50 as source models for CIFAR10-C, CIFAR100-C and ImageNet-C.

[0149] Using an adaptive threshold for pseudo-label filtering, the present invention successfully reduced the average error on CIFAR100-C and ImageNet-C from 32.5% to 29.7%, and from 66.8% to 59.3%, respectively, compared to CoTTA. To further investigate the effectiveness of the present invention over baselines, its adaptation performance on 10 different sequences was also evaluated on ImageNet-C. Compared to CoTTA, the average error reduction over 10 different sequences was 3%, demonstrating that the present invention's method is more robust to the order of the target domain sequence. Table 1 shows the experimental results of the present invention and different methods on the CIFAR10-C dataset.

[0150] Table 1

[0151]

[0152] As shown in Table 2, Table 2 shows the experimental results of the present invention and different methods on the CIFAR100-C dataset.

[0153] Table 2

[0154]

[0155] As shown in Table 3, Table 3 shows the experimental results of the present invention and different methods on the ImageNet-C dataset.

[0156] Table 3

[0157]

[0158]

[0159] Example 2 uses the online domain change continuous learning method proposed in the present invention to evaluate the three benchmark dataset test tasks of CIFAR-10-C, CIFAR-100-C and ImageNet-C in the field of image processing. The experimental results show that in the comparative experiments with many other methods (such as TENT, Ada, DSS, CoTTA, etc.), the comprehensive performance of the present invention on the three datasets is also better, demonstrating the excellent robustness of the method in dealing with input data corruption and interference, and can effectively improve the performance of machine learning models in image processing tasks.

[0160] The third embodiment provides an image classification system based on continuous test time adaptation, including:

[0161] The data acquisition module is used to obtain image datasets corresponding to T time periods in the target area, and based on the image dataset corresponding to time period t, obtain the features and predicted image category of each image output by the student model at time period t; where t = 1, 2…T;

[0162] The initial class centroid calculation module is used to calculate the initial class centroid corresponding to different image categories at time t based on the features of each image at time t and the predicted image category;

[0163] The class centroid calculation module is used to obtain the class centroids corresponding to different image categories at time t based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1 by using the exponential moving average algorithm;

[0164] The inter-class topology graph construction module is used to construct the inter-class topology graph at time t by taking the class centroids corresponding to different image categories at time t as nodes and the distances between the class centroids corresponding to different image categories at time t as edges;

[0165] The inter-class uniformity loss construction module is used to take the logarithm of the average Gaussian potential of all node pairs in the inter-class topology graph at time t as the inter-class uniformity loss at time t;

[0166] The intra-class compactness loss construction module is used to construct the intra-class compactness loss at time t based on the distances between all node pairs corresponding to features in the inter-class topology graph at time t;

[0167] The total loss function construction module is used to construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss;

[0168] The parameter update module is used to update the parameters of the student model at time t through the total loss function at time t, and update the parameters of the teacher model at time t based on the parameters of the student model at time t. After completing the training at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

[0169] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. An image classification method based on continuous test time adaptation, characterized in that: The following steps are involved: Obtain image datasets corresponding to T time periods within the target area. Based on the image dataset corresponding to time period t, obtain the features and predicted image category of each image output by the student model at time period t; where t = 1, 2…T; Based on the features of each image at time t and the predicted image category, calculate the initial class centroid corresponding to different image categories at time t; Based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1, the class centroids corresponding to different image categories at time t are obtained through the exponential moving average algorithm; The centroids of different image categories at time t are used as nodes, and the distances between the centroids of different image categories at time t are used as edges to construct an inter-class topology graph at time t. The logarithm of the average Gaussian potential of all node pairs in the inter-class topology graph at time t is used as the inter-class uniformity loss at time t; Based on the distances between all pairs of corresponding features in the inter-class topology graph at time t, the intra-class compactness loss at time t is constructed; Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t, the total loss function at time t is constructed; Through the total loss function at time t, the parameters of the student model at time t are updated. Based on the parameters of the student model at time t, the parameters of the teacher model at time t are updated. After completing the training at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

2. The image classification method based on continuous test time adaptation according to claim 1, characterized in that: The logarithm of the average Gaussian potential of all nodes in the inter-class topology graph at time t is used as the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is: in, is the inter-class uniformity loss at time t, |V (t) | is the number of nodes in the inter-class topology graph at time t, T is the set parameter, V (t) is the node set of the inter-class topology graph at time t, v i is the node i, v in the inter-class topology graph at time t j is the node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, and w ij is the edge between node i and node j in the inter-class topology graph at time t.

3. The image classification method based on continuous test time adaptation according to claim 1, characterized in that: Based on the distances between all nodes and corresponding features in the inter-class topology graph at time t, the intra-class compactness loss at time t is constructed. The formula of the intra-class compactness loss at time t is: in, is the intra-class compactness loss at time t, is the centroid v of image category k in the inter-class topology graph at time t k The edge set of V (t) is the node set of the inter-class topology graph at time t, v k is the centroid of image category k in the inter-class topology graph at time t, is the expectation, T is the setting parameter, z α is the αth feature corresponding to image category k in the inter-class topology graph at time t, z β is the βth feature corresponding to image category k in the inter-class topology graph at time t, |z α -z β | 2 For z α With z β The square of the distance between To find the mathematical expectation of the square of the distance of all possible feature pairs extracted in the image category k in the inter-class topology graph at time t, k is the image category index.

4. The image classification method based on continuous test time adaptation according to claim 1, characterized in that: The total loss function at time t also includes: inter-class uniformity loss at time t. The construction process of inter-class uniformity loss at time t includes: Based on the distribution of different image categories in the image dataset corresponding to time t, the initial weighted items of each edge in the inter-class topological graph at time t are obtained; Based on the initial weighted items of each edge in the inter-class topology graph at time t and the weighted items of each edge in the inter-class topology graph at time t-1, the weighted items of each edge in the inter-class topology graph at time t are obtained by the exponential moving average algorithm; By weighting the edges in the inter-class topology graph at time t, the inter-class uniformity loss at time t is weightedly adjusted to obtain the inter-class uniformity loss at time t; Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t, the total loss function at time t is constructed.

5. The image classification method based on continuous test time adaptation according to claim 4, characterized in that: The initial weighted items of each edge in the inter-class topology graph at time t are obtained based on the distribution of different image categories in the image data set corresponding to time t, including: Based on the distribution of each image category in the image dataset corresponding to time t, the weight coefficient of each image category in the topological structure at time t is calculated. The formula is: Based on the weight coefficient of each image category in the topological structure at time t, calculate the initial weighted terms of each edge in the inter-class topological graph at time t; in, is the weight coefficient of image category k in the topological structure at time t, K represents the number of images belonging to image category k in the image dataset corresponding to time t. (t) is the number of image categories in the image dataset corresponding to time t, k is the image category index, ∈ is a very small constant used to avoid the denominator being zero, is the initial weighted item of the edge between node i and node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, is the weight coefficient of the image category corresponding to node i in the inter-class topology graph at time t in the topological structure, is the weight coefficient of the image category corresponding to node j in the inter-class topology graph at time t in the topological structure.

6. The image classification method based on continuous test time adaptation according to claim 4, characterized in that: The inter-class uniformity loss at time t is weightedly adjusted by the weighted terms of each edge in the inter-class topology graph at time t to obtain the inter-class uniformity loss at time t. The formula for the inter-class uniformity loss at time t is: in, is the uniformity loss between layers at time t, T is the setting parameter, V (t) is the node set of the inter-class topology graph at time t, v i is the node i, v in the inter-class topology graph at time t j is the node j in the inter-class topology graph at time t, i is the first node index, j is the second node index, and w ij is the edge between node i and node j in the inter-class topology graph at time t, is the weighted term of the edge between node i and node j in the inter-class topology graph at time t.

7. The image classification method based on continuous test time adaptation according to claim 4, characterized in that: Based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss at time t, the total loss function at time t is constructed. The total loss function at time t is: in, is the total loss function at time t, is the symmetric cross entropy loss at time t, is the feature alignment loss at time t, is the uniformity loss between classes at time t, is the intra-class compactness loss at time t, λ1 is the feature alignment weight, and λ2 is the topological stability weight.

8. The image classification method based on continuous test time adaptation according to claim 1, characterized in that: After data enhancement of the image dataset corresponding to time t, the enhanced image dataset corresponding to time t is obtained; Input the enhanced image dataset corresponding to time t into the feature extractor of the student model to obtain the random enhanced features of each image at time t; Based on the randomly enhanced features and predicted image categories of each image at time t, the initial class centroids corresponding to different image categories at time t are calculated.

9. The image classification method based on continuous test time adaptation according to claim 8, characterized in that: Based on the random enhancement features and predicted image categories of each image at time t, the initial class centroid corresponding to different image categories at time t is calculated. The calculation formula is: in, represents the initial class centroid corresponding to the image category k at time t, It represents the index corresponding to the maximum value in the output result after the a-th image at time t is processed by the student model. Aug(·) represents data enhancement. represents the ath image in the image dataset corresponding to time t, argmax(·) is the argmax function, I(·) represents the indicator function, B (t) Indicates the number of images in the image dataset corresponding to time t, represents the set of images belonging to image category k in the image dataset corresponding to time t, represents the number of images belonging to image category k in the image dataset corresponding to time t, Express judgment Is it equal to k? If so, then If not equal to Represents the features extracted by the student model for the enhanced data at time t, is the feature extractor of the student model at time t, a is the image index, and k is the image category index.

10. An image classification system based on continuous test time adaptation, characterized in that: include: The data acquisition module is used to obtain image datasets corresponding to T time periods in the target area, and based on the image dataset corresponding to time period t, obtain the features and predicted image category of each image output by the student model at time period t; where t = 1, 2…T; The initial class centroid calculation module is used to calculate the initial class centroid corresponding to different image categories at time t based on the features of each image at time t and the predicted image category; The class centroid calculation module is used to obtain the class centroids corresponding to different image categories at time t based on the initial class centroids corresponding to different image categories at time t and the class centroids corresponding to different image categories at time t-1 by using the exponential moving average algorithm; The inter-class topology graph construction module is used to construct the inter-class topology graph at time t by taking the class centroids corresponding to different image categories at time t as nodes and the distances between the class centroids corresponding to different image categories at time t as edges; The inter-class uniformity loss construction module is used to take the logarithm of the average Gaussian potential of all node pairs in the inter-class topology graph at time t as the inter-class uniformity loss at time t; The intra-class compactness loss construction module is used to construct the intra-class compactness loss at time t based on the distances between all node pairs corresponding to features in the inter-class topology graph at time t; The total loss function construction module is used to construct the total loss function at time t based on the inter-class uniformity loss, intra-class compactness loss, symmetric cross entropy loss, and feature alignment loss; The parameter update module is used to update the parameters of the student model at time t through the total loss function at time t, and update the parameters of the teacher model at time t based on the parameters of the student model at time t. After completing the training at time T, the teacher model at time T with updated parameters is used as the target image classification model to classify the target domain image.

Images