Realization method of reusable cache mechanism oriented to self-adaptive scene during testing

Abstract

The invention discloses a reusable cache mechanism implementation method for a self-adaptive scene during testing, which comprises the following steps of: constructing a ViT backbone network, inputting image samples in batches, and obtaining a feature statistic vector of each batch of image samples; performing difference calculation and weight setting on the basis of the feature statistic vector and the source domain statistical distribution to obtain a total offset feature vector of each batch of image samples, and performing dimension reduction compression to further obtain a cache key; performing cache hit judgment and distribution offset judgment, and obtaining an optimal learnable vector of the current batch according to cache hit judgment and distribution offset judgment results, and using the optimal learnable vector as a value corresponding to a cache key to construct a cache entry; whether the cache space is updated or not is judged, and final reasoning calculation of the current batch is carried out. The method has further innovations in the aspects of a cache key construction mechanism, a replacement strategy, a system multiplexing capability, equipment adaptability and the like, and shows stronger technical effects and implementability compared with the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cache management and online optimization of deep learning models in test-time adaptation, and in particular to a reusable cache mechanism implementation method for test-time adaptation scenarios. BACKGROUND

[0002] In modern computer vision tasks, the actual test environment after the deployment of the neural network model often differs significantly from the data distribution during training, thus leading to a significant decline in model test performance. This phenomenon is known as distribution shift. In order to alleviate the problem of distribution shift, test-time adaptation (TTA) has gradually become a research hotspot in the field of deep learning. Specifically, in the deployment / inference stage, the model adapts itself using the data distribution flowing into the test without accessing the source domain training data and usually without changing the task labels, thereby achieving the effect of alleviating distribution shift and improving the performance of the model in real-time environmental changes. Unlike unsupervised domain adaptation, the advantage of TTA is that it can improve model accuracy without accessing source data for alignment. Based on this main advantage, TTA has begun to be applied in many fields, such as automatic driving vehicle detection, pose estimation, video depth prediction, frame interpolation, and medical diagnosis scenarios.

[0003] Although TTA has been applied, there are still many challenges: for example, as a current mainstream TTA method, the entropy minimization TTA represented by TENT, although it achieves good results under certain conditions by obtaining the target solution that minimizes the prediction entropy, studies have shown that simply minimizing the prediction entropy encourages the model to be "overconfident", which may result in a solution with very low entropy but poor performance, i.e., there is a risk of obtaining a degenerate solution. Because high-entropy or low-confidence prediction solutions will bring larger and noisier gradients, they will dominate gradient updates, and because the cross-entropy loss is too strict for prediction solutions, even a small number of incorrect prediction solutions will cause incorrect updates to the model, i.e., error accumulation. In addition, since TTA does not require training the model from scratch, it is very suitable for rapid deployment on resource-constrained platforms such as edge devices or mobile devices, but this also puts forward the demand for lightweight TTA, and TTA methods that rely on backpropagation to update parameters are difficult to apply directly to these devices, so it is of great significance to explore TTA methods with high stability and low computational overhead for the application expansion of TTA in practical scenarios. The emerging non-backpropagation TTA method represented by Forward-Only Adaptation (FOA) inserts a small number of learnable vectors into the pre-trained Vision Transformer (ViT) model and uses evolutionary strategies to update these learnable vectors during the test phase through multiple forward evaluations, achieving TTA without source data, relying only on forward propagation, and being quite lightweight. These learnable vectors do not belong to model weights and do not contain any source domain model training information, but only participate in forward propagation as an external prefix vector at the input layer, and are pluggable inputs during inference. Updating these learnable vectors does not belong to the parameter update of the model itself. Although this method completely solves the device requirement problem of backpropagation TTA, it still faces the following challenges: First, the high latency bottleneck. The non-backpropagation TTA method represented by FOA performs multiple forward propagations on each batch of test samples, and after multiple forward propagations, the learnable vector with the smallest loss is taken as the optimal learnable vector, and its corresponding result is taken as the prediction solution for this batch of samples. This approach avoids backpropagation, but the overall delay is high in practical applications, and the computational complexity is mainly determined by the number of evaluation rounds (the number of forward evaluations per batch). Second, the state is not persistent and it is difficult to reuse across samples. The non-backpropagation TTA method represented by FOA needs to reset the state every time it enters a different damaged domain, starting from the learnable vector initialized randomly. Therefore, in the face of domain switching scenarios such as sunny to rainy and then sunny again, it cannot reuse the learnable vector that has been deeply optimized previously and needs to update the learnable vector from scratch, i.e., it cannot efficiently recover the historical state when the distribution is repeated, which will waste computational resources and seriously hinder its application in edge scenarios.To address the issue of redundant computation caused by TTA state reset, some research has begun to introduce caching mechanisms. However, these methods primarily rely on ResNet or multimodal visual-language models (VLMs) as their backbone networks. There are relatively few TTA methods based on the pure image model ViT that incorporate caching mechanisms, and these ViT-based caching methods still depend on backpropagation fine-tuning of parameters. Therefore, designing a caching mechanism suitable for TTA without backpropagation for ViT is both practically necessary and innovative. Summary of the Invention

[0004] The purpose of this invention is to provide a reusable caching mechanism for adaptive testing scenarios, overcoming the shortcomings of current ViT-based TTA methods without backpropagation. This method offers advantages such as high reuse efficiency, controllable latency, robustness to continuous domain shifts, and strong consistency with existing pipeline structures. It can be quickly implemented without modifying model weights or requiring backpropagation, achieving a plug-and-play caching mechanism. It is important to note that "without backpropagation" in this invention means that during the TTA process, no gradient-based parameter updates are performed, the gradient of model parameters or learnable vectors relative to any objective function is calculated, error backpropagation is not performed, a computational graph is not constructed, and there is no backpropagation of weights or structural parameters. The loss calculation involved in this invention serves only as a performance evaluation scalar based on the forward inference result, used for comparing the relative merits of different candidate learnable vectors during the inference stage and determining cache reliability. Its calculation process does not participate in any gradient derivation or chained derivative process, and it is not used as a supervisory signal for gradient updates of model weights. The statistical measures such as mean, standard deviation, and standardized score involved in this invention are used as engineering metrics for changes in characteristic distribution. Their role is to provide a stable and computable scale reference for cache reuse decisions and adaptive triggering.

[0005] To achieve the above functions, this invention designs a reusable caching mechanism implementation method for adaptive scenarios during testing, executing the following steps S1-S4: for image samples and cache space, the reusable caching mechanism is constructed, and the image samples are compressed and cached.

[0006] Step S1: Construct a ViT backbone network containing one or more Transformer layers. Input image samples into the ViT backbone network in batches. Select several Transformer layers in the ViT backbone network to perform inference on each batch of image samples, obtain the feature statistics vectors corresponding to each batch of image samples, save the results, and then proceed to step S2.

[0007] Step S2: Perform standardized difference calculation between the feature statistics vector of each batch and the existing source domain statistical distribution. Set weights according to the depth of the Transformer layer. Based on the weights, concatenate the difference calculation results of different layers to obtain the total offset feature vector of each batch of image samples. Perform dimensionality reduction and compression. Based on the dimensionality reduction and compression of the total offset feature vector, further obtain the cache key, and then proceed to step S3.

[0008] Step S3: Based on the cached key obtained in Step S2 and the key of each cached entry in the current cache space, calculate the similarity and distribution distance between the two to obtain the comprehensive confidence score, and set the threshold of the comprehensive confidence score accordingly; according to the comprehensive confidence score and its threshold condition, perform cache hit judgment and distribution offset judgment; according to the results of cache hit judgment and distribution offset judgment, obtain the optimal learnable vector of the current batch as the cache value, form a key-value pair with the cache key, further construct cache entries, and proceed to Step S4;

[0009] Step S4: Based on the cache entries of the current batch, construct replacement scores, determine whether to replace the cache entries in the cache space with the cache entries obtained in the current batch according to the replacement scores, update the cache space, and after completing the cache space update, use the best learnable vector of the current batch for the final inference calculation of the current batch. After the calculation is completed, jump back to step S1 to start processing the image samples of the next batch.

[0010] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0011] Based on the inherent properties of Vision-Language Models (ViT), this invention constructs a caching design method for test-time adaptive models. The method extracts statistical information from input features during the model inference phase, generates corresponding cache keys, and stores the learnable vectors obtained from the current batch optimization as cache values. When a new batch of input arrives, the system retrieves the cache based on feature similarity. If a match is found and passes verification, the learnable vectors in the cache are directly invoked to accelerate the adaptive inference process; otherwise, a new learnable vector is generated through the optimization module, and the cache is updated. This method significantly reduces computational latency during model testing and improves inference stability and reuse efficiency under scenarios of changing distributions. Compared to existing technologies such as ACE (Adaptive Cache Enhancement for Test-Time Adaptation of Vision-Language Models), which only selects low-entropy / high-confidence samples for cache using class-specific thresholds and employs an update mechanism to address significant distribution shifts, this invention uses statistical features and incorporates random projection dimensionality reduction to construct keys that approximate the position of the current batch in the source domain statistical space. This key does not require a unique identifier for the input distribution but serves as a candidate retrieval clue for batches with similar distributions. Subsequently, by comprehensively constraining reuse risks through mechanisms such as confidence calculation, distribution offset detection, and lightweight verification, the stability and reliability of cache reuse are still guaranteed even when statistical keys have near overlap. This approach differs significantly from ACE, which directly uses feature embeddings as keys. Secondly, it employs a frequency-driven replacement mechanism and similarity determination, while ACE relies solely on confidence / low-entropy filtering and class-specific thresholds, without using frequency replacement logic. Furthermore, this invention emphasizes the historical reuse capability of cache across tasks and scenarios, while ACE primarily focuses on cache updates for the current task. Simultaneously, this invention considers low-computing-power / edge device deployment scenarios, enhancing engineering from a system implementation perspective, while ACE mainly focuses on model adaptation. Therefore, this invention demonstrates further innovation in cache key construction mechanisms, replacement strategies, system reuse capabilities, and device adaptability, exhibiting stronger technical effects and feasibility than existing technologies. Attached Figure Description

[0012] Figure 1 This is a flowchart of a reusable caching mechanism implementation method for test-time adaptive scenarios provided by an embodiment of the present invention;

[0013] Figure 2 This is a schematic diagram of a preprocessing unit provided according to an embodiment of the present invention;

[0014] Figure 3 This is a schematic diagram of a lightweight adaptive initialization unit provided according to an embodiment of the present invention;

[0015] Figure 4 This is a schematic diagram of a cache module consisting of a cache retrieval unit and a cache update unit according to an embodiment of the present invention;

[0016] Figure 5 This is a schematic diagram of a distribution offset detection unit provided according to an embodiment of the present invention;

[0017] Figure 6 This is a schematic diagram of a reliability verification unit provided according to an embodiment of the present invention;

[0018] Figure 7 This is a schematic diagram of a cache perturbation adaptive unit provided according to an embodiment of the present invention;

[0019] Figure 8 This is a schematic diagram of a disturbance adaptive fine-tuning unit provided according to an embodiment of the present invention;

[0020] Figure 9 This is a schematic diagram of an inference execution unit provided according to an embodiment of the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0022] This invention provides a reusable caching mechanism implementation method for test-time adaptive scenarios, used to build a knowledge reuse system for Test-Time Adaptive (TTA). (Refer to...) Figure 1 Perform the following steps S1-S4 to complete the construction of a reusable caching mechanism, as well as the compression and caching of image samples, for the image samples and cache space:

[0023] Step S1: Construct a ViT backbone network containing one or more Transformer layers. Input image samples into the ViT backbone network in batches. Select several Transformer layers in the ViT backbone network to perform inference on each batch of image samples, obtain the feature statistics vectors corresponding to each batch of image samples, save the results, and then proceed to step S2.

[0024] To achieve backpropagation-free TTA (Time-to-Age-Induced Targeting), the ViT backbone network can selectively insert a small number of learnable vectors into modules such as the input layer, and perform targeted searches using algorithms for different inputs, achieving online optimization without backpropagation. In practical applications, the main differences between different ViTs such as ViT-B, ViT-L, and ViT-Huge lie in the number of hidden layers, the total number of layers, and the number of parameters, while maintaining a consistent input data dimensionality and using the same feature extraction principles during inference. Furthermore, the backbone network can also be selected from ViT variants, including but not limited to MobileViT and TinyViT. Here, we will use a reusable caching mechanism based on a ViT-B / 16 backbone network as an example for illustration.

[0025] Based on the ViT-B / 16 backbone network, the last Transformer layer in ViT-B / 16 is selected to extract features, and the learnable vectors are a set of Prompt vectors. This results in a finite number of evolutionary searches. Taking the FOA algorithm in TTA to complete an image classification task under distribution shift as an example, the following steps are performed:

[0026] Step S1.1: Refer to Figure 2 In the preprocessing unit, image samples from the same batch are processed and transformed into image feature vectors of dimension D, with batch size... As a unit, the dimension is The optimal learnable vector of the current batch, and the image feature vector obtained after processing the image samples, are jointly input into the ViT backbone network, where... This indicates the number of learnable vectors in this batch. The learnable vectors are insertable prefix cue vectors, which are concatenated with image information in the input layer and used together as the input sequence of ViT. They are only inserted and used in the entire input layer of ViT. In the input stage, they are combined with the original CLS token and patch embedding of ViT through sequence concatenation to form the input sequence of ViT. They are not part of the model weights. In the inference stage, they are used as adjustable parameters to affect subsequent attention calculations, so as to achieve TTA without backpropagation.

[0027] Select one or more Transformer layers in the ViT backbone network for forward inference. Each dimension is eigenvectors ,in The hidden dimension represents the feature vector of an image. This indicates the selected Transformer layer; a larger number indicates a deeper layer. This represents the total number of Transformer layers in the ViT backbone network. Indicates the input after the first Feature vectors obtained from layer forward inference This represents all input image samples in the current batch at the selected Transformer layer;

[0028] In the embodiment, the batch size typically selected in a conventional scenario is... And must meet To ensure statistical stability, and because the backbone network is ViT-B / 16, therefore , In the practice of algorithms such as FOA, it is usually taken that... To achieve a balance between accuracy and efficiency, this embodiment will... Maintain the same settings. If this is the first run, the optimal learnable vector for the current batch is initialized according to a Gaussian distribution; otherwise, the optimal learnable vector for the current batch is the optimal learnable vector obtained in the previous run.

[0029] Analyzing the characteristics of features extracted from different layers in ViT, shallow layers primarily extract local features such as edge textures, making them suitable for distribution-shift sensitive detection tasks. Deeper layers, on the other hand, progressively increase the level of semantic abstraction, yielding information primarily consisting of global and category features, making them suitable for classification tasks with distribution shifts. Since this example focuses on image classification, only the global features extracted from the last layer are used to calculate statistics. Only the 12th floor, , This refers to all input samples in the last Transformer layer of the current batch;

[0030] Step S1.2: Obtain the eigenvectors By batch size To process, This indicates the total number of layers selected for calculating statistics after feature extraction. It calculates the feature statistic vectors corresponding to each batch of image samples, including vectors with dimensions of [missing information]. The mean of the current batch of image samples and standard deviation The specific formula is as follows:

[0031] ;

[0032] in, Indicates the number of the current batch One input image sample, This represents the square of a dimension (element-wise), for example, if there is a dimension... vector ,but , This represents the transpose of a vector. It is a preset positive number to prevent underflow or division by zero when the variance is 0, thus ensuring numerical stability; The dimension is The all-one vector is stored. To avoid the key generation process being disturbed by learnable vectors, this invention calculates feature statistics... and At that time, the participation paths of learnable vectors are masked, meaning that calculations are performed only based on the backbone feature extraction results of the input image. and Then proceed to step S2.

[0033] In the example, the average value of the current batch and standard deviation The vectors of isostatistics are all .

[0034] It is worth noting that although the distribution of input data may shift over time or with environmental changes, such shifts are typically characterized by phases and repeatability. For example, in autonomous driving or visual inspection tasks, changes in weather, lighting, or viewing angle remain relatively stable over a certain period, and the distributions of different phases often repeat in the future (e.g., a cycle of "rainy day - sunny day - rainy day"). Therefore, completely non-repeating input distributions are extremely rare in real-world environments. Thus, the caching mechanism described in this invention demonstrates stable applicability and engineering feasibility in real-world continuous domain variation scenarios, and is suitable for test flow scenarios where the distribution is stable in segments over time and exhibits phased repetition.

[0035] Step S2: Perform standardized difference calculation between the feature statistics vector of each batch and the existing source domain statistical distribution. Set weights according to the depth of the Transformer layer. Based on the weights, concatenate the difference calculation results of different layers to obtain the total offset feature vector of each batch of image samples. Perform dimensionality reduction and compression. Based on the dimensionality reduction and compression of the total offset feature vector, further obtain the cache key, and then proceed to step S3.

[0036] The specific steps of step S2 are as follows:

[0037] Step S2.1: Refer to Figure 1 Based on the feature statistics vector obtained in step S1, the mean offset normalized vector of the current batch of image samples is calculated. and standard deviation offset standardized vector :

[0038] ;

[0039] ;

[0040] in, and These are the mean and standard deviation of the source domain statistics, respectively, which are calculated by the pre-trained ViT model during inference on the source domain test set before deploying this invention. and The data is obtained offline in the same way as in this invention. When using this invention, it is treated as a read-only constant, belonging to the runtime statistics already built into the model. It does not belong to any form of recoverable source domain data, and the source domain sample content cannot be deduced from it. The inference phase does not involve access to the original source domain data, fully meeting the requirements of TTA for scenarios without source domain data. The minimum lower limit is set to To avoid abnormal scaling; This is a preset value used to prevent division by zero or inputting zero during logarithmic operations. This means that the result is the element with the largest value in the corresponding dimension, evaluated on an element-by-element basis. This represents the element-wise logarithmic operation with the natural constant e as the base;

[0041] Will , By concatenating within the layer, we obtain a dimension of... spliced ​​vector :

[0042] ;

[0043] in, This indicates that the vectors are concatenated row by row; for example... The dimensions are vector conduct The concatenation operation will result in a dimension of [dimensional value missing]. The vector, i.e. .

[0044] Then, the weights are set sequentially according to the increasing depth of the layers. All weight values ​​are greater than 0, for different Transformer layers. concatenated vectors The layers are stitched together from shallow to deep to obtain a dimension of High-dimensional statistical vectors :

[0045] ;

[0046] in, Indicates that there is a common The concatenation vectors corresponding to each layer Used for column-wise concatenation operations;

[0047] In the example, the average value of the current batch and standard deviation Vector of equal statistics, vector of mean shift standardized and the vector standardized by the standard deviation offset All dimensions are ,vector The dimension is Since only the last Transformer layer is taken, ,so and same, No layer-by-layer splicing operation is required;

[0048] Step S2.2: Convert the high-dimensional statistical vector obtained in step S2.1 into... To perform dimensionality reduction and compression, first fix the random seed and generate a generator with dimension . random projection matrix , All elements All of them meet the following conditions:

[0049] ;

[0050] in, This indicates that the mean is 0 and the variance is 0. The Gaussian (normal) distribution of the random projection matrix makes the random projection matrix... Able to work for any vector that meets the dimensional requirement All of them meet the following conditions:

[0051] ;

[0052] That is, preserving the mean in the sense of mean means that the distance / similarity structure is preserved as much as possible during vector dimensionality reduction, which conforms to the properties of Johnson–Lindenstrauss random projection, and is approximately true in the high-probability sense of the current finite-dimensional random projection scenario. Among them, This represents the operation of finding the expectation. Denotes the square of the L2 norm. This represents a high-dimensional statistical vector. The dimension of the reduced-dimensional vector obtained after dimensionality reduction compression satisfies ;

[0053] In this embodiment, based on the random projection approximation of the Johnson–Lindenstrauss (JL) lemma, the Euclidean distance structure between features is preserved in a high-probability sense. Therefore, a Gaussian matrix generated by a fixed random seed is used for feature compression, thus balancing retrieval accuracy and computational efficiency. To ensure that the vector is sufficiently compressed while retaining relatively more feature information, the following steps are taken: Setting it to 64, based on the properties of the JL lemma, ensures stable separability between statistical bonds at different distribution stages, preventing severe distance collapse in the bond space. Experimental tests also show that no erroneous reuse due to collisions occurred when applied in the current scenario. Furthermore, in scenarios with high distribution density or increased collision probability, it is also possible to... Upgrading to 128 dimensions or employing a multi-projection redundancy mechanism (i.e., generating multiple independent random projections and jointly judging similarity) reduces the risk of key collisions. Even in extreme cases where dimensionality reduction leads to increased key similarity, subsequent steps utilize a triple-judgment mechanism consisting of confidence assessment, distribution offset detection, and credibility verification to prevent erroneous reuse, thus ensuring the robustness and feasibility of the caching system. Because high-dimensional vectors exhibit a "distance concentration" phenomenon in distribution offset scenarios, statistical keys across different batches tend to become inseparable, leading to inaccurate cache hits. Therefore, this dimensionality reduction approach is not only for accelerating subsequent data retrieval but, more importantly, to ensure that statistical keys maintain generalization and stable separability under varying distribution scenarios, avoiding noise amplification effects in high-dimensional statistics and significantly reducing retrieval confusion rates under cross-domain conditions. In addition, in edge devices, directly performing the approximate nearest neighbor search required for subsequent steps on the 1536-dimensional statistical vector will cause memory usage to exceed the limit. Therefore, reducing the key dimension can ensure the distribution separability while satisfying the device memory access constraint. Unlike dimensionality reduction methods such as PCA that require training, random projection can be reused in different tasks without retraining, which can meet the efficient dimensionality reduction requirements of edge constraints.

[0054] Step S2.3: Apply the random projection matrix obtained in step S2.2 The high-dimensional statistical vector obtained in step S2.1 By performing a linear mapping through the product, we obtain a dimension of... Dimensionality reduction vector :

[0055] ;

[0056] This means a feature representation with a smaller physical space footprint. After dimensionality reduction, to ensure comparability in similarity metrics when the vector is used as a key, the dimensionality-reduced vector... Normalization is performed:

[0057] ;

[0058] in Represents the L2 norm. The key representing the cache. The purpose of normalization, using preset values, is to ensure that the geometric structure of the statistical bonds reflects only the offset direction between the current batch and the source domain distribution, unaffected by the offset magnitude scale. This addresses bond instability caused by distribution jitter and enhances bond robustness. (Refer to...) Figure 2 , with cached key As the output of the preprocessing unit, it then proceeds to step S3.

[0059] Step S3: Based on the cached key obtained in Step S2 and the key of each cached entry in the current cache space, calculate the similarity and distribution distance between the two to obtain the comprehensive confidence score, and set the threshold of the comprehensive confidence score accordingly; according to the comprehensive confidence score and its threshold condition, perform cache hit judgment and distribution offset judgment; according to the results of cache hit judgment and distribution offset judgment, obtain the optimal learnable vector of the current batch as the cache value, form a key-value pair with the cache key, further construct cache entries, and proceed to Step S4;

[0060] The specific steps of step S3 are as follows:

[0061] Step S3.1: Input the keys of each cache entry in the current cache space, as well as the keys of the cache entries obtained in step S2, into the cache retrieval unit. Since the cache space is empty when this step is executed for the first time, it will definitely be determined that the cache has not been hit. Input the currently obtained learnable vectors and the image samples of this batch into the lightweight adaptive initialization unit for online evolutionary search. Through a finite number of evolutionary searches, the currently obtained learnable vectors of this batch are used to obtain the optimal learnable vector of this batch, which is used as the cache value to form a key-value pair, construct a cache entry, and then execute step S4; otherwise, refer to Figure 4 The index is constructed using the Approximate Nearest Neighbor Search (ANN) method. Since the normalized inner product and cosine similarity are equivalent in monotonicity, the keys in the cached entries obtained in the current batch can be used to perform a top-k approximate search on the constructed index using the inner product form of the following formula to calculate the similarity between the two:

[0062] ;

[0063] in, Represents the set of cache entries in the current cache space. The The key of each cache entry This indicates the total number of cached entries currently in the cache space. This represents the key in the cache entry obtained in step S2. Representing vectors with vector The cosine similarity, with values ​​ranging from 1 to 10. ;like The closer the value is to 1, the more similar the two are in terms of statistical characteristics; conversely, the less similar they are, the less similar they are.

[0064] To further enhance the robustness of this method in determining non-stationary distributions, a distribution distance is introduced:

[0065] ;

[0066] in, and Indicates cached entries The mean and standard deviation vector of the current batch of image samples. and standard deviation , The statistical distance between the current batch offset key and the historical key in the source domain space; the dynamic learnability coefficient is calculated using the following formula. :

[0067] ;

[0068] in, and They represent The lower and upper bounds of the value, Indicates the historical misuse rate. This represents the threshold for misuse rate. Indicates control Influence Sensitivity based on similarity. , and Calculate the overall confidence level To jointly characterize the reliability of cache reuse:

[0069] ;

[0070] Comprehensive confidence level This serves as the output of the cache retrieval unit. The overall confidence score, inspired by the Gaussian kernel function, can be viewed as a confidence metric that balances feature similarity and distribution distance. The exponential decay function remains smooth when the distance is small and decays rapidly as the distance increases, consistent with the law that cache similarity weakens with distribution shift. and The degree of matching at the feature direction and amplitude levels is characterized separately. To achieve a dynamic balance between reusability and reliability, a dynamic learnability coefficient is introduced. Its reusability rate increases with the error rate. The system increases with increasing confidence levels, thereby accelerating confidence decay and reducing false reuse, and vice versa. This adjustment originates from steady-state feedback control based on the feedback regulation concept, and there is no circular dependency, allowing the system to maintain a statistical balance between hit rate and false reuse rate.

[0071] In this embodiment, the learnable vector used as the cache value is a set of Prompt vectors inserted into the input layer. for , for To ensure stable calculated values ​​while avoiding the introduction of excessive noise, the sliding period... Set it to 5.

[0072] by Instead of simple This serves as the basis for determining whether a cache hit has occurred. , , , , These parameters can be set once during system initialization, requiring no manual tuning during operation. The overall confidence calculation involves only a few scalar operations and one exponential operation; the additional computational cost is negligible compared to the forward inference of the backbone network. Compared to the TTA method, which relies on backpropagation, this invention adds only a small amount of statistical calculations, approximate nearest neighbor retrieval, and small-scale proxy verification in each batch, while maintaining the same overall time complexity as the number of forward evaluation rounds. Experimental results show that on a typical ViT-B / 16 model, the average inference latency of this invention is significantly lower than the FOA baseline, which relies entirely on evolutionary search, and significantly lower than methods requiring gradient backpropagation.

[0073] In this embodiment, the ANN method for constructing the index can selectively employ graph indexing methods with hierarchical navigation structures, such as HNSW, IVF-Flat, HNSW-PQ, or IVF-PQ. HNSW requires no training, is incrementally update-friendly, and is suitable for small to medium-sized online systems. IVF-Flat first performs coarse K-means clustering, then groups the vectors into buckets, with each bucket storing the vectors at their original precision. HNSW-PQ and IVF-PQ add product quantization (PQ) to the above two methods, further reducing memory usage. These methods can be integrated into the current process using existing tools in the FAISS library. While ensuring lightweight implementation, existing cached information is utilized as much as possible. HNSW-PQ is preferred, and read-only queries and batch delayed reconstruction are enabled on the client side to reduce maintenance overhead. k is selected as an integer between 5 and 10, including 5 and 10. The initial value is set to 1.

[0074] Step S3.2: Set the initial threshold for the overall confidence level. If this is the first execution, a cache hit is determined when the following formula is met:

[0075] ;

[0076] in, The initial threshold representing the overall confidence level. This represents the overall confidence level, calculated in step S3.1;

[0077] If this is the nth execution, first calculate the median of the overall confidence scores of the k candidate cache entries obtained in step S3.1. and mean And among them, the overall confidence level is higher than the median. quantity This is used to dynamically adjust the threshold; a cache hit is determined when the following formula is met:

[0078] ;

[0079] in, The dynamic threshold representing the overall confidence level is as follows:

[0080] ;

[0081] in, , Represents the smoothing coefficient, if Then, in the above formula Change to use This is to prevent excessive reuse that may result from lowering the threshold, thereby enhancing the robustness of the method.

[0082] If a cache hit is detected, proceed to the distribution offset detection unit and execute step S3.3;

[0083] If the overall confidence level does not reach the set overall confidence level threshold, it is determined that the cache has not been hit. The currently obtained learnable vector and the image samples of the current batch are input into the lightweight adaptive initialization unit for online evolutionary search. The currently obtained learnable vector of the current batch is used to obtain the optimal learnable vector of the current batch through a finite number of evolutionary searches, which is used as the cache value. A key-value pair is formed to construct a cache entry. Then, step S4 is executed.

[0084] In this embodiment, since only the last Transformer layer in ViT-B / 16 is used, its features have a high degree of semantic abstraction and feature separability. To reduce the misuse of dissimilar features without compromising the effectiveness of the caching mechanism, a comprehensive confidence threshold is used. Set it to 0.9.

[0085] Reference Figure 3 The lightweight adaptive initialization unit receives the currently obtained learnable vectors and image samples of the current batch. The ViT backbone network performs inference calculations on the image samples of the current batch, calculates the loss of each learnable vector during inference, takes the learnable vector with the minimum loss as the starting point of the evolutionary search, performs evolutionary search based on the CMA-ES method, and outputs the optimal learnable vector of the current batch.

[0086] Step S3.3: Refer to Figure 5 After determining that a cache hit has occurred, the distribution offset detection unit first uses a vector normalized by the mean offset of the current batch. and the vector standardized by the standard deviation offset The statistical offset intensity is constructed using the following formula:

[0087] ;

[0088] in, Indicates the statistical offset intensity. This represents the hidden dimension of the image feature vector; that is, to approximate the feature distribution of the current batch in the source domain statistical space as a statistical state characterized by the mean and scale parameters, and to construct the statistical offset strength based on its offset relative to the source domain statistics. It should be noted that the mean, standard deviation, and their standardized forms are not used to establish strict probability distribution hypotheses or statistical significance tests, but rather as an engineering approximation statistical measure to characterize the feature scale, location, and their relative trends over time. The larger the value, the greater the offset. To detect the offset of the current batch relative to historical batches, a sliding window mean is defined. With sliding window variance Update as follows:

[0089] ;

[0090] ;

[0091] After obtaining the results, normalize the degree of offset using the sliding window mean and variance according to the following formula, and calculate the standardized drift score. :

[0092] ;

[0093] For standardized drift scores Determine whether a significant distribution shift has occurred using the following formula:

[0094] ;

[0095] in The threshold selection follows the principle of scale change detection in engineering experience. Its value does not depend on the assumption that the input features satisfy a Gaussian distribution. Instead, it adaptively reflects the range of changes under historical stable conditions through online sliding statistics. It can be dynamically adjusted according to the application scenario and does not constitute a method limitation. This represents the distribution shift detection factor. A value of 1 indicates that a significant distribution shift has occurred in the current batch. In this case, the learnable vectors from the hit cache entries need to be extracted and input into the cache perturbation adaptive unit for online evolutionary search. The processed learnable vectors are then used as the values ​​corresponding to the cache keys of the current batch, and the processed learnable vectors are used as the optimal learnable vectors for the current batch. Proceed to step S4. A value of 0 indicates that the current batch is basically maintained within the historical distribution. Proceed to step S3.4.

[0096] In the embodiment, the initial timing... , . It was selected with reference to the settings of the FOA algorithm. The threshold is set to 2 to determine Binarization does not lead to excessively large binarization errors because the threshold here represents the confidence level of "significant shift," which is determined based on the 2–3σ statistical principle commonly used in unsupervised shift detection. This indicates a statistically significant distribution shift; otherwise, the batch statistics remain largely within the historical distribution. Besides adhering to a strict normality assumption, empirical calibration results from various real-world tasks show that when the distribution shift reaches twice the historical average shift, the cache reuse error rate increases significantly by more than 18%. Therefore, this empirical value is used as the engineering threshold for the trigger condition to maintain the system's lowest failure rate.

[0097] Step S3.4: Refer to Figure 6 Enter the reliability verification unit to begin verification, starting from the current batch size. The method uses stratified sampling based on predicted entropy to select... For each image sample, the learnable vector corresponding to the key with the highest overall confidence obtained in step S3.1 is generated. Extract the optimal learnable vector obtained from the previous batch. , respectively used for the selected For each image sample, the inference loss is calculated as follows: The performance degradation ratio is calculated using the formula below. :

[0098] ;

[0099] in, This represents the loss value obtained after inference using learnable vectors. Represents absolute value; obtains It is then used in the following formula for the misuse rate. Calculation:

[0100] ;

[0101] in, This indicates the percentage of cases that meet the conditions in parentheses. This represents the performance degradation percentage threshold, calculated in this study. This will be used for updates after the current sliding cycle ends. The calculation is as follows. Since the cache is empty when step S3.2 is executed for the first time, a cache hit is not possible, therefore step S3.4 will not be executed and is not involved. and The calculation; if the following conditions are met, online evolutionary search will not be performed, and the result will be taken. The best learnable vector for the current batch enters the inference execution unit for inference calculation. After the calculation is completed, it jumps back to step S1 to start processing the next batch of image samples.

[0102] ;

[0103] If the above formula is not satisfied, the optimal learnable vector obtained in the previous batch will be... Input perturbation adaptive fine-tuning unit, perform a finite number of evolutionary searches:

[0104] ;

[0105] in Indicated by The learnable vector obtained by performing a finite number of evolutionary searches starting from the evolutionary starting point. This indicates that a finite number of evolutionary search operations are performed using existing vectors. After the evolutionary search is completed, the learnable vector obtained by the evolutionary search is taken as the optimal learnable vector for this batch, and used as the value. This vector, along with the key, forms a cache entry and is input into the cache update unit; then step S4 is executed.

[0106] This validation step effectively mitigates the risk of misuse due to insufficient statistical keys to uniquely represent the distribution while maintaining lightweight operation. Since using different learnable vectors for the same sample yields paired loss observations, a t-test is used to assess the statistical significance of the difference between the two. Stratified sampling based on prediction entropy sorts the samples in the current batch according to their prediction entropy, dividing them into high-entropy, medium-entropy, and low-entropy strata. This increases the proportion of samples taken from the high-entropy stratum, focusing on high-entropy samples where the model is uncertain, thus improving the representativeness of the finite sample size.

[0107] In this embodiment, since backpropagation is not used, the evolutionary algorithm employs the gradient-free CMA-ES algorithm as a finite-time gradient-free search method to complete the online evolutionary search task. To control the computational overhead of lightweight calibration, the batch size is adjusted. The value is 6, which is the batch size. The 10% increase in latency due to calibration is negligible. The batch size for verification is determined based on the overall confidence level. The adaptive setting increases the sample size as the confidence level decreases. This is because lightweight calibration is designed not for rigorous statistical inference, but for rapid detection of significant mismatches. Since the overall confidence level and dynamic threshold pre-filter most non-matching samples, it is only necessary to verify whether cache reuse causes significant performance degradation on a small sample set. This process is more similar to anomaly detection or risk screening than large-sample statistical testing. In high-variance tasks or when the misuse rate of consecutive batches increases, the proportion of calibration samples can be adaptively adjusted to 20%–30% of the batch size, or a sliding window method using the cumulative surrogate loss mean can be used for re-verification to improve detection stability and robustness. The cache perturbation adaptive unit and the perturbation adaptive fine-tuning unit are respectively based on… and This serves as a functional unit for online evolutionary searches, acting as the starting point for evolution. (The above...) The calculation is used to drive the query path selection of the ANN index in the cache retrieval module, thereby directly affecting the system's storage access mode and data flow scheduling. Therefore, it is a technical improvement to the computer operation process.

[0108] Reference Figure 7 The cache perturbation adaptive unit receives the learnable vectors from the hit cache entries. The ViT backbone network performs inference calculations on the current batch of image samples, calculates the loss during inference of each learnable vector, performs evolutionary search based on the CMA-ES method, and outputs the optimal learnable vector for the current batch.

[0109] Reference Figure 8 The perturbation adaptive fine-tuning unit receives the best learnable vector obtained from the previous batch. The ViT backbone network performs inference calculations on the image samples of the current batch, calculates the loss during inference of each learnable vector, performs evolutionary search based on the CMA-ES method, and outputs the best learnable vector of the current batch.

[0110] Step S4: Based on the cache entries of the current batch, construct a replacement score. Determine whether to replace the cache entries in the cache space with the cache entries obtained in the current batch based on the replacement score. Update the cache space. After the cache space update is completed, use the best learnable vector of the current batch for the final inference calculation of the current batch. If it is an online scenario, after the calculation is completed, jump back to step S1 to start processing the next batch of image samples. Repeat steps S1-S4.

[0111] The specific steps of step S4 are as follows:

[0112] Step S4.1: Refer to Figure 4 The cache update unit receives cache entries from the current batch, including keys and the best learnable vector (i.e., the value in the key-value pair). Based on metrics such as usage frequency, it employs a reuse-aware, lightweight cache replacement strategy to guide cache replacement or insertion. Its core idea is no longer to rely solely on hit counts or low-entropy filtering, but rather to combine confidence, usage frequency, and performance feedback to quantitatively evaluate the value of cache entries. This allows for more stable retention of high-value entries and deletion of low-value or expired entries in scenarios with changing distributions. Specifically, cache entries... Replacement fractions Define the following formula: a higher score indicates that the item should be retained, and a lower score indicates that it has a higher priority for replacement:

[0113] ;

[0114] ;

[0115] in Indicates cached entries Cumulative hit count since being added to the cache space express normalized frequency, This indicates the timestamp of the last successful reuse or verification of a cached entry. express The reuse freshness obtained by normalization This represents the maximum number of hits for all entries in the current cache. This indicates the timestamp of the most recent update in the current cache. This represents the timestamp of the oldest item in the current cache that has not been updated. Replacement score The structure is determined by engineering constraints and aims to solve the problem of cache pollution, distribution offset and historical failure information coexisting in adaptive scenarios during testing. It is not an empirical engineering stacking, but a unified replacement model built on four irreplaceable factors such as statistical similarity, historical validity, time decay and performance risk in adaptive scenarios during testing. It can effectively prevent cache pollution and improve cross-distribution robustness. The key is measured to have statistical similarity to the current batch; the higher the similarity, the less likely it should be replaced. This characterizes the general applicability of an entry in history. Entries that appear frequently are considered to be long-term stable patterns; the stronger their historical validity, the less likely they are to be replaced. To prevent long-unupdated entries from being reused unintentionally, ensure that the cache does not become obsolete. The system assesses whether an item has caused performance degradation in past batches; items with a high risk of degradation should be prioritized for replacement. It also considers hit frequency, freshness, and historical degradation risk. An additive structure allows frequency-based replacements to dominate, leading to cache pollution in long-tailed distribution scenarios. A multiplicative structure, however, can immediately reduce replacement priority when any metric deteriorates, providing early warning capabilities, which is particularly important under the low-latency constraints of edge devices. The numerical scale can be further stabilized through normalization. Additionally, The global scaling factor, defined as the replacement fraction, is calculated using the following formula:

[0116] ;

[0117] Step S4.2: Based on the replacement score Determine whether to replace cache entries in the cache space with cache entries obtained in the current batch, and then determine the replacement score in the cache space. The smallest cache entry, if its replacement score Replacement score of cache entries less than the current batch If the cached entry is obtained in the current batch, the cached entry in the cache space will be replaced; otherwise, the cache space will not be updated. Then, the inference execution unit will be entered.

[0118] Reference Figure 9 The inference execution unit receives the best learnable vector for the current batch and the image samples for the current batch. The ViT backbone network performs inference to obtain the final inference result for the current batch.

[0119] When a significant change in the input distribution is detected, a penalty is imposed on the calculation of the replacement score, reducing the importance of the current cache entry by decreasing the replacement score. When the cache needs to be replaced, a min-heap structure combined with lazy updates is used to maintain the replacement score of the cache entry. The replacement score is updated only in real-time for entries that are hit, inserted, or validated. The order of items in the heap is adjusted; when the cache is full and needs to be evicted, the top item is popped and removed from the heap. Only the smallest entry is needed, eliminating the need for a full recalculation, thus achieving incremental normalization and on-demand updates. It's worth noting that cached scores are replaced. It is designed based on the principles of multi-objective weighting and normalization. Confidence is enhanced through multiplication in the numerator. Access frequency and freshness Positive indicators, and the degradation ratio is added to the denominator. Access count The inclusion of negative constraints achieves a balance between reliability and timeliness. All the above factors are derived from real-time statistical results during system operation, requiring no manual setting or offline parameter tuning, and can automatically complete replacement decisions under lightweight conditions. The defined structure belongs to a typical weighted utility optimization function, which has good interpretability and numerical stability. The calculation involves only a few scalar multiplication, addition, and division operations, with a computational complexity of O(1). It does not impose an additional burden on the system's real-time performance and can be considered a linear approximation of multi-objective optimization. It comprehensively considers reuse confidence, historical performance improvement, timeliness, and distribution shift, thus achieving a balance between lightweight design, robustness, and distribution adaptability. When the number of cached entries exceeds a certain threshold, a multi-projection redundancy mechanism will be automatically enabled to avoid key collisions. After completing the cache update, the optimal learnable vector for the current batch is used for the final inference calculation of the current batch. After the calculation is complete, the process jumps back to step S1 to begin processing the next batch of samples.

[0120] In this embodiment, to ensure the method is lightweight and create conditions for edge deployment, the cache size is set to 128MB, and the cache replacement strategy is as described above. The primary criterion is used for replacement decisions; TinyLFU is only used as an implementation component for frequency estimation (such as distributed counting or Bloom filtering assistance), without changing the underlying criterion. The core decision-making logic is based on this. The historical window length for calculating the moving average is set to... ,in While ensuring statistical stability, we should reduce interference from premature historical information.

[0121] To further improve the execution efficiency of the overall inference and optimization process and ensure real-time performance without backpropagation, this invention introduces an automatic mixed-precision module in the evolutionary algorithm's evaluation of learnable vectors and the regular inference stage. This module, based on a dynamic adjustment strategy for floating-point precision, automatically switches operations with lower numerical stability requirements to FP16 half-precision, while maintaining full FP32 precision for critical numerical operations (such as normalization and loss accumulation). This achieves significant acceleration without affecting the final model's output accuracy and expected calibration error. Similarity calculation, threshold update, and proxy loss statistics in this invention are all performed in FP32 precision to ensure threshold stability and repeatability. With the help of the automatic mixed-precision module, GPU computing units can achieve higher parallel utilization while significantly reducing memory usage and bandwidth pressure. This mechanism creates more favorable conditions for the deployment of this invention on edge hardware without changing the backbone network structure, ensuring that the system still has real-time responsiveness and high energy efficiency in resource-constrained environments.

[0122] The value ranges of each parameter involved in this method embodiment are summarized in Table 1 below:

[0123] Table 1. Parameter Definitions and Value Ranges

[0124]

[0125] In summary, this invention improves the TTA method by introducing a retrieval-caching mechanism. Building upon the original backpropagation-free approach, it introduces key vector construction based on batch feature statistics, approximate nearest neighbor (ANN) retrieval, and a frequency-driven replacement strategy to reuse optimized hint parameters among batches with similar distributions. Furthermore, after a cache hit, a distribution offset detection and "cache mismatch detection / lightweight calibration" mechanism are added: a surrogate target re-evaluation without backpropagation is performed on a minimal subset within the batch; reuse is only allowed if the surrogate loss does not deteriorate; otherwise, online evolutionary optimization is initiated. This avoids reuse mismatches caused by non-unique statistical keys, improving reuse reliability and controlling risk. This invention is applicable to adaptive frameworks in backpropagation-free and lightweight testing scenarios, including but not limited to methods such as FOA and ZOA. Lightweighting is reflected in a significant reduction in overall system computation and storage overhead, rather than minimizing single-step computations. Compared to traditional online optimization methods, this invention significantly reduces computational load and completely avoids gradient propagation, thus enabling edge deployment. To balance real-time performance and stability, this invention employs a hierarchical index maintenance mechanism: during the online phase, only incremental insertions and lightweight updates are performed; after the number of cached entries reaches a threshold or a stable period ends, index reconstruction is performed in batches with a delay, thereby maintaining index performance without the need for additional threads and avoiding resource consumption caused by frequent reconstructions. This mechanism has broad engineering and industrial application prospects in scenarios such as streaming inference and continuous domain offset adaptation, edge computing and low-computing-power device deployments (terminals sensitive to latency and energy consumption).

[0126] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for implementing a reusable caching mechanism for test-time adaptive scenarios, characterized in that, Perform the following steps S1-S4 to complete the construction of a reusable caching mechanism, as well as the compression and caching of image samples, for the image samples and cache space: Step S1: Construct a ViT backbone network containing one or more Transformer layers. Input image samples into the ViT backbone network in batches. Select several Transformer layers in the ViT backbone network to perform inference on each batch of image samples, obtain the feature statistics vectors corresponding to each batch of image samples, save the results, and then proceed to step S2. Step S2: Perform standardized difference calculation between the feature statistics vector of each batch and the existing source domain statistical distribution. Set weights according to the depth of the Transformer layer. Based on the weights, concatenate the difference calculation results of different layers to obtain the total offset feature vector of each batch of image samples. Perform dimensionality reduction and compression. Based on the dimensionality reduction and compression of the total offset feature vector, further obtain the cache key, and then proceed to step S3. Step S3: Based on the cached key obtained in Step S2 and the key of each cached entry in the current cache space, calculate the similarity and distribution distance between the two to obtain the comprehensive confidence score, and set the threshold of the comprehensive confidence score accordingly; according to the comprehensive confidence score and its threshold condition, perform cache hit judgment and distribution offset judgment; according to the results of cache hit judgment and distribution offset judgment, obtain the optimal learnable vector of the current batch as the cache value, form a key-value pair with the cache key, further construct cache entries, and proceed to Step S4; Step S4: Based on the cache entries of the current batch, construct replacement scores, determine whether to replace the cache entries in the cache space with the cache entries obtained in the current batch according to the replacement scores, update the cache space, and after completing the cache space update, use the best learnable vector of the current batch for the final inference calculation of the current batch. After the calculation is completed, jump back to step S1 to start processing the image samples of the next batch.

2. The method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S1.1: For image samples in the same batch, process them into image feature vectors of dimension D, and then distribute them according to the batch size. As a unit, the dimension is The optimal learnable vector of the current batch, and the image feature vector obtained after processing the image samples, are jointly input into the ViT backbone network, where... This indicates the number of learnable vectors in this batch. The learnable vectors are insertable prefix cue vectors, which are concatenated with image information in the input layer and used together as the input sequence of ViT. They are only inserted and used in the entire input layer of ViT. In the input stage, they are combined with the original CLS token and patch embedding of ViT through sequence concatenation to form the input sequence of ViT. They are not part of the model weights. In the inference stage, they are used as adjustable parameters to affect subsequent attention calculations, so as to achieve TTA without backpropagation. Select one or more Transformer layers in the ViT backbone network for forward inference. Each dimension is eigenvectors ,in The hidden dimension represents the feature vector of an image. Indicates the selected Transformer layer. This represents the total number of Transformer layers in the ViT backbone network. Indicates the input after the first Feature vectors obtained from layer forward inference This represents all input image samples in the current batch at the selected Transformer layer; Step S1.2: Obtain the eigenvectors By batch size The process involves calculating the feature statistics vectors corresponding to each batch of image samples, including vectors with dimensions of 1. The mean of the current batch of image samples and standard deviation The specific formula is as follows: ； in, Indicates the number of the current batch One input image sample, Represents the square of each dimension. It is a pre-defined positive number. The dimension is The vector of all ones is stored; the result is then used to calculate the characteristic statistics. and At that time, the participating paths of learnable vectors are masked, and calculations are performed only based on the backbone feature extraction results of the input image. and Then proceed to step S2.

3. The method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 2, characterized in that, If this is the first run, the optimal learnable vector for the current batch mentioned in step S1.1 is obtained by initializing according to a Gaussian distribution; otherwise, the optimal learnable vector for the current batch is the optimal learnable vector obtained in the previous run.

4. The method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 2, characterized in that, The specific steps of step S2 are as follows: Step S2.1: Based on the feature statistics vector obtained in step S1, calculate the mean offset normalized vector of the current batch of image samples. and standard deviation offset standardized vector : ； ； in, and These are the mean and standard deviation of the source domain statistics, respectively. This is a preset value used to prevent division by zero or inputting zero during logarithmic operations. This means that the result is the element with the largest value in the corresponding dimension, evaluated on an element-by-element basis. This represents the element-wise logarithmic operation with the natural constant e as the base; Will , By concatenating within the layer, we obtain a dimension of... spliced ​​vector : ； in, This indicates that the vectors will be concatenated row by row. Then, the weights are set sequentially according to the increasing depth of the layers. All weight values ​​are greater than 0, for different Transformer layers. concatenated vectors The layers are stitched together from shallow to deep to obtain a dimension of High-dimensional statistical vectors : ； in, Indicates that there is a common The concatenation vectors corresponding to each layer Used for column-wise concatenation operations; Step S2.2: Convert the high-dimensional statistical vector obtained in step S2.1 into... To perform dimensionality reduction and compression, first fix the random seed and generate a generator with dimension . random projection matrix , All elements All of them meet the following conditions: ； in, This indicates that the mean is 0 and the variance is 0. The Gaussian distribution of the random projection matrix makes the random projection matrix... Able to work for any vector that meets the dimensional requirement All of them meet the following conditions: ； in, This represents the operation of finding the expectation. Denotes the square of the L2 norm. This represents a high-dimensional statistical vector. The dimension of the reduced-dimensional vector obtained after dimensionality reduction compression satisfies ; Step S2.3: Apply the random projection matrix obtained in step S2.2 The high-dimensional statistical vector obtained in step S2.1 By performing a linear mapping through the product, we obtain a dimension of... Dimensionality reduction vector : ； For dimensionality reduction vectors Normalization is performed: ； in Represents the L2 norm. The key representing the cache. Set the preset value, and then proceed to step S3.

5. The method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 4, characterized in that, The specific steps of step S3 are as follows: Step S3.1: Input the keys of each cache entry in the current cache space, as well as the keys of the cache entries obtained in Step S2, into the cache retrieval unit. Since the cache space is empty when this step is executed for the first time, it will definitely be determined that the cache has not been hit. Input the currently obtained learnable vectors and the image samples of this batch into the lightweight adaptive initialization unit for online evolutionary search. The currently obtained learnable vectors of this batch are used to obtain the optimal learnable vector of this batch through a finite number of evolutionary searches, which is used as the cache value to form a key-value pair, construct a cache entry, and then execute Step S4; otherwise, calculate the similarity between the two using the following formula: ； in, Represents the set of cache entries in the current cache space. The The key of each cache entry This indicates the total number of cached entries currently in the cache space. This represents the key in the cache entry obtained in step S2. Representing vectors with vector The cosine similarity, with values ​​ranging from 1 to 10. ; Introducing distribution distance: ； in, and Indicates cached entries The mean and standard deviation vector of the current batch of image samples. and standard deviation , The statistical distance between the current batch offset key and the historical key in the source domain space; the dynamic learnability coefficient is calculated using the following formula. : ； in, and They represent The lower and upper bounds of the value, Indicates the historical misuse rate. This represents the threshold for misuse rate. Indicates control Influence Sensitivity based on similarity; , and Calculate the overall confidence level : ； Comprehensive confidence level As the output of the cache retrieval unit; Step S3.2: Set the initial threshold for the overall confidence level. If this is the first execution, a cache hit is determined when the following formula is met: ； in, The initial threshold representing the overall confidence level. This represents the overall confidence level, calculated in step S3.1; If this is the nth execution, first calculate the median of the overall confidence scores of the k candidate cache entries obtained in step S3.

1. and mean And among them, the overall confidence level is higher than the median. quantity A cache hit is determined when the following condition is met: ； in, The dynamic threshold representing the overall confidence level is as follows: ； in, , Represents the smoothing coefficient, if Then, in the above formula Change to use ; If a cache hit is detected, proceed to the distribution offset detection unit and execute step S3.3; If the overall confidence level does not reach the set overall confidence level threshold, it is determined that the cache has not been hit. The currently obtained learnable vector and the image samples of the current batch are input into the lightweight adaptive initialization unit for online evolutionary search. The currently obtained learnable vector of the current batch is used to obtain the optimal learnable vector of the current batch through a finite number of evolutionary searches, which is used as the cache value. A key-value pair is formed to construct a cache entry. Then, step S4 is executed. Step S3.3: After determining a cache hit, the distribution offset detection unit first uses the vector normalized by the mean offset of the current batch. and the vector standardized by the standard deviation offset The statistical offset intensity is constructed using the following formula: ； in, Indicates the statistical offset intensity. Represents the hidden dimension of the image feature vector; Define the sliding window mean With sliding window variance Update as follows: ； ； After obtaining the results, construct the standardized drift score using the following formula. : ； For standardized drift scores Determine whether a significant distribution shift has occurred using the following formula: ； in This represents the distribution shift detection factor. A value of 1 indicates that a significant distribution shift has occurred in the current batch. In this case, the learnable vectors from the hit cache entries need to be extracted and input into the cache perturbation adaptive unit for online evolutionary search. The processed learnable vectors are then used as the values ​​corresponding to the cache keys of the current batch to construct cache entries. The processed learnable vectors are then used as the optimal learnable vectors for the current batch, and the process proceeds to step S4. If the value is 0, the process proceeds to step S3.

4. Step S3.4: Input the reliability verification unit to start the verification, starting from the current batch size. The method uses stratified sampling based on predicted entropy to select... For each image sample, the learnable vector corresponding to the key with the highest overall confidence obtained in step S3.1 is generated. Extract the optimal learnable vector obtained from the previous batch. , respectively used for the selected For each image sample, the inference loss is calculated as follows: The performance degradation ratio is calculated using the formula below. : ； in, This represents the loss value obtained after inference using learnable vectors. Represents absolute value; obtains It is then used in the following formula for the misuse rate. Calculation: ； in, This indicates the percentage of cases that meet the conditions in parentheses. This represents the performance degradation percentage threshold, calculated in this study. This will be used for updates after the current sliding cycle ends. The calculation; since the cache is empty when step S3.2 is executed for the first time, the cache cannot be hit, so step S3.4 will not be executed and is not involved. and The calculation; if the following conditions are met, online evolutionary search will not be performed, and the result will be taken. The best learnable vector for the current batch enters the inference execution unit for inference calculation. After the calculation is completed, it jumps back to step S1 to start processing the next batch of image samples. ； If the above formula is not satisfied, the optimal learnable vector obtained in the previous batch will be... Input perturbation adaptive fine-tuning unit, perform a finite number of evolutionary searches: ； in Indicated by The learnable vector obtained by performing a finite number of evolutionary searches starting from the evolutionary starting point. This indicates that a finite number of evolutionary search operations are performed using existing vectors. After the evolutionary search is completed, the learnable vector obtained by the evolutionary search is taken as the optimal learnable vector for this batch, and used as the value. This vector, along with the key, forms a cache entry and is input into the cache update unit; then step S4 is executed.

6. The method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 5, characterized in that, The lightweight adaptive initialization unit receives the currently obtained learnable vectors and image samples of the current batch. The ViT backbone network performs inference calculations on the image samples of the current batch, calculates the loss of each learnable vector during inference, takes the learnable vector with the minimum loss as the starting point of the evolutionary search, performs evolutionary search based on the CMA-ES method, and outputs the optimal learnable vector of the current batch.

7. The method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 5, characterized in that, The cache perturbation adaptive unit receives the learnable vectors from the hit cache entries. The ViT backbone network performs inference calculations on the current batch of image samples, calculates the loss during inference of each learnable vector, performs evolutionary search based on the CMA-ES method, and outputs the optimal learnable vector for the current batch.

8. The method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 5, characterized in that, The perturbation adaptive fine-tuning unit receives the best learnable vector obtained from the previous batch. The ViT backbone network performs inference calculations on the image samples of the current batch, calculates the loss during inference of each learnable vector, performs evolutionary search based on the CMA-ES method, and outputs the best learnable vector of the current batch.

9. A method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 5, characterized in that, The specific steps of step S4 are as follows: Step S4.1: The cache update unit receives the cache entries obtained in the current batch and calculates the cache entries according to the following formula. Replacement fractions : ； ； in, Indicates cached entries Cumulative hit count since being added to the cache space express normalized frequency, This indicates the timestamp of the last successful reuse or verification of a cached entry. express The reuse freshness obtained by normalization This represents the maximum number of hits for all entries in the current cache. This indicates the timestamp of the most recent update in the current cache. This represents the timestamp of the oldest item in the current cache; additionally, The global scaling factor, defined as the replacement fraction, is calculated using the following formula: ； in, Indicates the distribution offset detection factor; Step S4.2: Based on the replacement score Determine whether to replace cache entries in the cache space with cache entries obtained in the current batch, and then determine the replacement score in the cache space. The smallest cache entry, if its replacement score Replacement score of cache entries less than the current batch If the cached entry is obtained in the current batch, the cached entry in the cache space will be replaced; otherwise, the cache space will not be updated. Then, the inference execution unit will be entered.

10. A method for implementing a reusable caching mechanism for test-time adaptive scenarios according to claim 9, characterized in that, The inference execution unit receives the best learnable vector for the current batch and the image samples for the current batch. The ViT backbone network performs inference to obtain the final inference result for the current batch.

Images