A collaborative continuous testing time adaptation method for edge environments

Abstract

This invention proposes a collaborative continuous test-time adaptation method for edge environments, belonging to the fields of edge intelligence and model continuous test-time adaptation technology. Through cloud-edge collaboration and two-stage learning, it achieves continuous and stable adaptation of edge device models to dynamic data distributions under resource-constrained conditions. Image inversion generation technology is used in the cloud to synthesize images and preserve historical knowledge; a pre-trained model including a frozen backbone and lightweight branch networks is deployed on the edge device. A two-stage replay mechanism is introduced, decoupling active forgetting and knowledge integration into two independent stages. This allows the edge model to first focus on learning new task information and then collaboratively integrate it with historical knowledge, thus balancing high plasticity and stability during continuous adaptation. This invention achieves continuous test-time adaptation in a storage-efficient manner while maintaining high plasticity and stability during continuous adaptation, supporting the long-term stable performance of edge models in dynamic environments.

Description

Technical Field

[0001] This invention belongs to the field of edge intelligence and model continuous testing adaptation technology, and particularly relates to a collaborative continuous testing adaptation method for edge environments. Background Technology

[0002] Edge intelligence is a technological paradigm that deploys intelligent models on edge devices that generate data, enabling local real-time processing and decision-making. It has significant value in numerous application scenarios, such as autonomous driving and real-time monitoring. However, although the deployment and inference of models on edge devices and on-device learning have been extensively studied, their adaptation during continuous testing in dynamic environments still lacks in-depth research.

[0003] While resource-constrained on-device learning methods, such as adding lightweight branch networks, can effectively reduce storage and computational overhead, their design is primarily geared towards single-cycle learning and does not adequately consider the need for continuous adaptation of the model in continuously changing target domains. On the other hand, although on-test adaptation methods have shown significant potential in dealing with data distribution shifts, current mainstream methods often consume high storage overhead and have not fully considered the memory constraints of edge devices, making it difficult to support the storage efficiency requirements of edge devices during long-term continuous adaptation.

[0004] Furthermore, current adaptation methods are generally susceptible to catastrophic forgetting during long-term adaptation: models often lose learned knowledge when adapting to new domains, leading to a gradual degradation in overall performance over time, which in turn affects their long-term deployment feasibility in real-world environments. Therefore, there is an urgent need for a continuous test-time adaptation framework that balances storage efficiency and resistance to forgetting to maintain the long-term stable performance of edge device models in dynamic environments. Summary of the Invention

[0005] The purpose of this invention is to provide a collaborative, continuous test-time adaptation method for edge environments, enabling edge device models to maintain long-term stable operation under resource-constrained conditions to cope with dynamic shifts in data distribution. This addresses the technical problems of existing test-time adaptation methods in edge environments, such as excessive memory consumption and catastrophic forgetting during continuous adaptation.

[0006] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0007] A collaborative continuous testing adaptation method for edge environments, the method comprising the following steps:

[0008] Step S1: Obtain the unlabeled dataset, define the task sequence, deploy the pre-trained edge model on the edge device, initialize the generation model in the cloud, and initialize the cloud-edge collaborative system;

[0009] Step S2: The edge device receives the unlabeled dataset; after saving the current edge model parameter state and uploading the current edge model and parameter state to the cloud, it starts active forgetting learning, so that the edge model focuses on learning the distribution characteristics of the new task data; after the edge model has adapted to the new data distribution, the updated edge model is obtained, and the updated edge model generates pseudo-labels for the unlabeled dataset to obtain images with pseudo-labels.

[0010] Step S3: The cloud receives the edge model uploaded by the edge device, uses it as a discriminator to guide the training of the generation model, optimizes the generation model through image inversion generation technology, synthesizes a pseudo-labeled synthetic image, selects representative pseudo-labeled synthetic images, and sends the pseudo-labeled representative synthetic images to the edge device.

[0011] Step S4: In the real learning phase of the edge device, the edge device first restores the edge model parameter state to the initial model parameter state saved in the active forgetting phase. Then, it summarizes the pseudo-labeled images of the current task and the representative synthetic images with pseudo-labels sent from the cloud to build an integrated dataset. The edge model is trained on the integrated dataset to achieve the fusion of knowledge of the current task and knowledge of the historical task, thus completing this round of adaptation.

[0012] Step S5: In the new task adaptation phase, when the edge device receives unlabeled data with distribution change characteristics from the new task domain, it uses the edge model state after the previous round of task completion as the initial state and re-executes steps S2 to S4 to achieve rapid adaptation to the new task distribution. By repeatedly executing the above process, the cloud-edge collaborative system can continuously learn and adapt in a dynamic environment.

[0013] Further, step S1 includes the following steps:

[0014] Step S11: Continuously acquire image sequences to obtain several unlabeled datasets, define a task for each unlabeled dataset, and obtain a continuous task sequence;

[0015] Step S12: Deploy the pre-trained edge model on the edge device and initialize the generative model in the cloud. The edge model and the generative model together constitute the cloud-edge collaborative system.

[0016] Furthermore, the pre-trained edge model consists of a frozen backbone network and an updatable lightweight branch network. The backbone network is composed of 12 stacked residual blocks, each containing two convolutional layers, two batch normalization layers, and two ReLU activation functions. The lightweight branch network is composed of meta-networks, which are stacked with batch normalization layers and standard convolutional blocks. The residual blocks of the backbone network are divided into 4 or 5 segments, and the number of meta-networks is the same as the number of segments in the backbone network. Each meta-network is integrated into a residual segment in an appended manner.

[0017] Further, step S2 includes the following steps:

[0018] Step S21: The edge device receives the unlabeled dataset; saves the current edge model and parameter status, and uploads the current edge model and parameter status to the cloud;

[0019] Step S22: The edge device enters the active forgetting stage. The edge device updates the lightweight branch network based on the unlabeled dataset of the current task to obtain the updated edge model.

[0020] Step S23: Use the updated edge model to generate pseudo-labels for the unlabeled dataset of the current task, and obtain images with pseudo-labels.

[0021] Furthermore, in step S22, the edge device updates the lightweight branch network based on the unlabeled dataset of the current task, using the information maximization loss function. The information maximization loss function is used to optimize the edge model as follows:

[0022]

[0023]

[0024]

[0025]

[0026] in, This represents the information maximization loss function; This represents the loss function that minimizes entropy; Represents the diversity loss function; To start from the current unlabeled dataset The batch size of the sampled data is Unlabeled image batches; , Represents the first unlabeled image sampled. One image; This represents the total number of categories, i.e., the category dimension. Representing the edge model For images Predicted as the first The probability of a class; Representing the edge model For images The predicted probability distribution; Represents the discrete probability distribution of an image; Discrete probability distribution of the image Information entropy.

[0027] Further, step S3 includes the following steps:

[0028] Step S31: The cloud receives the edge model uploaded by the edge device. and parameter status , edge model As a discriminator, it is used to guide the generative model. Training;

[0029] Step S32: Train the generative model based on image inversion generation technology; Step S32 includes steps S321 to S322:

[0030] Step S321: Input random noise with the same dimension as the category. To generative models To obtain a synthetic image ;

[0031] Step S322: Combine the images Input discriminator The loss function is calculated based on the discriminator's prediction results, and then applied to the generative model. Perform reverse optimization;

[0032] Step S33: In generating the model After training, each synthesized image The corresponding pseudo-labels can be obtained after prediction by the discriminator. ; Calculate the representative score of each synthetic image; Select representative synthetic images with representative scores exceeding the threshold score and their corresponding pseudo-labels from the synthetic images to form representative synthetic images with pseudo-labels, and send the representative synthetic images with pseudo-labels to the edge devices.

[0033] Further, step S4 includes the following steps:

[0034] Step S41: The edge device restores the current edge model parameter state to the initial model parameter state saved during the active forgetting stage;

[0035] Step S42: The edge device merges the pseudo-labeled image predicted by the current task with the representative synthetic image with pseudo-label received from the cloud to obtain an integrated dataset for knowledge integration.

[0036] Step S43: Apply edge models to the integrated dataset The model is trained to integrate new knowledge learned from the current task with historical task knowledge extracted from synthetic images. After training, a final edge model is obtained that can adapt to the current task while mitigating the forgetting of historical knowledge. .

[0037] Compared with the prior art, the present invention has the following beneficial technical effects:

[0038] 1) To address the issues of excessive memory consumption and catastrophic forgetting in existing adaptive methods on edge devices during testing, this invention designs a cloud-edge collaborative framework that uses cloud-based generative models to synthesize images to approximate historical task data. This effectively alleviates knowledge forgetting in edge models during the learning process while avoiding the transmission of original data, significantly reduces memory consumption on edge devices, and protects user privacy.

[0039] 2) To address the dilemma of model plasticity-stability trade-off caused by the difference in distribution between synthetic images and real task data, this invention proposes a two-stage replay mechanism, which decouples active forgetting and knowledge integration into two independent stages. This allows the marginal model to first focus on learning new task information and then integrate it with historical knowledge, thereby balancing high plasticity and stability in the process of continuous adaptation.

[0040] 3) This invention can achieve continuous testing adaptation in a storage-efficient manner, while taking into account high plasticity and stability during continuous adaptation, and supporting the long-term stable performance of edge models in dynamic environments. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the storage-efficient collaborative continuous testing adaptation method for edge environments according to the present invention.

[0043] Figure 2 This is a schematic diagram illustrating the principle of the storage-efficient collaborative continuous testing adaptation method for edge environments according to the present invention.

[0044] Figure 3 This is a schematic diagram illustrating the process of adapting to new tasks in this invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] This invention aims to propose a collaborative continuous test-time adaptation method for edge environments, overcoming the shortcomings of existing test-time adaptation methods, such as excessive memory consumption and catastrophic forgetting during continuous adaptation. The overall process of the method is as follows: Figure 1 As shown, S1: Define the task sequence, initialize the cloud-edge collaborative system, deploy the pre-trained model on the edge device, and initialize the generation model in the cloud; S2: The edge device initiates active forgetting. After receiving real-time data, the edge device first saves the initial model state and uploads the initial model parameters to the cloud. Subsequently, it updates the lightweight branch network based on the current task data and uses the updated edge model to generate pseudo-labels for the current task data; S3: In the cloud generation model training and synthetic image generation stage, the cloud receives the edge model uploaded by the edge device and uses it as a discriminator to guide the training of the generation model. The generation model is optimized through image inversion generation technology to synthesize high-quality pseudo-labeled synthetic images. The discriminator selects synthetic images with high representativeness and confidence as representative synthetic images and sends these pseudo-labeled synthetic images to the edge device; S4: In the edge device's real learning stage, the edge device first restores the edge model state to the initial state saved in the active forgetting stage, and then summarizes the new task data with pseudo-labels and the representative synthetic images with pseudo-labels sent from the cloud to construct an integrated dataset. The edge model is trained on this integrated dataset to fuse knowledge of the current task with knowledge of historical tasks, completing this round of adaptation. In the new task adaptation phase (S5), when the edge device receives unlabeled data with changing distribution characteristics from the new task domain, it uses the edge model's state after the previous task as the initial state and re-executes steps S2 to S4, thereby achieving rapid adaptation to the new task distribution. By repeatedly executing the above process, the cloud-edge collaborative system can continuously learn and adapt in a dynamic environment, enabling the edge model to maintain high storage efficiency and performance stability on resource-constrained edge devices over the long term.

[0047] The method of the present invention will be described in detail below, such as Figure 1-2 As shown, the method includes the following steps:

[0048] Step S1: Obtain the unlabeled dataset, define the task sequence, deploy the pre-trained edge model on the edge device, initialize the generation model in the cloud, and initialize the cloud-edge collaborative system.

[0049] This invention uses an intelligent connected autonomous driving system as its application scenario, defining onboard cameras and edge computing units as edge devices for real-time processing of visual data and execution of inference tasks. The task sequence consists of dynamic visual data streams captured by the onboard cameras during continuous vehicle operation, covering image sequences generated under different temporal and environmental conditions such as urban roads, highways, tunnels, rain and snow, and day / night cycles. The cloud-edge collaborative system comprises edge devices and a cloud server. Through a cloud-edge collaboration mechanism, the onboard model can continuously adapt to dynamically changing driving environments without the need for source domain data and manual annotation, ensuring the environmental perception stability of the autonomous driving system during long-term operation.

[0050] Step S11: The vehicle-mounted camera continuously acquires image sequences to obtain several unlabeled datasets. A task is defined for each unlabeled dataset to obtain a continuous task sequence.

[0051] No. An unlabeled dataset is represented as All unlabeled datasets constitute a continuous unlabeled dataset, denoted as... , Represents the number of unlabeled datasets, the th The task representation for each unlabeled dataset is as follows: All tasks constitute a continuous task sequence, represented as: ,in, Indicates the first One task.

[0052] Each unlabeled dataset includes The images follow a specific data distribution. The data distributions of different unlabeled datasets are not the same, that is, for any All have , Indicates the first The data distribution that an unlabeled dataset follows is... Indicates the first The data distribution that an unlabeled dataset follows.

[0053] The above settings can be used to simulate the continuous and dynamically changing data environment faced by edge devices during long-term operation after deployment, thereby providing a testing basis for the adaptive learning and performance maintenance of edge models in non-stationary scenarios.

[0054] Step S12: Deploy the pre-trained edge model on the edge device, initialize the generation model in the cloud, and initialize the cloud-edge collaborative system.

[0055] The pre-trained edge model deployed on edge devices and the generative model initialized in the cloud together constitute the cloud-edge collaborative system.

[0056] Deploying pre-trained edge models on edge devices Its initial parameter state is denoted as Let the edge model be defined for the first... A list of unlabeled datasets The parameter state after adaptation is In the beginning of adaptive... The parameter state of the previous unlabeled dataset was: As the edge model sequentially receives unlabeled datasets from a series of consecutive tasks... The edge model parameter state will change from the initial parameter state. Gradually update to the final parameter state This enables the edge model to continuously adapt in dynamic environments.

[0057] The pre-trained edge model consists of a frozen backbone network and an updatable lightweight branch network.

[0058] The backbone network employs a pre-trained deep convolutional neural network architecture, specifically the WideResNet-28 model, to extract multi-level features from the input image. The backbone network consists of 12 stacked residual blocks, each containing two convolutional layers, two batch normalization layers, and two ReLU activation functions. A width factor is used to expand the number of channels to enhance feature representation capabilities.

[0059] The input of each residual block is processed sequentially through the first batch of normalization layers, the first ReLU activation function, the first convolutional layer, the second batch of normalization layers, the second ReLU activation function, and the second convolutional layer, and then the original input of the residual block is added to obtain the final output of the residual block.

[0060] During the pre-training phase, the backbone network is trained on a large image classification dataset and its weights are fixed to avoid additional storage and computational overhead when edge devices adapt to new tasks.

[0061] The lightweight branch network is composed of a meta-network, which consists of batch normalization layers and standard convolutional blocks stacked together. The input of the meta-network is processed by the batch normalization layers and standard convolutional blocks in sequence to obtain the output of the meta-network.

[0062] In one alternative implementation, the lightweight branch network includes four meta-networks, in which case the backbone network is divided into four residual segments with two residual blocks, two residual blocks, four residual blocks, and four residual blocks, and each meta-network is integrated into a residual segment in an additional manner.

[0063] In another alternative implementation, the lightweight branch network contains 5 meta-networks, in which case the backbone network is divided into five residual segments with 2 residual blocks, 2 residual blocks, 2 residual blocks, 2 residual blocks, and 4 residual blocks, and each meta-network is integrated into a residual segment in an additional manner.

[0064] The inputs to the edge model are fed into the first residual block and the meta-network of the first residual segment. The meta-network output is obtained after processing. The output of the meta-network is obtained after cascading processing of each residual block in the first residual segment. The output of the meta-network and the output of the residual block are then fused to obtain the output of the first residual segment. This process is repeated. The output of the first residual segment is fed into the first residual block and the meta-network of the second residual segment to obtain the output of the second residual segment. Finally, the output of the last residual segment is obtained as the output of the edge model.

[0065] When the pre-trained edge model adapts to new task data, the backbone network parameters remain frozen, and only the trainable parameters in a few meta-networks are updated. This design effectively reduces the intermediate activation values ​​that need to be stored during backpropagation, thereby significantly reducing storage overhead.

[0066] Randomly initialize and generate models in the cloud. The generative model uses the GAN (Generative Adversarial Network) architecture.

[0067] Step S2: The edge device receives the unlabeled dataset; after saving the current edge model parameter state and uploading the current edge model and parameter state to the cloud, it starts active forgetting learning, so that the edge model focuses on learning the distribution characteristics of the new task data; after the edge model has adapted to the new data distribution, the updated edge model generates pseudo-labels for the unlabeled dataset to obtain pseudo-labeled images.

[0068] Step S21: The edge device receives the unlabeled dataset; saves the current edge model and parameter status, and uploads the current edge model and parameter status to the cloud.

[0069] When the edge device receives the first Unlabeled datasets for each task First, save the current model parameter state. Let the initial model parameter state be denoted as . The current model parameter state indicates that the edge model has not yet undergone the first step. Task The model adaptation can be reloaded as the initial state for integrating new and old knowledge in subsequent real learning stages. Then, the current edge model... and parameter status Uploaded to a cloud server, it guides the training of the generative model to synthesize image data that conforms to the distribution characteristics of historical tasks, thus providing a foundation for knowledge replay.

[0070] Step S22: The edge device enters the active forgetting stage. The edge device updates the lightweight branch network based on the unlabeled dataset of the current task to obtain the updated edge model.

[0071] Based on information maximization loss Optimize the edge model to focus on learning the current task. Unlabeled datasets The distribution characteristics are considered without taking into account the forgetting of historical knowledge. In this stage, only the trainable parameters in the lightweight branch network are updated, while the weights of the backbone network remain frozen during backpropagation, thus effectively controlling memory usage and maintaining the adaptability of the edge model. The optimized edge model is then obtained, adapted to the new task. .

[0072] Information maximization loss function Defined as:

[0073]

[0074] in, To start from the current unlabeled dataset The batch size of the sampled data is Unlabeled image batches, , Represents the first unlabeled image sampled. One image; The entropy minimization loss is used to encourage the marginal model to make high-confidence predictions on the input data. To mitigate diversity loss, the information entropy of the batch prediction distribution is maximized to encourage marginal models to generate more diverse predictions. Both are specifically defined as follows:

[0075]

[0076]

[0077]

[0078] In the formula, For batch size, This represents the total number of categories, i.e., the category dimension. Representing the edge model For images Predicted as the first The probability of a class; Representing the edge model For images The predicted probability distribution; Represents the discrete probability distribution of an image. , Indicates the image number 1 The average predicted probability of the class; Discrete probability distribution of the image Information entropy is defined as .

[0079] Step S23: Generate pseudo-labels for the unlabeled dataset of the current task using the updated edge model.

[0080] Complete the current task After learning, the optimized edge model is used. Data on this task Generating pseudo-labels for each image; Current task of Each image is represented as Specifically, each image Through edge model Predict the most likely category as the pseudo-label to obtain the pseudo-labeled image for the current task. :

[0081]

[0082] in, Represents a pseudo-label vector, containing The predicted category for each image; This indicates that the index of the class corresponding to the highest probability in the model's output probability distribution is taken.

[0083] Step S3: The cloud receives the edge model uploaded by the edge device and uses it as a discriminator to guide the training of the generative model. The generative model is optimized through image inversion generation technology to synthesize high-quality images with pseudo-labels. The discriminator selects the synthetic images with high representativeness and confidence as representative synthetic images and sends the representative synthetic images with pseudo-labels to the edge device.

[0084] Step S31: The cloud receives the edge model uploaded by the edge device. and parameter status , edge model As a discriminator, it is used to guide the generative model. Training.

[0085] Step S32: Train the generative model based on image inversion generation technology.

[0086] Step S321: Input random noise with the same dimension as the category. To generative models To obtain a synthetic image .

[0087]

[0088] in, Indicates random noise; Represents a generative model; Indicates a composite image. , Indicates the first A composite image, This indicates the batch size of the composite images.

[0089] Step S322: Combine the images Input discriminator Based on the discriminator's prediction results, a multinomial loss function is calculated for the generative model. Perform reverse optimization.

[0090] The optimization objective specifically includes the following four loss terms:

[0091] To ensure that the synthesized image output by the generative model can be accurately classified as noise by the discriminator. For the corresponding categories, define the cross-entropy loss:

[0092]

[0093] in, Represents cross-entropy loss, Represents the cross-entropy loss function. Indicates noise The index corresponding to the maximum value in the middle is used as the composite image. Target category label, For the discriminator to analyze the synthesized image The predicted probability distribution.

[0094] To encourage generative models to cover all categories and synthesize diverse images, a diversity loss is defined:

[0095]

[0096]

[0097] in, Represents the diversity loss function; This represents the discrete probability distribution of the synthesized image. , Indicates the composite image of the first The average predicted probability of the class; Indicates the batch size of the synthesized images. Represents the discrete probability distribution of the synthesized image Information entropy is defined as ; Indicates the first A composite image; For the discriminator to the first A composite image The predicted probability distribution.

[0098] To align the statistical distribution of the synthesized image in the feature space with that of the real image, thereby improving the realism of the synthesized image, a hierarchical batch normalization loss is defined:

[0099]

[0100] in, This represents the hierarchical batch normalized loss function. Discriminator The total number of normalized layers in the middle batch; This represents the KL divergence, used to measure the difference between two probability distributions; , Discriminators No. The running mean and variance stored in each batch normalization layer represent the characteristic distribution of historical real data. , This is the number of times the current synthesized image is propagated forward. The mean and variance of each batch normalization layer represent the feature distribution of the synthesized image. The hierarchical batch normalization loss minimizes the KL divergence between these two sets of statistics, thus ensuring the optimal feature distribution of the synthesized image. Towards the true distribution To move closer.

[0101] To encourage generative models to synthesize visually natural images, i.e., to ensure that the synthesized images are smooth in pixel space, a synthesized image smoothing loss is defined:

[0102]

[0103] in, This represents the loss function for smoothing the synthesized image. Represents a composite image The smoothed version after Gaussian filtering. The synthetic image smoothing loss calculates the L2 distance between the synthetic image and its smoothed version, which is used as a regularization term to suppress unnatural pixel value abrupt changes.

[0104] Ultimately, the generative model The optimization objective is a weighted combination of the above four loss functions, forming a weighted loss function:

[0105]

[0106] in, This indicates that the parameters of the generative model are optimized to minimize the weighted loss function; , , , These are the non-negative weighting coefficients for cross-entropy loss, diversity loss, hierarchical batch normalization loss, and synthetic image smoothing loss, respectively, used to balance the relative contributions of different losses during training.

[0107] Step S33: Selection and distribution of representative synthetic images. In the generative model... After training, each synthesized image The corresponding pseudo-labels can be obtained after prediction by the discriminator. :

[0108]

[0109] in, Represents a composite image The corresponding pseudo-tags; Discriminator For synthetic images The predicted probability distribution; This indicates the index of the class corresponding to the highest probability in the probability distribution output by the discriminator.

[0110] To further evaluate the quality of the synthesized image, a discriminator was used. The predicted probability distribution of the output synthetic images is used to calculate the representative score (RS) for each synthetic image:

[0111]

[0112] in, This indicates that the representative score is being sought. Denotes KL divergence, Discriminator For synthetic images The predicted probability distribution; This represents the average predicted probability distribution of the synthesized images in the current batch, i.e. , This represents the batch size of the synthetic images. The representativeness score, measured by the KL divergence between the predicted probability distribution of a single synthetic image and the batch average predicted probability distribution, characterizes the discriminativeness and confidence of the synthetic images. Synthetic images with higher scores typically possess more explicit class-discriminating features and higher prediction confidence, and are therefore considered high-quality representative synthetic images. Finally, representative synthetic images with representativeness scores exceeding a threshold score, along with their corresponding pseudo-labels, are selected from the synthetic images to constitute pseudo-labeled representative synthetic images. , The pseudo-label of the synthetic image represents the representative synthetic image whose score exceeds the threshold score. The threshold score is set reasonably as needed and then sent to the edge device for subsequent data playback and continuous adaptive updating of the model.

[0113] Step S4: In the edge device's real learning phase, the edge device first restores the edge model's parameter state to the initial model parameter state saved during the active forgetting phase. Then, it aggregates the pseudo-labeled data for the current task with the representative pseudo-labeled synthetic images distributed from the cloud to construct an integrated dataset. The edge model is trained on this integrated dataset to achieve the fusion of knowledge from the current task and historical tasks, completing this round of adaptation.

[0114] Step S41: The edge device restores the initial model parameter state. The edge device first restores the current edge model parameter state to the initial model parameter state saved in step S21. This ensures that the learning process starts from a stable initial state:

[0115]

[0116] Step S42: Construct the integrated dataset. This involves combining the pseudo-labeled images predicted based on the current task in step S23. The representative synthetic image with pseudo-labels received from the cloud in step S33 Merge the datasets to build an integrated dataset for knowledge integration. :

[0117]

[0118] in, This represents all the input data after concatenation, including the current task image. and representative synthetic images ; This indicates a tensor concatenation operation; This represents the concatenated pseudo-label vector, containing pseudo-labels for the current task data. and representative synthetic image pseudo-labels .

[0119] Step S43: Perform real learning. Integrate the dataset... Upper edge model Training is performed to integrate new knowledge learned from the current task with historical task knowledge extracted from synthetic images. During training, only the trainable parameters in the lightweight branch network are updated, while the weights of the backbone network remain frozen during backpropagation, thus effectively controlling memory overhead while achieving knowledge integration. After training, a final edge model is obtained that can adapt to the current task while mitigating the forgetting of historical knowledge. This stage uses standard cross-entropy loss as the optimization objective:

[0120]

[0121] in, Represents cross-entropy loss; Represents the cross-entropy loss function; This indicates the current edge model's response to all concatenated input data. The predicted probability distribution This is the truth value of the corresponding pseudo-label.

[0122] Step S5: In the new task adaptation phase, when the edge device receives unlabeled data with distributional variation characteristics from the new task domain, it uses the edge model state after the previous round of task completion as the initial state and re-executes steps S2 to S4, thereby achieving rapid adaptation to the new task distribution. By repeatedly executing the above process, the cloud-edge collaborative system can continuously learn and adapt in a dynamic environment, enabling the edge model to maintain storage efficiency and performance stability on resource-constrained edge devices in the long term.

[0123] The cloud-edge collaboration framework is applied to the continuous testing and adaptation process of the edge model.

[0124] For new task data arriving in the data stream, the edge device saves the initial model parameter state and uploads it to the cloud. Under the active forgetting mechanism, it focuses on learning the distribution characteristics of the new data, updates the lightweight branch network, and uses the updated edge model to generate pseudo-labels for this batch of task data.

[0125] The cloud receives model parameter status uploaded by edge devices, trains generative models based on image inversion generation technology, synthesizes synthetic images that can replace the distribution of historical task data, selects representative synthetic images from the synthetic images, assigns pseudo-labels to the representative synthetic images, and sends the pseudo-labeled representative synthetic images to the edge devices.

[0126] Edge devices receive representative synthetic images with pseudo-labels from the cloud, aggregate them with pseudo-labeled images for the current task, and form an integrated dataset. The edge model is trained on this integrated dataset, thus completing the current round of adaptation.

[0127] During the continuous adaptation process, the edge device continuously receives unlabeled data from different task domains with varying distributions. As new task data arrives, this process is repeated cyclically to achieve continuous adaptation between the cloud-generated model and the edge model, enabling continuous synthesis of images.

[0128] When a new task arrives, the storage-efficient collaborative continuous testing framework for edge environments enables rapid adaptation to the new task. The edge device uses the model parameter state saved after the completion of the previous task as the initial state and re-executes steps S2 to S4, thereby fully preserving historical knowledge while adaptively updating the distribution of the new task.

[0129] like Figure 3 As shown, for example, given an unlabeled dataset from a new task domain. The system's model parameter state after the previous round of tasks was completed. As the initial state, the following steps are executed sequentially: active forgetting learning phase for this task, cloud-based generative model training and synthetic image generation phase, and real-world learning phase on edge devices, to obtain an edge model adapted to the new task distribution. .

[0130] By repeating the above process, the cloud-edge collaborative system can continuously learn and adapt in a dynamically changing real environment, enabling the model to achieve long-term storage efficiency and performance stability on resource-constrained edge devices.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for edge environment oriented, collaborative, continuous testing time adaptation, characterized in that, The method includes the following steps: Step S1: Obtain the unlabeled dataset, define the task sequence, deploy the pre-trained edge model on the edge device, initialize the generation model in the cloud, and initialize the cloud-edge collaborative system; Step S2: The edge device receives the unlabeled dataset; after saving the current edge model parameter state and uploading the current edge model and parameter state to the cloud, it starts active forgetting learning, so that the edge model focuses on learning the distribution characteristics of the new task data; Once the edge model has adapted to the new data distribution, an updated edge model is obtained. The updated edge model is then used to generate pseudo-labels for the unlabeled dataset, resulting in images with pseudo-labels. Step S3: The cloud receives the edge model uploaded by the edge device, uses it as a discriminator to guide the training of the generation model, optimizes the generation model through image inversion generation technology, synthesizes a pseudo-labeled synthetic image, selects representative pseudo-labeled synthetic images, and sends the pseudo-labeled representative synthetic images to the edge device. Step S4: In the real learning phase of the edge device, the edge device first restores the edge model parameter state to the initial model parameter state saved in the active forgetting phase. Then, it summarizes the pseudo-labeled images of the current task and the representative synthetic images with pseudo-labels sent from the cloud to build an integrated dataset. The edge model is trained on the integrated dataset to achieve the fusion of knowledge of the current task and knowledge of the historical task, thus completing this round of adaptation. Step S5: In the new task adaptation phase, when the edge device receives unlabeled data with distribution change characteristics from the new task domain, it uses the edge model state after the previous round of task completion as the initial state and re-executes steps S2 to S4 to achieve rapid adaptation to the new task distribution. By repeatedly executing the above process, the cloud-edge collaborative system can continuously learn and adapt in a dynamic environment.

2. The method of claim 1, wherein the method is adapted for edge environment, characterized in that, Step S1 includes the following steps: Step S11: Continuously acquire image sequences to obtain several unlabeled datasets, define a task for each unlabeled dataset, and obtain a continuous task sequence; Step S12: Deploy the pre-trained edge model on the edge device and initialize the generative model in the cloud. The edge model and the generative model together constitute the cloud-edge collaborative system.

3. The method of claim 2, wherein the method is adapted for edge environment, characterized in that, The pre-trained edge model consists of a frozen backbone network and an updatable lightweight branch network. The backbone network is composed of 12 stacked residual blocks, each containing two convolutional layers, two batch normalized layers, and two ReLU activation functions. The lightweight branch network is composed of meta-networks, which are stacked with batch normalized layers and standard convolutional blocks. The residual blocks of the backbone network are divided into 4 or 5 segments, and the number of meta-networks is the same as the number of segments in the backbone network. Each meta-network is integrated into a residual segment in an append-only manner.

4. The method of claim 3, wherein the method is adapted for edge environment, characterized in that, Step S2 includes the following steps: Step S21: The edge device receives the unlabeled dataset; saves the current edge model and parameter status, and uploads the current edge model and parameter status to the cloud; Step S22: The edge device enters the active forgetting stage. The edge device updates the lightweight branch network based on the unlabeled dataset of the current task to obtain the updated edge model. Step S23: Use the updated edge model to generate pseudo-labels for the unlabeled dataset of the current task, and obtain images with pseudo-labels.

5. The collaborative continuous testing adaptation method for edge environments according to claim 4, characterized in that, In step S22, the edge device updates the lightweight branch network based on the unlabeled dataset of the current task, using the information maximization loss function. The information maximization loss function is used to optimize the edge model as follows: in, This represents the information maximization loss function; This represents the loss function that minimizes entropy; Represents the diversity loss function; To start from the current unlabeled dataset The batch size of the sampled data is Unlabeled image batches; , Represents the first unlabeled image sampled. One image; This represents the total number of categories, i.e., the category dimension. Representing the edge model For images Predicted as the first The probability of a class; Representing the edge model For images The predicted probability distribution; Represents the discrete probability distribution of an image; Discrete probability distribution of the image Information entropy.

6. The collaborative continuous testing adaptation method for edge environments according to claim 5, characterized in that, Step S3 includes the following steps: Step S31: The cloud receives the edge model uploaded by the edge device. and parameter status , edge model As a discriminator, it is used to guide the generative model. Training; Step S32: Train the generative model based on image inversion generation technology; Step S32 includes steps S321 to S322: Step S321: Input random noise with the same dimension as the category. To generative models To obtain a synthetic image ; Step S322: Combine the images Input discriminator The loss function is calculated based on the discriminator's prediction results, and then applied to the generative model. Perform reverse optimization; Step S33: In generating the model After training, each synthesized image The corresponding pseudo-labels can be obtained after prediction by the discriminator. ; Calculate the representative score of each synthetic image; Select representative synthetic images with representative scores exceeding the threshold score and their corresponding pseudo-labels from the synthetic images to form representative synthetic images with pseudo-labels, and send the representative synthetic images with pseudo-labels to the edge devices.

7. The collaborative continuous testing adaptation method for edge environments according to claim 6, characterized in that, Step S4 includes the following steps: Step S41: The edge device restores the current edge model parameter state to the initial model parameter state saved during the active forgetting stage; Step S42: The edge device merges the pseudo-labeled image predicted by the current task with the representative synthetic image with pseudo-label received from the cloud to obtain an integrated dataset for knowledge integration. Step S43: Apply edge models to the integrated dataset The model is trained to integrate new knowledge learned from the current task with historical task knowledge extracted from synthetic images. After training, a final edge model is obtained that can adapt to the current task while mitigating the forgetting of historical knowledge. .

Images