Self-adaptive federal continuous learning method and device for resource-constrained client

The adaptive federated continuous learning method addresses data drift and privacy issues in IoT devices by splitting the global model into task-specific components, ensuring efficient and accurate continuous learning with reduced resource consumption.

CN119090023APending Publication Date: 2024-12-06NAT UNIV OF DEFENSE TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410878657.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-02
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Resource-constrained IoT devices face challenges in continuous learning due to data drift and privacy concerns, as traditional centralized methods are inefficient and invasive.

Method used

An adaptive federated continuous learning method that splits the global model into a task robust encoder and a task sensitive decoder, allowing iterative training and data drift detection, with the encoder shared across tasks and decoder updated locally, while ensuring privacy through decentralized storage and aggregation.

Benefits of technology

Enables efficient, adaptive continuous learning on resource-constrained devices with reduced storage and computing overhead, maintaining model accuracy and privacy by leveraging a lightweight decoder pool and data drift detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090023A_ABST
    Figure CN119090023A_ABST
Patent Text Reader

Abstract

The invention discloses a resource-constrained client-oriented adaptive federal continuous learning method and device. The method comprises the following steps: receiving a task robust encoder and a task sensitive decoder issued by a server after decoding a global model; iteratively training a global model consisting of a task robust encoder and a task sensitive decoder according to local current task data, uploading the trained task robust encoder and task sensitive decoder to a server, and aggregating the trained task robust encoder and task sensitive decoder in the server to obtain a new global model; in the federal iteration training process, if one time of local training is completed, data drift detection is carried out, and if data drift occurs, the task sensitive decoders are reinitialized, and the stored task sensitive decoder pool is updated. Through the above mode, the embodiment of the invention can be applied to the occasion of federal continuous learning on the premise of communication privacy protection, the storage and calculation overhead of the client is greatly reduced, and the resource utilization efficiency of the continuous learning process is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of federated learning processing, and in particular to an adaptive federated continuous learning method and device for a resource-constrained client. Background Art

[0002] With the rapid development of the Internet of Things technology, a large number of user terminals such as mobile phones and tablets have emerged, forming a distributed intelligent computing landscape. Under this distributed computing framework, different terminals generate a large amount of data such as pictures, text, and audio every day. As time goes by, the data collected by the terminal devices always changes with time. For example, a travel enthusiast has different preferences for scenery at different times. For a period of time, he prefers island scenery, so he mainly collects scenery such as beaches and seas through his mobile phone. After a period of time, he prefers desert scenery, and the same smart device mainly collects pictures of the Gobi Desert. This phenomenon is called client data drift. At this time, if the next task (desert scenery recognition) is trained using a model based on the previous task (island scenery recognition), the performance of the model on the previous task will drop sharply, that is, catastrophic forgetting will occur.

[0003] In the context of the rapid replacement of artificial intelligence technology, people not only put forward higher requirements for the calculation accuracy of intelligent models, but also how to overcome the problem of catastrophic forgetting has become a hot topic. However, in the context of the rapid development of IoT terminals such as mobile phones and tablets, on the one hand, a single terminal device does not have enough data to support the continuous update of the model; on the other hand, transmitting a large amount of data to the server will cause the leakage of user privacy. The traditional centralized continuous learning method is no longer applicable. In the context of the rapid development of the Internet of Things (IoT) technology, how to enable multiple terminal devices to continuously learn collaboratively while ensuring that client data is not leaked has become an urgent problem to be solved. Summary of the invention

[0004] In view of the above problems, an embodiment of the present invention provides an adaptive federated continuous learning method and apparatus for resource-constrained clients, which overcomes the above problem of how to allow multiple terminal devices to continuously learn collaboratively while ensuring that client data is not leaked.

[0005] According to one aspect of an embodiment of the present invention, an adaptive federated continuous learning method for resource-constrained clients is provided, the method comprising: receiving a task robust encoder and a task sensitive decoder after decoding a global model sent by a server; iteratively training a model composed of the task robust encoder and the task sensitive decoder according to local current task data, and uploading the trained task robust encoder and the task sensitive decoder to the server, so as to aggregate them on the server to obtain a new global model and update a stored task robust encoder pool to complete the current federated iteration; during the federated iterative training process, if a local training has been completed based on the current task data, data drift detection is performed, and if data drift occurs, the task sensitive decoder is reinitialized, and the stored task sensitive decoder pool is updated.

[0006] Optionally, data drift detection is performed, including: applying a task robust encoder received from a server to extract a first data feature from the current task data; applying a task robust encoder obtained after a local training to extract a second data feature from the current task data; calculating the Manhattan distance between the first data feature and the second data feature; and determining whether data drift has occurred based on the Manhattan distance.

[0007] Optionally, if data drift occurs, the task-sensitive decoder is reinitialized and the stored task-sensitive decoder pool is updated, including: if data drift occurs, the task-robust encoder remains unchanged and the task-sensitive decoder is initialized; the task-sensitive decoder after local training in the previous round of federated iterations is updated and stored in the task-sensitive decoder pool.

[0008] Optional, global model received from the server Includes task-robust encoder and task-sensitive decoder:

[0009]

[0010] in is the task-sensitive decoder for client k at the i-th federated iteration of the current task t; is the task robust encoder of client k in the i-th federated iteration of the current task t, obtained by two-stage aggregation:

[0011]

[0012] E . t (i,0) is the globally aggregated task robust encoder obtained after i-1 iterations of the current task t, D . t(i,0) is the global aggregated task-sensitive decoder obtained after i-1 iterations of the current task t, E · t-1 (I t-1 ,N) is the previous task t-1 after I t-1 The task-robust encoder is globally aggregated after iterations and stored in the task-robust pool; is the task-sensitive decoder of the previous task t-1 after i-1 iterations of global aggregation, is the amount of data of client k in the current task t.

[0013] Based on the same inventive concept, an adaptive federated continuous learning method for resource-constrained clients is provided, including: decoding the global model to obtain a task robust encoder and a task sensitive decoder, and sending the task robust encoder and the task sensitive decoder to each client; receiving the locally trained task robust encoder and task sensitive decoder uploaded by each client; respectively aggregating the task robust encoder and task sensitive decoder received from each client to obtain the aggregated task robust encoder and task sensitive decoder; updating and storing the aggregated task robust encoder in a task robust encoder pool.

[0014] Optionally, the task robust encoder and task sensitive decoder received from each client are aggregated respectively to obtain an aggregated task robust encoder and task sensitive decoder, including: aggregating and averaging the task robust encoders received from each client to obtain an aggregated task robust encoder; aggregating and averaging the task sensitive decoders received from each client to obtain an aggregated task sensitive decoder.

[0015] Based on the same inventive concept, an adaptive federated continuous learning device for resource-constrained clients is provided, comprising: a first receiving unit, used to receive a task-robust encoder and a task-sensitive decoder after decoding a global model sent by a server; a local training unit, used to iteratively train a model composed of the task-robust encoder and the task-sensitive decoder according to local current task data, and upload the trained task-robust encoder and the task-sensitive decoder to the server, so as to aggregate them on the server to obtain a new global model and update the stored task-robust encoder pool to complete the current federated iteration; a drift processing unit, used to perform data drift detection during the federated iterative training process if a local training has been completed based on the current task data, and reinitialize the task-sensitive decoder if data drift occurs, and update the stored task-sensitive decoder pool.

[0016] Based on the same inventive concept, an adaptive federated continuous learning device for resource-constrained clients is provided, including: a model sending unit, used to decode the global model to obtain a task robust encoder and a task sensitive decoder, and send the task robust encoder and the task sensitive decoder to each client; a second receiving unit, used to receive the locally trained task robust encoder and task sensitive decoder uploaded by each client; a model aggregation unit, used to aggregate the task robust encoder and task sensitive decoder received from each client respectively to obtain the aggregated task robust encoder and task sensitive decoder; an update storage unit, used to update and store the aggregated task robust encoder in a task robust encoder pool.

[0017] Based on the same inventive concept, an embodiment of the present invention further proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the aforementioned method when executing the program.

[0018] Based on the same inventive concept, an embodiment of the present invention further proposes a computer storage medium, in which at least one executable instruction is stored, and the executable instruction enables a processor to execute the aforementioned method.

[0019] The beneficial effect is: the embodiment of the present invention receives the task robust encoder and the task sensitive decoder after decoding the global model sent by the server; it iteratively trains the model composed of the task robust encoder and the task sensitive decoder and monitors data changes according to the local current task data, and uploads the trained task robust encoder and the task sensitive decoder to the server to aggregate them on the server to obtain a new global model and update the stored task robust encoder pool, thereby realizing adaptive federated continuous learning of the global model, which is applied to the occasions of federated continuous learning under the premise of communication privacy protection, and also greatly improves the resource utilization efficiency of the continuous learning process.

[0020] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0022] Figure 1 A schematic diagram of an adaptive federated continuous learning framework for resource-constrained clients provided by an embodiment of the present invention is shown;

[0023] Figure 2 A schematic diagram of a process of an adaptive federated continuous learning method for a resource-constrained client provided by an embodiment of the present invention is shown;

[0024] Figure 3 A schematic diagram showing the results of a simple continuous learning experiment with 3 tasks on the FashionMNIST dataset according to an embodiment of the present invention;

[0025] Figure 4 A schematic diagram showing the results of a simple continuous learning experiment with 3 tasks on the Cifar10 dataset according to an embodiment of the present invention;

[0026] Figure 5 A schematic diagram showing the results of a simple continuous learning experiment with 3 tasks on the THUCNews dataset according to an embodiment of the present invention;

[0027] Figure 6 A schematic diagram showing the results of a simple continuous learning experiment with 5 tasks on the FashionMNIST dataset according to an embodiment of the present invention;

[0028] Figure 7 A schematic diagram showing the results of a simple continuous learning experiment with 5 tasks on the Cifar10 dataset according to an embodiment of the present invention;

[0029] Figure 8 A schematic diagram showing the results of a simple continuous learning experiment with 5 tasks on the THUCNews dataset according to an embodiment of the present invention;

[0030] Fig. 9 A schematic diagram showing the results of a continuous continuous learning experiment with 10 tasks on the Cifar100 dataset according to an embodiment of the present invention;

[0031] Fig.10 A schematic diagram showing the results of a continuous continuous learning experiment with 15 tasks on the Cifar100 dataset according to an embodiment of the present invention;

[0032] Fig.11 A schematic diagram showing the results of a continuous continuous learning experiment with 20 tasks on the Cifar100 dataset according to an embodiment of the present invention;

[0033] Fig.12 A schematic diagram showing the results of a continuous learning experiment on the Cifar10 dataset in a domain increment scenario according to an embodiment of the present invention;

[0034] Fig.13A schematic diagram showing a comparison of continuous learning experiments for data drift detection on the Cifar100 dataset according to an embodiment of the present invention is shown;

[0035] Fig.14 A schematic diagram of a process flow of an adaptive federated continuous learning method for resource-constrained clients provided by another embodiment of the present invention is shown;

[0036] Fig.15 A schematic diagram of the structure of an adaptive federated continuous learning device for resource-constrained clients provided by an embodiment of the present invention is shown;

[0037] Fig.16 A schematic diagram of the structure of an adaptive federated continuous learning device for resource-constrained clients provided by another embodiment of the present invention is shown;

[0038] Fig.17 A schematic diagram of an electronic device in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0039] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0040] In the process of continuous learning, as the data changes, the last few layers of the model (for example, the fully connected layer) always change significantly. In comparison, the number of model layers before this changes less. Taking the FashionMNIST dataset as an example, a LeNet neural network consisting of a convolutional layer, an activation layer, a maximum pooling layer, and a fully connected layer is constructed. There are two types of parameters in the convolutional layer and the fully connected layer, namely weights and biases. For the setting of continuous learning, in the ten categories with labels from 0 to 9, 10 categories of data are divided into 5 tasks, and two categories are grouped into one task, namely {0,1}, {2,3}, {4,5}, {6,7}, {8,9}. Each task is trained for 100 rounds, and the initialization model of the next task is the model trained for the previous task. By observing the changes in the model parameters of two adjacent tasks, the impact of task changes on the model can be found.

[0041] In an embodiment of the present application, the global model M on the device is split into a task robust encoder Encoder and a task sensitive decoder Decoder, that is, M = E + D, E represents the task robust encoder, and D represents the task sensitive decoder. The parameters of the task robust encoder show relative stability in different tasks. In general, the task robust encoder learns to share knowledge between tasks, thereby showing good generalization ability. A task robust encoder pool is constructed on the server to store the global task robust encoder model. The global task robust encoder can be updated in a two-stage fusion manner during each iteration without storing a separate task robust encoder for each client and historical task. The parameters of the task sensitive decoder show high variability to the dynamics of a specific task, resulting in reduced compatibility between tasks. In order to solve this problem, a task sensitive decoder pool is constructed on the client, that is, the terminal device, to store the task sensitive decoders of historical tasks. In the inference stage, the user can use the corresponding task sensitive decoder Decoder according to the task attributes, and combine it with the current task robust encoder Encoder to build a complete on-device global model for calculation. See Figure 1 Each client discovers Task 0, Task 1, and Task 2 through data drift detection and adaptively triggers continuous learning (CL). The task robust encoder Decoder and task sensitive decoder Encoder are found and transmitted to the server for aggregation (Fusion). The server maintains a task robust encoder pool for the client, while the task sensitive decoder pool is maintained locally.

[0042] The federated learning process can be divided into four stages: server distribution model, client local training, client upload, and server aggregation. That is, first, the server distributes the parameters of the task robust encoder and task sensitive decoder that constitute the global model. The client performs local training and data change monitoring based on the received model parameters and the local current task data, and uploads the trained task robust encoder and task sensitive decoder to the server. The server aggregates the task robust encoder and task sensitive decoder uploaded by each client, and obtains the aggregated task robust encoder and task sensitive decoder, thus completing a federated iteration. Repeating the above process can realize adaptive federated continuous learning of the global model, which can be applied to federated continuous learning under the premise of communication privacy protection, can reduce client resource consumption, and also greatly improve the resource utilization efficiency of the continuous learning process.

[0043] Figure 2 The flowchart of the adaptive federated continuous learning method for resource-constrained clients provided by an embodiment of the present invention is shown. Figure 2As shown, the adaptive federated continuous learning method for resource-constrained clients includes:

[0044] Step S11: receiving the task robust encoder and task sensitive decoder after decoding the global model sent by the server.

[0045] When the global model is federated, the server decodes the global model and sends the decoded task-robust encoder and task-sensitive decoder to each client. Each client receives the task-robust encoder and task-sensitive decoder sent by the server as the initial task-robust encoder and task-sensitive decoder for local training.

[0046] If it is the first federated iteration, the client receives the parameters of the task robust encoder and task sensitive decoder randomly initialized by the server when decoding the global model. If at least one federated iteration has been performed, the task robust encoder and task sensitive decoder obtained after the previous federated iteration sent by the server are received as the initial task robust encoder and task sensitive decoder for local training of each client in the current federated iteration.

[0047] Step S12: Locally train the global model composed of the task robust encoder and the task sensitive decoder according to the local current task data, and upload the trained task robust encoder and the task sensitive decoder to the server, so as to aggregate them on the server to obtain a new global model and update the stored task robust encoder pool to complete the current federated iteration.

[0048] On the client side, the local model is locally trained according to the local current task data using the received parameters of the task robust encoder and task sensitive decoder as initial parameters to obtain the trained task robust encoder and task sensitive decoder and upload them to the server so that the server can perform federated aggregation based on the task robust encoder and task sensitive decoder uploaded by each client to obtain the aggregated task robust encoder and task sensitive decoder, i.e., the new global model, update the task robust encoder parameters stored in the task robust encoder pool, and complete the current federated iteration.

[0049] Step S13: During the federated iterative training process, if local training has been completed once based on the current task data, data drift detection is performed. If data drift occurs, the task-sensitive decoder is reinitialized and the stored task-sensitive decoder pool is updated.

[0050] In any federated training process after a federated iteration, the client needs to determine whether data drift has occurred after completing each round of local training. If data drift has occurred, the task-sensitive decoder needs to be reinitialized and the stored task-sensitive decoder pool needs to be updated, that is, the task-sensitive decoder parameters obtained after the last federated aggregation are stored in the task-sensitive decoder pool.

[0051] Task-sensitive decoder pooling can mitigate the impact of catastrophic forgetting on terminal devices. Considering that the task-sensitive decoder contains the last few layers of the model, or even a single layer, it takes up less storage space than the method of retaining most of the historical model on the terminal device. This design provides a significant improvement in storage efficiency. In addition, dividing the global model into task-robust layers and task-sensitive layers ensures the independence of different tasks, thereby reducing interference between tasks and can be applied to the occasion of privacy protection of federated learning communication.

[0052] In the embodiment of the present application, it is assumed that there are K clients in the federated learning framework, each of which faces T continuous learning tasks, and the task sequence can be expressed as {0,…,…,}, and the number of federated iterations of task t is I t The client model is represented by M, then the client model set is {M1,M2,…,M K This application will mainly elaborate on the two processes of client local training and server aggregation. In the client local training phase, in each round of iteration, the number of client local training is N. When client k faces task t, the trained model is Assuming the client learning rate is η, the client local training process can be expressed as:

[0053]

[0054] in, is the local model obtained by client k in the nth local training in the i-th federated iteration of task t, For client k facing task t, the model is Before the client starts local training, the global model obtained from the server is It includes a task-robust encoder and a task-sensitive decoder, which are composed as follows:

[0055]

[0056] in, is the task-sensitive decoder for client k at the i-th federated iteration of the current task t; is the task robust encoder of client k in the i-th federated iteration of the current task t, obtained by two-stage aggregation:

[0057]

[0058] E · t (i,0) is the globally aggregated task robust encoder obtained after i-1 iterations of the current task t, D · t (i,0) is the global aggregated task-sensitive decoder obtained after i-1 iterations of the current task t, The weighted sum of · t-1 (I t-1 ,N) is the previous task t-1 after I t-1 The task-robust encoder is globally aggregated after iterations and stored in the task-robust pool; is the task-sensitive decoder of the previous task t-1 after i-1 iterations of global aggregation, is the amount of data of client k in the current task t.

[0059] It is obtained by two-stage aggregation. The first stage is the aggregation of the spatial dimension. As shown in formula (4), in formula (4), E · t (i,0) is the globally aggregated task robust encoder obtained after i-1 iterations of the current task t. The second stage is the aggregation of the time dimension, as shown in formula (3). In formula (3), E · t-1 (I t-1 ,N) is the task t-1 after I t-1 The task robust encoders that are globally aggregated after iterations are stored in the task robust pool. In the above process, when the client faces the first task and there is no previous task, the training process of the task robust encoder Encoder is similar to the traditional federated learning step. The training process of the task sensitive decoder is shown in formula (5). When the client encounters a new task, the task sensitive decoders are reinitialized and then updated in a conventional federated learning manner. After a specific task is trained, its corresponding lightweight task sensitive decoder is stored in the task pool. In the above continuous learning process, the shared knowledge contained in different tasks is inherited through the task robust encoder. Only one task robust encoder needs to be stored to inherit the common knowledge of historical tasks, and a branch with extremely small memory usage is saved for a specific task, which greatly saves storage resources and is also conducive to long-term continuous learning. In addition, the initialization of the task sensitive decoder refers to the random initialization of the model parameters of the task sensitive decoder.

[0060] It is worth noting that in the process of continuous learning, E · t (·,·) and E · t-1 The structure of (·,·) is always the same, but D · t (·,·) and D. t-1 The structure of (·,·) is not always consistent. For example, in a class increment task, when the model faces a new classification type, it will automatically expand the branch structure of the model to adapt to the new task. · t The structure of (·,·) will change significantly.

[0061] In an embodiment of the present application, during the federated iterative training process, data drift detection is also required to detect whether the data has changed. If the data changes, it means that the current training of the model may be inaccurate and relevant adjustments need to be made. The common data drift detection method is a data-based method, which determines whether the data has changed by measuring the feature distance between the previous and next data. This requires a considerable amount of memory to store historical data, which is very unfriendly to resource-constrained clients. At the same time, the present application is a model-based continuous learning method, which does not save historical data, and the client only stores the data of the current task. Therefore, inspired by contrastive learning, a model-based data drift detection method is proposed for the above method. By comparing the models before and after local learning on the client, data drift detection is performed. The specific approach is as follows: after the server distributes the aggregate model to the client, the client uses local data for local training. In order to measure the difference before and after the client uses the current task data to train the model, a certain amount of current task data is selected and input into the models before and after training. In order to detect data changes more accurately and subtly, the features extracted by the task robust encoders in the two models can be compared. If the change value exceeds a certain threshold, it is considered that the features extracted by different models for the same data are very different, then the model has changed significantly and data migration has occurred on the client.

[0062] There are two details worth noting in the above process. The first is that the difference in the output features of the task robust encoder is used to measure the difference in the model. This is because the task robust encoder is less sensitive to data changes than the task sensitive decoder. When the output of the task robust encoder changes significantly, it is considered that the data has migrated, which increases the accuracy of data drift detection. Second, it was found through experiments that compared with the Euclidean distance, the Manhattan distance is more sensitive to model changes and more stable. Therefore, the Manhattan distance is chosen to measure the difference between the features extracted by the task robust encoder. The calculation formula is as follows:

[0063]

[0064] in, represents the task-robust encoder trained once locally in the i-th federated iteration when the k-th client faces task t. is the data of the client k's t-th task, then Indicates that Input Task Robust Encoder Extracted data features. is the robust encoder of the task received by client k at the beginning of the i-th iteration when facing task t. Similarly, for Input Task Robust Encoder Extracted data features. Through the above data detection mechanism, the terminal device can automatically detect data changes and trigger continuous learning, which greatly enhances the client's adaptive ability.

[0065] Optionally, a task robust encoder received from a server is applied to extract a first data feature from the current task data; a task robust encoder obtained after a local training is applied to extract a second data feature from the current task data; the Manhattan distance between the first data feature and the second data feature is calculated; and whether data drift occurs is determined based on the Manhattan distance. If data drift occurs, the task robust encoder remains unchanged, the task sensitive decoder is initialized, and the task sensitive decoder after local training in the last round of federated iterations is updated and stored in the task sensitive decoder pool.

[0066] In order to more clearly illustrate the method proposed in this application, it is determined that the input includes the number of clients K, the total number of federated learning times I, the global model M, which can be divided into a task-robust encoder and a task-sensitive decoder, the number of local training rounds N, and the data DA of each client k. k , and the learning rate η. Finally, the global task robust encoder and each task specific task sensitive decoder are output. The specific algorithm process is as follows:

[0067]

[0068]

[0069] In Algorithm 1, first, the server initializes the task ID and M. t(Lines 1-2), and then decouple the model into a task-robust encoder and a task-sensitive decoder (Line 3) according to the changes in the model layers caused by the task changes. The task-robust encoder has low sensitivity to task changes, while the task-sensitive decoder has high sensitivity. Next, the initialized task-robust encoder and task-sensitive decoder are distributed to the clients (Line 5). After receiving the model, each client performs N rounds of local training. When the number of federated iterations is greater than 1, each client will check whether there is data drift after one round of local training (Line 11). Specific data drift monitoring is described in Algorithm 2. If a task change is detected, remain unchanged, is reinitialized and the task pool is updated (lines 13-14). Then, the entire model is trained to obtain and After N rounds of local training, clients upload their task-robust encoders and task-sensitive decoders to the server. Finally, the server aggregates all task-robust encoders and task-sensitive decoders to obtain E. t (+1,0) and D. t (+1,0) (Lines 19-20), and update the task-robust encoder pool (Line 21).

[0070] Algorithm 2 for data drift detection is as follows:

[0071]

[0072] In Algorithm 2, the model received at the beginning of round i is And the model obtained after a local training The comparison between the two determines whether the data has drifted. Only when the difference exceeds the threshold α, it is considered that the task has changed.

[0073] The following is an experimental description of the adaptive federated continuous learning method for resource-constrained clients of an embodiment of the present invention. The adaptive federated continuous learning method for resource-constrained clients of an embodiment of the present invention is referred to as SacFL. A federated learning framework including five clients and one server is designed. The framework is a cross-silo scenario in federated learning, that is, each client participates in each round of iteration. The experiment involves the image datasets FashionMNIST (Fa-MNIST), Cifar10 and Cifar100, with 10, 10 and 100 categories respectively; the text dataset is THUCNews, which contains 14 categories of news data collected from Sina News, from which 5,000 pieces of data from 10 categories including lottery, stocks, education, home, technology, fashion, sports, games, social and entertainment are taken out, of which 4,000 are used as training data and 1,000 are used as test data.

[0074] For the class increment problem, according to the total number of tasks T, the data categories are randomly divided into T parts. For example, if the total number of tasks is 3, then the number of categories of each task data is the total number of categories divided by the number of tasks, rounded down, and the number of categories of the last task is the number of remaining categories. For example, the first task label of client 1 is [1,4,5], the label of task 2 is [2,6,8], then task 3 is [0,3,7,9]. It should be noted that in real life, the data categories of different clients at the same time may be different. For the same category of data, some clients may have it at a certain time, while others may not. Therefore, in order to be closer to the real situation, the specific types of data included in each task of the client are randomly sampled. At the same time, the same category of data is randomly distributed equally among the clients to ensure that the same category of data between the clients is not repeated. In this way, the data distribution between clients covers two possibilities: independent and identically distributed and non-independent and identically distributed. In addition, in order to solve the domain increment problem, you can choose to add Gaussian noise and multiplicative noise to the image data to construct a domain increment dataset.

[0075] The experimental comparison framework mainly includes two categories, namely, methods based on continuous learning and traditional non-continuous learning methods. Among them, the methods based on continuous learning include (1) CFeD: This method uses the method of distilling the old model for federated continuous learning; (2) LwF-Fed: This method is a variant of the centralized continuous learning method LwF in the federated learning framework. This method also adjusts the loss function of the new task by distilling the old model; (3) EWC-Fed: This method is a variant of the centralized continuous learning method EWC in the federated learning framework. This method reduces the fluctuation of important parameters of the old task by introducing regularization terms, thereby maintaining memory ability; (4) MultiHead-Fed: This method is an improvement of the MultiHead method in the context of federated learning. By setting a separate classification for each type of task, knowledge memory is achieved. Traditional federated learning methods mainly include two classic algorithms in the field of federated learning: FedAvg and Fedprox. The training hyperparameters of each data set are shown in Table 1.

[0076] Table 1 Training hyperparameters for each dataset

[0077]

[0078] In the real world, when data changes, there are often no obvious signs. It is necessary to design a corresponding data drift monitoring mechanism to detect data changes that trigger a continuous learning mechanism. This application sets the threshold for FashionMNIST changes to 20,000, the threshold for Cifar10 to 600, and the threshold for THUCNews to 15,000. By setting appropriate thresholds, SacFL can accurately identify the offset of the data set, thereby providing support for subsequent continuous learning. After the local client is trained locally once, when the change value of the feature extracted by the task robust encoder exceeds the above threshold, it is considered that data drift has occurred on the client, that is, the task has changed.

[0079] First, based on the FashionMNIST, Cifar10 and THUCNews datasets, the above data drift detection is applied to all methods. Experimental verification is performed for a simple class-incremental continuous learning scenario. Specifically, in a simple continuous learning scenario, the data is relatively stable, with only a few drifts. Assuming that the data has 3 and 5 data offsets, corresponding to 3 tasks and 5 tasks. In the experimental setting, the number of categories contained in each task is as balanced as possible. Since the data of the above three datasets all contain 10 categories, when the number of tasks is 3, the number of categories contained in the data of the three tasks are 3, 3 and 4 respectively. When the number of tasks is 5, 10 is exactly divisible by 5, so each task contains 2 classes. The specific experimental results are shown in Table 2. Figure 3-Figure 8 As shown. Among them, Figure 3 This is the result of a simple continuous learning experiment with 3 tasks on the FashionMNIST dataset. Figure 4 This is the result of a simple continuous learning experiment on the Cifar10 dataset with 3 tasks. Figure 5 This is the result of a simple continuous learning experiment with 3 tasks on the THUCNews dataset. Figure 6 This is the result of a simple continuous learning experiment with 5 tasks on the FashionMNIST dataset. Figure 7 This is the result of a simple continuous learning experiment on the Cifar10 dataset with 5 tasks. Figure 8 The results of a simple continuous learning experiment with 5 tasks on the THUCNews dataset.

[0080] Table 2 Experimental results

[0081]

[0082] It can be seen from the experimental results that when the total number of tasks is 3, in the FashionMNIST dataset, when the task is 0, SacFL does not have a clear advantage. However, as the tasks change, when the data evolves to task 1 and task 2, SacFL shows a clear advantage. When the dataset is Cifar10, overall, compared with other methods, the test accuracy of SacFL is still at an advantageous level. When in the THUCNews dataset, SacFL is comparable to other methods and does not show a clear disadvantage. Furthermore, when the total number of tasks is 5, whether on the FashionMNIST dataset or the Cifar10 dataset, SacFL is close to other methods on task 0, but in subsequent task1 to task4, the accuracy is significantly higher than other methods. When other methods can only achieve an accuracy of 20%-30% in the Cifar10 dataset, SacFL can achieve an accuracy of 50%-60%. Compared with the case of 3 tasks, when the number of tasks is 5, the advantage of the method SacFL of the present application is more obvious. This shows that as the continuous learning process evolves and the number of tasks increases, the advantage of SacFL gradually increases.

[0083] This is because, by monitoring the changes of the model layer with the task, lightweight decoders that are sensitive to the task are discovered, and the historical information carried by these decoders is directly saved by the back-end server. On the one hand, the integrity of the memory of historical task information can be guaranteed. On the other hand, this lightweight decoder greatly reduces the requirements for storage resources compared to the complete preservation of historical models or data. In addition, in subsequent tasks, although SacFL still maintains a clear advantage, it may also decline to a certain extent. This is because it is tested on all historical data of the client. When encountering new tasks, the test data set changes. At the same time, subsequent task training will also bring minor changes to the encoder, which reduces the coupling between the encoder and the decoder of the previous task.

[0084] The previous part only shows the situation where the data changes slightly, that is, simple continuous learning. In real scenarios, continuous learning is a long-term learning task. The changes of 3 or 5 tasks are far from enough. SacFL needs to be verified under more task changes. Therefore, this application introduces the Cifar100 dataset, and uses more data classes in the dataset to build more tasks to verify the effectiveness of SacFL under the situation of large-scale task changes. Specifically, the method effects of 10, 15 and 20 tasks were tested respectively. Only some of the experimental results are shown in the figure. The experimental results are as follows. Figure 9-11 shown.

[0085] exist Figure 9-11 When the number of tasks is 10, task1 ( Fig. 9 a), task3( Fig. 9 b), task5( Fig. 9 c), task7( Fig. 9 d), task9( Fig. 9 e) results; when the number of tasks is 15, task2 ( Fig.10 a), task5( Fig.10 b), task8( Fig.10 c), task11( Fig.10 d), task14( Fig.10 e) results; when the number of tasks is 20, task3 ( Fig.11 a), task7( Fig.11 b), task11( Fig.11 c), task15( Fig.11 d), task19( Fig.11e) Results. It can be seen that no matter when the total number of tasks is 10, 15, or 20, other methods are almost ineffective under continuous tasks, and only the SacFL method has obvious advantages and stable convergence. This further illustrates that the method of the present application performs better in multiple continuous tasks.

[0086] In addition to class increment, domain increment is also an important setting in continuous learning. In the domain increment scenario, the label of the data remains unchanged, but the data itself changes. Therefore, this application constructs a domain increment dataset by adding Gaussian noise and multiplicative noise to the Cifar10 dataset to form three tasks. Task0 is the original dataset, task1 is Gaussian noise, and task2 is multiplicative noise. The experimental results are as follows: Fig.12 As shown, Fig.12 a. Fig.12 b and Fig.12 c represents the experimental results of task0, task1, and task2 respectively.

[0087] Fig.12 In the data set, under the original data (task0), the convergence results and convergence speed of using SacFL and other methods are consistent, and can achieve an accuracy of about 80%. However, when trying to add Gaussian noise to the Cifar10 dataset, all methods showed obvious fluctuations, and the convergence values ​​of other methods except SacFL dropped significantly, among which the FedProx method that did not adopt the continuous learning mechanism dropped the most significantly. When going a step further and migrating the model to the multiplicative noise dataset, the accuracy of SacFL is still high. Therefore, based on the previous experimental results, it can be found that the SacFL method has good performance in the continuous learning scenarios of class increment and domain increment.

[0088] In terms of adapting to the limited resources of the terminal, SacFL has obvious advantages in both computing and storage resources compared to other methods, as shown in Table 3. Especially in terms of storage resources, the traditional model-based federated continuous learning (FCL) method usually needs to store the complete model to retain historical knowledge, while SacFL only maintains a lightweight decoder, thereby reducing storage overhead. Taking the ResNet-18 model of Cifar10 as an example, the traditional method consumes 43.73MB, while the lightweight Decoder only occupies 0.19MB, which is a reduction of 99.9%; it is also reduced by 97.7% in the LeNet model and 99.9% in TextCNN. In terms of computing resources, the table shows the average time consumed by each federated iteration process when the total number of tasks is 3. It can be seen that the computing time of SacFL in the LeNet, Resnet18 and TextCNN models is 13.01%, 5.98% and 20.16% lower than the average of other methods. Therefore, the SacFL method consumes fewer resources overall and is more suitable for terminals with limited resources. It should be noted that the resource consumption of the multihead method is not listed in the table because the structure of this method varies greatly in each task and is not comparable to other methods.

[0089] Table 3 Computational and storage resource overhead of different continuous learning methods

[0090]

[0091] This application also performs ablation verification on the data drift detection component. The verification is only performed in the incremental scenario with a large number of tasks. The results are as follows Fig.13 As shown, taking the Cifar100 dataset as an example, Fig.13 a, 13b and Fig.13 c represents the experimental results of Task 5, Task 10, and Task 15, Accuracy represents accuracy, and Epoch represents the number of federated iterations. It can be seen that without the data drift detection component, the performance of the model will decrease as the task changes. However, with the addition of the data drift detection mechanism, the performance of the model only decreases temporarily after the task changes, but will recover in subsequent training.

[0092] This application aims to solve the problem of continuous learning of multi-device collaboration in distributed scenarios, and adopts federated learning technology to perform federated learning with adaptive continuous learning capabilities. By calculating the changing values ​​of model parameters with tasks, it is found that the last few layers of the model are highly sensitive to task changes. Based on this, the model is split into a task-general model encoder and a task-private lightweight model decoder. By directly utilizing the storage resources of the back-end server, a task-oriented lightweight model pool is built to store lightweight task-sensitive decoders, which can fully preserve historical knowledge. In addition, a data drift detection mechanism based on contrastive learning is proposed. When the features extracted by the encoder that is insensitive to task changes change by more than a threshold, it is determined that the task has changed. Based on FashionMNIST, Cifar10, Cifar100, THUCNews and multiple text and image datasets, this application has verified through experiments that: (1) When the client's task changes a few times, SacFL of this application has more advantages than the mainstream continuous learning method; (2) When the client's task changes multiple times in succession, the multiple task changes not only do not weaken the effect of SacFL, but the advantage of SacFL becomes more obvious; (3) In addition to class increments, SacFL is still effective when the client data undergoes domain increment changes; (4) SacFL greatly reduces the storage and computing overhead of the client, i.e., the terminal device, especially in terms of storage resources, with a reduction rate of up to 99.9%.

[0093] In summary, in the embodiment of the present invention, the server sends the initial task robust encoder and the initial task sensitive decoder obtained by decoupling the global model to each client; each client performs multiple rounds of local training based on the current task data, obtains the trained task robust encoder and task sensitive decoder, and uploads them to the server; the server aggregates the task robust encoder and task sensitive decoder uploaded by each client, respectively, to obtain the task robust encoder and task sensitive decoder of the current federated learning; at the same time, each client autonomously monitors the data changes during the local training process. When the task data changes, the client stores and updates the local task sensitive decoder pool, and the server updates the task robust encoder pool, so as to realize adaptive federated continuous learning of the global model, which can be applied to the occasions of federated continuous learning under the premise of communication privacy protection, greatly reducing the storage and computing overhead of the client, and also greatly improving the resource utilization efficiency of continuous learning.

[0094] The above specific embodiments of the present invention are described. In some cases, the actions or steps recorded in the embodiments of the present invention can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0095] Based on the same concept, the embodiment of the present invention also provides an adaptive federated continuous learning method for resource-constrained clients. Fig.14 As shown, the adaptive federated continuous learning method for resource-constrained clients includes:

[0096] Step S21: Decode the global model to obtain a task-robust encoder and a task-sensitive decoder, and send the task-robust encoder and the task-sensitive decoder to each client.

[0097] The server decodes the global model to obtain a task-robust encoder and a task-sensitive decoder. The decoding process also randomly initializes the task-robust encoder and task-sensitive decoder that constitute the global model, and then sends the initialized task-robust encoder and task-sensitive decoder to each client.

[0098] Step S22: receiving the locally trained task-robust encoder and task-sensitive decoder uploaded by each client.

[0099] After receiving the task robust encoder and task sensitive decoder sent by the server, each client performs local training based on the initialized task robust encoder and task sensitive decoder using local data, and uploads the task robust encoder and task sensitive decoder obtained after local training to the server. The server receives the task robust encoder and task sensitive decoder uploaded by each client.

[0100] Step S23: Aggregate the task-robust encoders and task-sensitive decoders received from each client to obtain aggregated task-robust encoders and task-sensitive decoders.

[0101] The server aggregates the task robust encoders and task sensitive decoders received from each client separately. Optionally, the task robust encoders received from each client are aggregated and averaged to obtain an aggregated task robust encoder; the task sensitive decoders received from each client are aggregated and averaged to obtain an aggregated task sensitive decoder; the aggregated task robust encoders and task sensitive decoders are combined to obtain the global model after the current federation iteration.

[0102] Step S24: Update and store the aggregated task robust encoder in the task robust encoder pool.

[0103] The server updates the task robust encoder obtained after aggregation and stores it in the task robust encoder pool, replacing the task robust encoder previously stored in the task robust encoder pool.

[0104] This application stores the task-robust encoder on the server, while the client only needs to store the task-sensitive decoder, which greatly reduces the client's storage and computing overhead, and also greatly improves the resource utilization efficiency of the continuous learning process.

[0105] Based on the same concept, an embodiment of the present invention also provides an adaptive federated continuous learning device for resource-constrained clients. Fig.15 As shown, the adaptive federated continuous learning device for resource-constrained clients includes: a first receiving unit, a local training unit, and a drift processing unit.

[0106] A first receiving unit is used to receive a task robust encoder and a task sensitive decoder after decoding the global model sent by the server;

[0107] A local training unit, used for iteratively training the model composed of the task robust encoder and the task sensitive decoder according to the local current task data, and uploading the trained task robust encoder and the task sensitive decoder to the server, so as to aggregate them on the server to obtain a new global model and update the stored task robust encoder pool to complete the current federation iteration;

[0108] The drift processing unit is used to perform data drift detection during the federated iterative training process if a local training has been completed based on the current task data. If data drift occurs, the task-sensitive decoder is reinitialized and the stored task-sensitive decoder pool is updated.

[0109] For the convenience of description, the above device is described as various modules according to their functions. Of course, when implementing the embodiment of the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0110] The apparatus of the above embodiment is applied to the corresponding method applied to the client in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0111] Based on the same concept, the embodiment of the present invention also provides an adaptive federated continuous learning device for resource-constrained clients. Applied to the server. Fig.16As shown, the adaptive federated continuous learning device for resource-constrained clients includes: a model sending unit, a second receiving unit, a model aggregation unit, and an update storage unit.

[0112] A model delivery unit, used for decoding the global model to obtain a task robust encoder and a task sensitive decoder, and delivering the task robust encoder and the task sensitive decoder to each client;

[0113] The second receiving unit is used to receive the locally trained task robust encoder and task sensitive decoder uploaded by each client;

[0114] A model aggregation unit, used to aggregate the task robust encoders and task sensitive decoders received from each client to obtain an aggregated task robust encoder and task sensitive decoder;

[0115] An update storage unit is used to store the aggregated task-robust encoder updates in the task-robust encoder pool.

[0116] For the convenience of description, the above device is described as various modules according to their functions. Of course, when implementing the embodiment of the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0117] The device of the above embodiment is applied to the corresponding method applied to the server in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0118] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of the above embodiments is implemented.

[0119] An embodiment of the present invention provides a non-volatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the method described in any of the above embodiments.

[0120] Fig.17 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1701, a memory 1702, an input / output interface 1703, a communication interface 1704, and a bus 1705. The processor 1701, the memory 1702, the input / output interface 1703, and the communication interface 1704 are connected to each other in communication within the device through the bus 1705.

[0121] The processor 1701 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided by the method embodiment of the present invention.

[0122] The memory 1702 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1702 may store an operating system and other application programs. When the technical solution provided by the method embodiment of the present invention is implemented by software or firmware, the relevant program code is stored in the memory 1702 and is called and executed by the processor 1701.

[0123] The input / output interface 1703 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0124] The communication interface 1704 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0125] The bus 1705 includes a path that transmits information between the various components of the device (eg, the processor 1701 , the memory 1702 , the input / output interface 1703 , and the communication interface 1704 ).

[0126] It should be noted that, although the above device only shows the processor 1701, the memory 1702, the input / output interface 1703, the communication interface 1704 and the bus 1705, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present invention, and does not necessarily include all the components shown in the figure.

[0127] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0128] This application is intended to cover all such substitutions, modifications and variations that fall within the broad scope of all embodiments. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the scope of protection of this disclosure.

Claims

1. An adaptive federated continuous learning method for resource-constrained clients, characterized in that: The adaptive federated continuous learning method comprises: Receive the task-robust encoder and task-sensitive decoder after decoding the global model sent by the server; According to the local current task data, the global model composed of the task robust encoder and the task sensitive decoder is locally trained, and the trained task robust encoder and the task sensitive decoder are uploaded to the server to aggregate them on the server to obtain a new global model and update the stored task robust encoder pool to complete the current federation iteration; During the federated iterative training process, if local training has been completed based on the current task data, data drift detection is performed. If data drift occurs, the task-sensitive decoder is reinitialized and the stored task-sensitive decoder pool is updated.

2. The method according to claim 1, characterized in that The data drift detection comprises: extracting a first data feature from the current task data using a task robust encoder received from a server; Extracting a second data feature from the current task data using a task robust encoder obtained after a local training; Calculating the Manhattan distance between the first data feature and the second data feature; Whether data drift occurs is determined according to the Manhattan distance.

3. The method according to claim 1, characterized in that If data drift occurs, the task-sensitive decoder is reinitialized and the stored task-sensitive decoder pool is updated, including: If data drift occurs, the task-robust encoder remains unchanged and the task-sensitive decoder is initialized; Store the task-sensitive decoder updates after local training in the previous federated iteration in the task-sensitive decoder pool.

4. The method according to claim 1, characterized in that: Global model received from the server Includes task-robust encoder and task-sensitive decoder: in, is the task-sensitive decoder for client k at the i-th federated iteration of the current task t; is the task robust encoder of client k in the i-th federated iteration of the current task t, obtained by two-stage aggregation: E. t (i,0) is the globally aggregated task robust encoder obtained after i-1 iterations of the current task t, D. t (i,0) is the global aggregated task-sensitive decoder obtained after i-1 iterations of the current task t, E. t-1 (I t-1 ,N) is the previous task t-1 after I t-1 The task-robust encoder is globally aggregated after iterations and stored in the task-robust pool; is the task-sensitive decoder of client k after global aggregation of i-1 iterations of the previous task t-1, is the amount of data of client k in the current task t.

5. An adaptive federated continuous learning device for resource-constrained clients, characterized in that: The device comprises: A first receiving unit is used to receive a task robust encoder and a task sensitive decoder after decoding the global model sent by the server; A local training unit, used for iteratively training the model composed of the task robust encoder and the task sensitive decoder according to the local current task data, and uploading the trained task robust encoder and the task sensitive decoder to the server, so as to aggregate them on the server to obtain a new global model and update the stored task robust encoder pool to complete the current federation iteration; The drift processing unit is used to perform data drift detection during the federated iterative training process if a local training has been completed based on the current task data. If data drift occurs, the task-sensitive decoder is reinitialized and the stored task-sensitive decoder pool is updated.

6. An adaptive federated continuous learning method for resource-constrained clients, characterized in that: The adaptive federated continuous learning method comprises: Decoding the global model to obtain a task-robust encoder and a task-sensitive decoder, and sending the task-robust encoder and the task-sensitive decoder to each client; Receive the locally trained task-robust encoder and task-sensitive decoder uploaded by each client; Aggregating the task-robust encoders and task-sensitive decoders received from each client to obtain aggregated task-robust encoders and task-sensitive decoders; Store the aggregated task-robust encoder updates in a task-robust encoder pool.

7. The method according to claim 6, characterized in that The step of respectively aggregating the task robust encoders and task sensitive decoders received from each client to obtain the aggregated task robust encoders and task sensitive decoders includes: Aggregate and average the task robust encoders received from each client to obtain an aggregated task robust encoder; The task-sensitive decoders received from each client are aggregated and averaged to obtain an aggregated task-sensitive decoder.

8. An adaptive federated continuous learning device for resource-constrained clients, characterized in that: The device comprises: A model delivery unit, used for decoding the global model to obtain a task robust encoder and a task sensitive decoder, and delivering the task robust encoder and the task sensitive decoder to each client; The second receiving unit is used to receive the locally trained task robust encoder and task sensitive decoder uploaded by each client; A model aggregation unit, used to aggregate the task robust encoders and task sensitive decoders received from each client to obtain an aggregated task robust encoder and task sensitive decoder; An update storage unit is used to store the aggregated task-robust encoder updates in the task-robust encoder pool.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 4 or 6 to 7 is implemented.

10. A computer storage medium, characterized in that: The storage medium stores at least one executable instruction, and the executable instruction enables the processor to execute the method as described in any one of claims 1-4 or 6-7.