Asynchronous federated learning method and system based on multi-agent model

Abstract

The present invention relates to the technical field of asynchronous federated learning, and provides an asynchronous federated learning method and system based on a multi-agent model, including: randomly selecting a number of pre-trained clients in each group of clients, and obtaining a decision result of whether each pre-trained client participates in the training and uploading of the model; receiving a local model obtained by training the clients participating in the training and uploading of the model in each group of clients, and obtaining a group model; performing weighted aggregation on the group model to obtain a global model. It can not only solve the long waiting delay problem in synchronous federated learning, but also solve the communication bottleneck problem in completely semi-asynchronous federated learning.

Description

Technical Field

[0001] The present invention belongs to the technical field of asynchronous federated learning, and in particular, relates to an asynchronous federated learning method and system based on a multi-agent model. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Federated learning algorithms involve hundreds of millions of remote devices training locally on the data generated by their devices and collectively training a global, shared model under the coordination of a central server acting as an aggregator.

[0004] The introduction of the Federated Average Algorithm is the beginning of federated learning. It breaks the inherent mode of traditional centralized and distributed training and protects data privacy by only passing model gradient parameters. Experiments have shown that this method can achieve flexible and efficient communication and reduce communication costs. However, due to the highly ideal environment requirements when the Federated Average Algorithm is running, there are also many problems in real heterogeneous client scenarios. The most typical problem is that in each round of training, the faster training client needs to wait for the slowest client to complete training before uploading the aggregated update global model. The training efficiency of the entire federated learning is determined by the slowest client, which will greatly reduce the training efficiency of the model and prolong the training time.

[0005] For synchronous federated learning such as federated averaging, experiments are conducted in a highly ideal scenario. However, in real scenarios, due to the heterogeneity of devices and the unreliability of the network, some stragglers (lagging devices or devices that exit training) are inevitable. Therefore, in actual scenarios, an asynchronous federated training method is more often used, in which the server does not need to wait for lagging devices to aggregate.

[0006] There is currently an exponentially weighted average asynchronous federated learning algorithm, in which different clients use their own local data sets to train their own local models. Once a client completes the training, it immediately sends the model parameters to the central server, and then the central server immediately aggregates the model parameters without waiting for any other edge devices. Its core idea is that the later the local model is uploaded, the lower the weight it is given. This method can adaptively balance the convergence speed and variance reduction. However, it still cannot solve the inherent problem in fully asynchronous federated learning, that is, the communication bottleneck caused by the frequent communication between the local client and the central server.

[0007] There are also methods to propose a semi-asynchronous way to solve the training waiting delay problem in synchronous federated learning and the communication bottleneck problem in fully asynchronous federated learning. However, facing the more complex heterogeneous client scenarios in real life, there is no good way to minimize communication overhead and training delay without losing model accuracy. Summary of the invention

[0008] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides an asynchronous federated learning method and system based on a multi-agent model, which can not only solve the long waiting delay problem in synchronous federated learning, but also solve the communication bottleneck problem in fully semi-asynchronous federated learning.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] A first aspect of the present invention provides an asynchronous federated learning method based on a multi-agent model, which comprises:

[0011] Randomly select a number of pre-trained clients from each group of clients, and obtain the decision result of whether each pre-trained client participates in the training and uploading of the model;

[0012] Receive the local models obtained by training the participating models and the uploaded clients in each group of clients to obtain a group model;

[0013] The group models are weighted aggregated to obtain the global model.

[0014] Furthermore, the method for obtaining the decision result is:

[0015] Get the status of each pre-trained client;

[0016] The state of each pre-trained client is input into the reinforcement learning agent network to obtain the decision result of whether each pre-trained client participates in the training and uploading of the model.

[0017] Furthermore, the status of the pre-trained client includes: training round index t, the amount of data on the client, the number of times the client has participated in local model updates and uploads up to the tth round, the number of updates of the group model to which the client belongs up to the tth round, the communication overhead of all pre-trained clients, and the training delay of all pre-trained clients.

[0018] Furthermore, the reinforcement learning agent network aims to maximize the cumulative reward.

[0019] Furthermore, when weighted aggregation is performed on the group models, the weight of each group model is related to the number of updates of the group model of each group.

[0020] A second aspect of the present invention provides an asynchronous federated learning system based on a multi-agent model, comprising:

[0021] A client intelligent selection module is configured to: randomly select a number of pre-trained clients from each group of clients, and obtain a decision result of whether each pre-trained client participates in the training and uploading of the model;

[0022] The intra-group synchronous training module is configured to: receive the local model obtained by training the participating models and the uploaded client training in each group of clients, and obtain a group model;

[0023] The inter-group asynchronous training module is configured to perform weighted aggregation on the group models to obtain a global model.

[0024] Furthermore, the client intelligent selection module is specifically configured as follows:

[0025] Get the status of each pre-trained client;

[0026] The state of each pre-trained client is input into the reinforcement learning agent network to obtain the decision result of whether each pre-trained client participates in the training and uploading of the model.

[0027] Furthermore, when weighted aggregation is performed on the group models, the weight of each group model is related to the number of updates of the group model of each group.

[0028] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the asynchronous federated learning method based on a multi-agent model as described above.

[0029] The fourth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in an asynchronous federated learning method based on a multi-agent model as described above are implemented.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention provides an asynchronous federated learning method based on a multi-agent model, which introduces multi-agent reinforcement learning to execute an efficient client intelligent selection mode to replace the random selection strategy in previous federated learning methods. It not only optimizes the model accuracy, but also greatly improves the model training efficiency. Compared with other more advanced semi-asynchronous federated learning methods, it spends less communication overhead and training delay when training to achieve a specified accuracy, and can be applied to a wide range of heterogeneous client machine learning model training scenarios in real life.

[0032] The present invention provides an asynchronous federated learning method based on a multi-agent model, in which each client has a reinforcement learning agent, which will decide whether to participate in the training and upload aggregation of the current round of model according to its own observations. It can solve the machine learning model training problem in more complex heterogeneous client scenarios, and can not only solve the long waiting delay problem in synchronous federated learning, but also solve the communication bottleneck problem in fully semi-asynchronous federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 This is an overall flow chart of an asynchronous federated learning method based on a multi-agent model according to the first embodiment of the present invention;

[0035] Figure 2 This is a graph of accuracy changes in complex scenarios on the F-MNIST dataset of Example 1 of the present invention;

[0036] FIG3( a ) is a diagram of communication overhead in a complex scenario on the F-MNIST dataset according to the first embodiment of the present invention;

[0037] FIG3( b ) is a diagram of training delay in a complex scenario on the F-MNIST dataset according to the first embodiment of the present invention;

[0038] Figure 4 This is a graph of accuracy changes in complex scenarios on the CIFAR-10 dataset according to the first embodiment of the present invention;

[0039] FIG5( a ) is a diagram of communication overhead in a complex scenario on the CIFAR-10 dataset according to the first embodiment of the present invention;

[0040] FIG5( b ) is a diagram of training delay in a complex scenario on the CIFAR-10 dataset according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] Terminology explanation:

[0045] The objective function of the federated learning algorithm is:

[0046]

[0047] in, is the local experience loss of client k, l i (x i ,y i ; w) is the data sample {x i ,y i}, w is the machine learning model to be trained; K is the total number of clients, D k (k∈{1,...,K}) represents the data samples stored on the local client k, n k =|D k | represents the number of data samples on client k; is the total number of data samples stored on K clients; assuming that for any k≠k',

[0048] The ultimate goal of the federated learning algorithm is to find a model w * To minimize the objective function:

[0049] w * = arg min f(w)

[0050] Federated Averaging Algorithm: A common method for solving the optimization problem defined in the ultimate goal of a federated learning algorithm in a synchronous update manner in a non-convex setting. This method trains the model by randomly sampling a subset of clients with a certain probability in each round, and each local client uses an optimizer (such as stochastic gradient descent) to perform several local iterations using its own data.

[0051] How to update the global model in the asynchronous federated learning algorithm with exponential weighted average:

[0052] α t ←α×s(t-τ)

[0053] w t ←(1-αt ) t-1 +α t w new

[0054] Among them, τ is the round index when the fastest client uploads and updates the global model, t is the current round index, α∈(0,1) is the mixing coefficient, and α t is the coefficient after dynamic update in the current round t, w t-1 is the old model obtained from the previous round of training, w new is the new model obtained by current training, w t is the new model obtained by weighted update for the next training, s(·) is the model staleness function, which can take values

[0055] Embodiment 1

[0056] This embodiment provides an asynchronous federated learning method based on a multi-agent model. Figure 1 As shown, the specific steps include:

[0057] Step 1: Client intelligent selection stage. In each round (tth round) of training, in each group of clients m A number of (|P|) pre-trained clients are randomly selected, the state of each pre-trained client is obtained, the state of each pre-trained client is input into the reinforcement learning agent network, and the decision result of whether each pre-trained client participates in the training and uploading of the model is obtained.

[0058] Assume that there is a federated learning task that divides all clients into M groups, and for one of the groups m Formal description is given, and other groups are similar. m In each training round t, |P| pre-trained clients are randomly selected, and then |P| clients decide whether to participate in the training and uploading of this round of models based on their own reinforcement learning agents. After the decision is made, |P'| clients will perform local synchronous federated training and model upload and update. The communication overhead of each communication between client n and the central server (uploading or downloading the model) is a fixed value B n , the response time CP of client n n It is expressed by the number of global rounds that the client undergoes in a local training. n The larger the value, the longer it takes for the client to perform local training, the slower the response, and the training delay of client n. The update of the group model needs to wait for the slowest client in the group to complete local training.

[0059] Step 101: The clients selected for pre-training obtain their respective current states:

[0060]

[0061] The state space of each reinforcement learning agent n consists of six parts: the current training round index t, the corresponding data size on client n | D n |, the number of times client n participates in local model update and upload until round t The group that client n belongs to until round t m The number of group model updates is The communication overhead of all pre-trained clients is B t = {B j |j∈P}, the training delay of all pre-trained clients

[0062] Step 102: Input the current state into the reinforcement learning agent network to obtain the decision result:

[0063]

[0064] Among them, 1 means that the client participates in the training and uploading of this round of models, and 0 means not participating.

[0065] Among them, the reinforcement learning agent network aims to maximize the cumulative return, specifically:

[0066] After the group completes the training, upload the group model to update the global model and then calculate the reward:

[0067]

[0068] Among them, u is a constant greater than 1, and the appropriate value will be selected according to the experimental conditions. t It indicates the accuracy of the global model updated on the test set after the appropriate client training is completed in the tth round, acc last represents the latest global model accuracy, represents the sum of the communication overhead of all clients selected intelligently in round t, represents the sum of the training delays of all clients intelligently selected in round t;

[0069] The reinforcement learning agent network will be trained to maximize the expected cumulative reward R, which is described as follows:

[0070]

[0071] Where E is the total number of updates of the global model and γ∈(0,1] is the discount factor for future rewards.

[0072] Step 2: Intra-group synchronous training phase: Receive the local model obtained by training the participating models in each group of clients and the uploaded client training to obtain the group model.

[0073] All clients that decide to train and upload models within a group will perform synchronous federated training to obtain the group model:

[0074]

[0075]

[0076] Among them, P t '、|P t '|、n k 、N c ,η, Respectively represent the group in round t m Intelligently selected client subset, number of intelligently selected clients, amount of data on client k, P t ' is the amount of data, learning rate, and gradient of the local experience loss of client k.

[0077] Step 3: Inter-group asynchronous training phase. Perform weighted aggregation on the group models to obtain the global model. When performing weighted aggregation on the group models, the weight of each group model is related to the number of updates of the group model of each group.

[0078] Assume that all clients are divided into M groups. So far, the number of updates for each group is T1, T2, …, T M , the total number of updates for all groups is T1+T2+...+T M =T, then the description of the global model obtained by weighted aggregation is as follows:

[0079]

[0080] in, For group m The corresponding weight, According to this formula, the relatively slow group M+1-m has a smaller value, T (M+1-m) The value is larger and is thus assigned a larger weight value.

[0081] In order to solve the problem of long waiting delays in synchronous federated learning and the communication bottleneck problem in fully semi-asynchronous federated learning, the present invention proposes an asynchronous federated learning method based on a multi-agent model (abbreviated as MAAFL). Different from other semi-asynchronous methods, the present invention introduces multi-agent reinforcement learning to perform efficient intra-group client intelligent selection. Each client has a reinforcement learning agent, which will decide whether to participate in the training and upload aggregation of this round of model according to its own observations. The invention combines multi-agent reinforcement learning in an asynchronous scenario to perform an efficient client intelligent selection strategy, which not only optimizes the model accuracy, but also greatly improves the model training efficiency. Compared with other more advanced semi-asynchronous federated learning methods, it spends less communication overhead and training delay when training to achieve a specified accuracy.

[0082] This embodiment conducts comparative experiments on MAAFL and a synchronous federated averaging method (abbreviated as FedAvg), a fully asynchronous asynchronous federated method (abbreviated as FedAsync), and a hierarchical semi-asynchronous method (abbreviated as FedAT).

[0083] The synchronized federated averaging method (abbreviated as FedAvg) is a baseline federated learning method. In each round, a certain proportion of all clients are randomly selected for training, and the server averages and aggregates the weights received from the selected clients.

[0084] The fully asynchronous asynchronous federated method (abbreviated as FedAsync) is a baseline asynchronous federated learning method that uses weighted average to update the server's global model. Unlike the previous synchronous federated learning method, all clients are trained at the same time. When the server receives weights from any client, it immediately weights these weights with the current global model weights to get the latest global model, and then communicates with all currently available clients for training.

[0085] The layered semi-asynchronous method (abbreviated as FedAT) is a semi-asynchronous federated learning method that combines synchronous intra-layer training and cross-layer asynchronous training. A random selection strategy is used within the layer to select some clients for synchronous federated training, and the layers communicate with the central server asynchronously to update the global model.

[0086] In order to verify the effectiveness of the asynchronous federated learning method based on the multi-agent model, the experiment compared the model accuracy of FedAvg, FedAsync, FedAT, and MAAFL training as well as the communication overhead and training delay required to achieve the specified accuracy.

[0087] In the experiment, three datasets were used, namely MNIST, Fashion-MNIST, and CIFAR-10. MNIST is a handwriting recognition dataset containing 60,000 training samples and 10,000 test samples. Each example is a 28×28 grayscale image associated with a label from 10 classes; Fashion-MNIST is a clothing image dataset containing 60,000 training samples and 10,000 test samples. Each example is a 28×28 grayscale image associated with a label from 10 classes; The CIFAR-10 dataset contains 60,000 32×32 color images, divided into 10 classes, 6,000 images per class, 5,000 for training and 1,000 for testing.

[0088] The experiment evaluates the model on these three different datasets, all of which use a non-independent and identically distributed data partitioning method. Specifically, all data of each dataset is first divided into 200 fragments according to the category label. Among the 10 category labels, all data corresponding to each label is divided into 20 fragments, and 2 fragments belonging to different label categories are allocated to each local client. This division is to ensure that among the 100 local clients, each client has only data with two category labels, and the data size on each client is different, thereby ensuring that the training is performed in a non-independent and identically distributed data environment.

[0089] The experiment randomly divides each client's local data into 80% training set and 20% test set. For the intra-group synchronous training, the same sampling method as the federated average is adopted. MAAFL first randomly selects a part of clients as pre-training clients, and then each pre-training client performs intelligent decision-making to decide whether to participate in this round of training, using stochastic gradient descent (SGD) as the local optimizer. For all data sets, the training configuration parameters of the local model are unified: the learning rate is 0.01, the number of local iterations is 3, and the batch size is 10. For all algorithms, the number of randomly selected pre-training clients is set to 20.

[0090] The experiment simulated different performance groups. First, all clients were evenly divided into 5 groups. Then, the response time of each group of clients was randomly assigned to 1-3 rounds, 3-5 rounds, 5-7 rounds, 7-9 rounds, and 9-11 rounds. In addition, in order to simulate unstable network connections, 20 "unstable" clients were randomly selected for all tests. These clients will exit the training with a certain small probability during the training process. Once a client exits, it will not come back to rejoin the federated training.

[0091] In the experiment, the communication cost of each local client communicating with the central server (uploading or downloading models) is fixed. In order to simulate the heterogeneity of local clients in actual scenarios, a fixed communication cost value is randomly assigned to each client within a fixed range. Different clients have different communication costs. The total communication cost is the sum of the communication costs of all clients with the central server during the entire training process.

[0092] In the experiment, due to the differences in computing power of different local clients and other factors, the local training model is uploaded with delay in each round. Among the local clients that are intelligently selected for training in one round, the fast clients need to wait for the slow clients. For example, a client with a response time of 2 rounds needs to wait for a client with a response time of 5 rounds, and the delay is 3 global rounds. The total training delay is the accumulation of the training delay of each client during the entire training process.

[0093] The MAAFL algorithm and three comparison methods are evaluated on three data sets. Each algorithm is run three times on each data set, and each time is run until the model converges to obtain the optimal global model accuracy. The three best results are averaged to obtain the optimal global model average accuracy, and the standard deviation of the three results is calculated to obtain the results shown in Table 1.

[0094] Table 1. Accuracy performance of different algorithms on three datasets

[0095]

[0096]

[0097] The communication overhead and training latency required to achieve a specified accuracy on the three datasets are shown in Tables 2, 3, and 4.

[0098] Table 2. Communication overhead and training delay of different algorithms to achieve a specified accuracy on the MNIST dataset

[0099]

[0100] Table 3. Communication overhead and training delay of different algorithms to achieve specified accuracy on F-MNIST dataset

[0101]

[0102] Table 4. Communication overhead and training delay of different algorithms to achieve specified accuracy on CIFAR-10 dataset

[0103]

[0104] The experiment further simulated a more complex scenario, setting the slowest client's response time to 21 rounds behind. The experimental results are as follows: Figure 2 , Figure 3(a), Figure 3(b), Figure 4 , as shown in Figure 5(a) and Figure 5(b).

[0105] From these experimental results, we can see that in general scenarios, although the overall performance of the MAAFL method is not as good as FedAT, which is also a semi-asynchronous method, it also has its own advantages. In more complex scenarios, MAAFL exceeds FedAT in both accuracy and training efficiency. The highlights of the experimental results are summarized as follows:

[0106] (1) Compared with FedAvg, MAAFL can effectively solve the training delay problem existing in FedAvg, which is of great significance in real heterogeneous scenarios and avoids the long waiting process between clients;

[0107] (2) Compared with FedAsync, MAAFL avoids the communication bottleneck problem caused by frequent communication with the central server in the fully asynchronous method;

[0108] (3) Compared with FedAT, which is also a semi-asynchronous method, MAAFL can better demonstrate its advantages in complex scenarios. Not only does it exceed FedAT in accuracy, but it also reduces the training delay by an average of 44% and the communication overhead by 36% compared with FedAT when achieving the specified accuracy on two more complex datasets.

[0109] This embodiment first groups all the clients participating in the federated training according to their response time, performs synchronous federated training within the group to obtain the group model, updates the group model between groups in an asynchronous weighted manner, and introduces multi-agent reinforcement learning. Each client has a reinforcement learning agent, which will decide whether to participate in the training of this round of models based on its own observations. The present invention can solve the problem of machine learning model training in more complex heterogeneous client scenarios, not only can it solve the problem of long waiting delays in synchronous federated learning, but also can solve the communication bottleneck problem in completely semi-asynchronous federated learning. In addition, the present invention introduces multi-agent reinforcement learning to execute an efficient client intelligent selection mode to replace the random selection strategy in the previous federated learning method, which not only optimizes the model accuracy, but also greatly improves the model training efficiency. Compared with other more advanced semi-asynchronous federated learning methods, it spends less communication overhead and training delay when training reaches the specified accuracy, and can be applied to the common and extensive heterogeneous client machine learning model training scenarios in real life.

[0110] Embodiment 2

[0111] This embodiment provides an asynchronous federated learning system based on a multi-agent model, which specifically includes the following modules:

[0112] A client intelligent selection module is configured to: randomly select a number of pre-trained clients from each group of clients, and obtain a decision result of whether each pre-trained client participates in the training and uploading of the model;

[0113] The intra-group synchronous training module is configured to: receive the local model obtained by training the participating models and the uploaded client training in each group of clients, and obtain a group model;

[0114] The inter-group asynchronous training module is configured to perform weighted aggregation on the group models to obtain a global model.

[0115] The client intelligent selection module is specifically configured as follows:

[0116] Get the status of each pre-trained client;

[0117] The state of each pre-trained client is input into the reinforcement learning agent network to obtain the decision result of whether each pre-trained client participates in the training and uploading of the model.

[0118] When weighted aggregation is performed on the group models, the weight of each group model is related to the number of updates of the group model of each group.

[0119] It should be noted here that each module in this embodiment corresponds to each step in Example 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0120] Embodiment 3

[0121] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the asynchronous federated learning method based on a multi-agent model as described in the first embodiment above are implemented.

[0122] Embodiment 4

[0123] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the asynchronous federated learning method based on a multi-agent model as described in the first embodiment above are implemented.

[0124] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0125] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0128] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An asynchronous federated learning method based on a multi-agent model, characterized in that: include: Randomly select a number of pre-trained clients from each group of clients, and obtain the decision result of whether each pre-trained client participates in the training and uploading of the model; Receive the local models obtained by training the participating models and the uploaded clients in each group of clients to obtain a group model; Perform weighted aggregation on the group models to obtain the global model; When performing weighted aggregation on group models, the weight of each group model is related to the number of updates of the group model of each group; The method for obtaining the decision result is: Get the status of each pre-trained client; Input the status of each pre-trained client into the reinforcement learning agent network to obtain the decision result of whether each pre-trained client participates in the training and uploading of the model; The status of the pre-trained client includes: training round index t, the amount of data on the client, the number of times the client has participated in local model updates and uploads up to the tth round, the number of updates of the group model to which the client belongs up to the tth round, the communication overhead of all pre-trained clients, and the training delay of all pre-trained clients.

2. The asynchronous federated learning method based on a multi-agent model as claimed in claim 1, characterized in that: The reinforcement learning agent network aims to maximize the cumulative reward.

3. An asynchronous federated learning system based on a multi-agent model, characterized in that: include: A client intelligent selection module is configured to: randomly select a number of pre-trained clients from each group of clients, and obtain a decision result of whether each pre-trained client participates in the training and uploading of the model; The intra-group synchronous training module is configured to: receive the local model obtained by training the participating models and the uploaded client training in each group of clients, and obtain a group model; An inter-group asynchronous training module, which is configured to: perform weighted aggregation on the group models to obtain a global model; When performing weighted aggregation on group models, the weight of each group model is related to the number of updates of the group model of each group; The method for obtaining the decision result is: Get the status of each pre-trained client; Input the status of each pre-trained client into the reinforcement learning agent network to obtain the decision result of whether each pre-trained client participates in the training and uploading of the model; The status of the pre-trained client includes: training round index t, the amount of data on the client, the number of times the client has participated in local model updates and uploads up to the tth round, the number of updates of the group model to which the client belongs up to the tth round, the communication overhead of all pre-trained clients, and the training delay of all pre-trained clients.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in an asynchronous federated learning method based on a multi-agent model as described in any one of claims 1-2 are implemented.

5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the asynchronous federated learning method based on a multi-agent model as described in any one of claims 1-2 are implemented.

Images