Multi-objective federated learning evolutionary method based on improved NSGA-III

Abstract

The invention belongs to the field of artificial intelligence, and discloses a multi-objective federated learning evolutionary method based on an improved NSGA-III, comprising the following steps: acquiring learning data; constructing a federated learning multi-objective optimization model; fast greedy initialization, and performing multi-objective evaluation; performing non-dominant sorting; performing iteration, and performing the following operations in each iteration: selecting, crossover, and mutation operator operations to generate Q t , conduct federated learning training and evaluation on the offspring population, and calculate the three objectives of each individual; mix the parent and offspring populations R t =Q t +P t ; for R t Non-dominant sorting, and selection of P based on reference point t+1 ; Find the Pareto optimal solution and output the annotation result corresponding to the Pareto optimal solution. Compared with the classical algorithm, the present invention can obtain a better Pareto solution, reduce the communication cost and the variance of the global model accuracy distribution while ensuring the accuracy of the global model.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a multi-objective federated learning evolutionary method based on an improved NSGA-III. Background Art

[0002] While the rapid development of artificial intelligence has brought great convenience to society, it has also brought some hidden dangers, such as data silos and privacy leaks. Traditional centralized machine learning requires the aggregation of scattered data for machine learning training, but in fact, data in many fields is difficult to aggregate together. For example, it is difficult for hospitals to share data, and there is a serious "data silo" problem. In addition, the problem of privacy leakage has emerged, people's awareness of privacy protection has gradually increased, and countries around the world have also introduced laws and regulations on privacy protection.

[0003] Therefore, federated learning has emerged as a feasible solution to the problems of data silos and privacy leakage. Federated learning can train a good global model while keeping the data in the local area of ​​the participants. After each participant downloads the current global model from the server, it is trained on the local data, and then the trained local model is uploaded to the server for model aggregation and update. After multiple rounds of iterations, a global model with good performance is finally obtained.

[0004] However, traditional federated learning still faces the challenges of high communication costs and structural heterogeneity. The transmission of parameters between the server and participants of federated learning consumes a lot of communication costs. At the same time, due to different computing and storage capabilities and network environments between different participants, participants may go offline during training and lose transmission model parameters, which affects the efficiency, accuracy, and fairness of federated learning. There are many papers devoted to the study of communication costs or structural heterogeneity, but there is little comprehensive consideration of these issues.

[0005] In view of the above problems, the present invention comprehensively considers the communication cost and structural heterogeneity problems, and studies the balance between model effectiveness, fairness and communication cost. The present invention first defines federated learning as a three-objective optimization model, with the goal of maximizing the global model accuracy, minimizing the variance of the global model accuracy distribution and the communication cost at the same time. Based on the characteristics of federated learning training, the initialization of the third-generation non-dominated sorting genetic algorithm (non-dominated sorted genetic algorithm-III, NSGA-III) is improved, and a NSGA-III algorithm based on fast greedy initialization (Fast greedy initialization NSGA-III, FNSGA-III) for multi-objective federated learning is designed. Experimental results show that the FNSGA-III algorithm can achieve a balance between the three objectives, and can effectively reduce the communication cost and the variance of the accuracy distribution of each participant while maintaining the overall performance of the FL model without serious loss, so that the accuracy distribution of the participants is more balanced. The work done by the present invention mainly includes:

[0006] (1) To the best of the authors’ knowledge, this paper is the first to comprehensively consider the objectives of maximizing the global model accuracy, minimizing the variance of the global model accuracy distribution, and minimizing the communication cost, and construct a multi-objective optimization model for federated learning.

[0007] (2) Propose the FNSGA-III algorithm. In order to quickly converge the NSGA-III algorithm and obtain high-quality solutions, and to facilitate the solution of the federated learning multi-objective optimization model, an initial solution construction algorithm based on fast greedy initialization is proposed, and binary and real-valued encoding and decoding strategies are introduced to accelerate the evolution efficiency of the NSGA-III algorithm.

[0008] (3) Through the MNIST dataset experiment, it is verified that the Pareto solution obtained by the FNSGA-III algorithm is better than the NSGA-III algorithm. The hypervolume HV index value of the FNSGA-III algorithm can reach up to 127.55% of the NSGA-III algorithm, and the running time is 73.96% of the NSGA-III algorithm under the optimal condition. In addition, the FNSGA-III algorithm is compared with other classic evolutionary algorithms NSGA-II and SPEA2. The results show that the Pareto solution obtained by the FNSGA-III algorithm has higher quality. Finally, some Pareto solutions are selected for federated learning experiments. The Pareto solution obtained by the algorithm can effectively reduce the communication cost and the variance of the global model accuracy distribution while ensuring the accuracy of the global model.

[0009] In recent years, federated learning has received widespread attention. McMahan first proposed the concept of federated learning and the Federated Averaging (FedAvg) algorithm in 2016, which has important application significance for privacy protection and machine learning training under data silos. The research on federated learning continues to deepen, but it still has some unsurmounted challenges, such as high communication costs and structural heterogeneity.

[0010] In order to make federated learning applicable to massive data, it is necessary to consider reducing the communication overhead of federated learning. The FedAvg algorithm proposed by McMahan et al. reduces the number of global communication rounds by increasing the amount of local training calculations in each round of communication, thereby improving communication efficiency. Other scholars have reduced the amount of communication transmission by reducing the size of parameters uploaded by participants. Chen et al. proposed a hierarchical asynchronous update algorithm. The authors hierarchically divide the parameters into shallow parameters and deep parameters according to the structural characteristics of the deep neural network model. In the early global communication iteration process, only shallow parameters are transmitted between local participants and the server, and only in the last few rounds of communication are the deep parameters of the global model transmitted and aggregated. The algorithm reduces the communication overhead by reducing the size of the transmission model parameters and reducing the update frequency of the deep parameters in the neural network. The disadvantage is that the accuracy of the model will be affected. Zhu et al. introduced the sparse evolutionary training algorithm (SET) into federated learning. The main idea of ​​the SET algorithm is to control the connection sparsity between the fully connected layers of the neural network through the sparsity parameters, thereby reducing the size of the transmission model parameters and effectively reducing the communication cost.

[0011] In addition to communication costs, structural heterogeneity is also one of the main problems of federated learning optimization. Due to different computing and storage capabilities and network environments among different participants, participants may go offline during training and lose transmission model parameters. In order to enhance the robustness of federated learning, some scholars have conducted multi-faceted research on the problem of structural heterogeneity. Hao et al. designed a secure aggregation protocol that allows participants to exit at any time as long as the remaining number of participants can meet the update of federated learning, thereby improving the fault tolerance and robustness of the system. Some scholars have studied how to reasonably allocate heterogeneous device resources. Kang et al. considered the differences in participants' overhead to encourage more high-quality participants to conduct federated learning training. Li et al. used the global model accuracy variance as a fairness measure and designed a q-FFL (q-Fair federated learning) optimization algorithm, which increases the model aggregation weight of high-loss participants. Experiments show that the algorithm can improve the accuracy of low-accuracy participants, achieve balanced performance distribution among different participants, and promote fair resource allocation for federated learning.

[0012] The above studies are conducted on one aspect of communication cost or structural heterogeneity, and achieve optimization of federated learning algorithms to different degrees and different goals. However, the application of federated learning often requires model accuracy, fairness, communication cost, etc. In order to achieve a balance of multiple goals under the federated learning framework, some scholars have tried to combine intelligent optimization algorithms with federated learning. Zhu et al. defined federated learning as a dual-objective optimization problem, with the goal of minimizing the model test error rate and communication cost. They used the NSGA-II (non-dominated sorted genetic algorithm-II) algorithm to optimize the structural parameters of the neural network of federated learning. The Pareto solution evolved by the algorithm improves the model performance and communication efficiency to a certain extent compared with the standard FedAvg algorithm. However, the algorithm does not consider other situations such as communication instability and uneven distribution of accuracy among participants caused by the structural heterogeneity of federated learning. At the same time, the NSGA-II algorithm used has poor scalability for multi-objective federated learning models. Basheer et al. used a particle swarm algorithm to update and optimize the number of hidden layers, number of neurons, and global communication rounds of the neural network of federated learning, but its optimization goal is a single goal, and other goals of federated learning are not comprehensively considered. Summary of the invention

[0013] In response to the above problems, the present invention comprehensively considers the communication cost and structural heterogeneity issues, introduces fairness as an optimization goal, and explores the multi-objective equilibrium relationship between the model accuracy, fairness, and communication cost of federated learning. In the experiment, the communication environment is set to be unstable to enhance the robustness of the algorithm.

[0014] The multi-objective federated learning evolutionary method based on the improved NSGA-III proposed in the present invention is applied to a server and multiple participants, and includes the following steps:

[0015] Acquire learning data, where the learning data is used for labeling;

[0016] Constructing a federated learning multi-objective optimization model, wherein the multi-objective optimization model includes three objectives: maximizing global model accuracy, minimizing global model accuracy distribution variance, and communication cost;

[0017] Fast Greedy Initialization P t =0, use FedAvg algorithm for multi-objective evaluation;

[0018] P t Perform non-dominant sorting;

[0019] Iterate, and each iteration performs the following operations: selection, crossover, and mutation operator operations to generate Q t , conduct federated learning training and evaluation on the offspring population, and calculate the three objectives of each individual; mix the parent and offspring populations R t =Q t +P t ; for R t Non-dominant sorting, and selection of P based on reference point t+1 ;

[0020] Find the Pareto optimal solution and output the annotation results corresponding to the Pareto optimal solution.

[0021] Furthermore, in the evolution of federated learning, we have a dataset D k The loss function of the kth participant is:

[0022]

[0023] The global goal of the federated learning evolutionary method is to minimize the following global loss function L(w):

[0024]

[0025] Among them, k is the serial number of the participant, L k (w) is the loss function of the kth participant, l i (w) is the loss function on data sample i, n k is the dataset D of participant k k Size n k =|D k |, n is the total data sample size of K participants.

[0026] Furthermore, in each round of training in the federated learning process, each participant receives the global model w from the server. t , and use local data to train the global model to obtain an updated local model The participants then send the updated local models to the server, which aggregates the models according to certain rules to obtain a new global model w t+1 , used for the next round of iterative training, and the subscript t represents the communication round of federated learning.

[0027] Furthermore, the three-objective optimization model of federated learning evolution is:

[0028]

[0029] Among them, F(v) is the objective function of the three-objective optimization model. This model has three minimization objectives, namely minimizing the global model test error rate f1, the global model accuracy distribution variance f2, and the communication cost f3. Conv is the number of convolutional layers, kc is the number of convolutional kernels, ks is the size of the convolutional kernel, L is the number of fully connected layers, N is the number of neurons in the fully connected layer, η is the learning rate, and ε is the connectivity parameter of the neural network. The number of connections between two fully connected layers is determined by the connectivity parameter ε. The total number of connections is n=ε(n k +n k-1 ), where n k and n k-1 are the number of neurons in the k layer and k-1 layer respectively.

[0030] Furthermore, the global model test error rate of target f1 is E=1-A, where A is the average test accuracy of the global model {a1,a2,...,a K} is the accuracy of each participant.

[0031] Furthermore, the target f2 is the global model accuracy distribution variance

[0032] Furthermore, the target f3 can be expressed as K is the total number of participants, C is the proportion of participants participating in each round, and σ is the size of the model parameters.

[0033] Furthermore, the process of training and evaluating the federated learning of the offspring population is implemented by the FedAvg algorithm based on static SET, which specifically includes the following steps:

[0034] i is an individual in the population of the FNSGA-III algorithm, P is the population size, and after decoding individual i, we can get the relevant federated learning neural network hyperparameters, the connectivity of the neural network, and the participation ratio C of participants in each round.i ;

[0035] Use the communication parameter ε i Initialize a static SET topology and use the static SET topology as a global model used in the algorithm;

[0036] In each round of training, the local data is trained using the mini-batch stochastic gradient descent method;

[0037] After a certain number of rounds, the three objectives of the global model’s test error rate, global model accuracy distribution variance, and communication cost are calculated.

[0038] Furthermore, the number of convolutional layers, the number of convolutional kernels, the size of convolutional kernels, the number of fully connected layers, the number of neurons in each layer of the fully connected layer, and the SET parameter ε of MLP and CNN are encoded in binary, and the learning rate η and the participation ratio C of each round are encoded in real value.

[0039] Furthermore, the learning data is the MNIST data set.

[0040] The beneficial effects of the present invention are as follows:

[0041] The present invention proposes the FNSGA-III algorithm to solve the problem of multi-objective federated learning model, and conducts experimental verification under unstable communication. We first construct a three-objective template for federated learning, set the optimization goal to minimize the global model test error rate, communication cost, and global model accuracy distribution variance, and the decision variables are the hyperparameters of the neural network and the federated learning parameters. The NSGA-III algorithm is introduced to solve the federated learning multi-objective model, and the initialization of NSGA-III is changed. The experimental results show that the improved FNSGA-III algorithm is better than the original NSGA-III algorithm. And the Pareto optimal solution optimized by the FNSGA-III algorithm is compared with the benchmark federated average algorithm, which effectively improves the global model accuracy, reduces the global model accuracy distribution variance and communication cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Diagram of the federated learning training process;

[0043] Figure 2 An example diagram of chromosome encoding of the MLP model of the present invention;

[0044] Figure 3 An example diagram of chromosome encoding of the CNN model of the present invention;

[0045] Figure 4 Algorithm flow chart of the present invention;

[0046] Figure 5Experimental comparison of the MLP of the present invention and the NSGA-III algorithm on IID;

[0047] Figure 6 Experimental comparison of the MLP of the present invention and the NSGA-III algorithm on non-IID;

[0048] Figure 7 Experimental comparison of CNN of the present invention and NSGA-III algorithm on IID;

[0049] Figure 8 Experimental comparison of the CNN of the present invention and the NSGA-III algorithm on non-IID;

[0050] Fig. 9 Pareto optimal solution diagram of the present invention, NSGA-II and SPEA2. DETAILED DESCRIPTION

[0051] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0052] To achieve this object, the technical solution adopted by the present invention includes the following steps:

[0053] In order to make the technical solution and beneficial effects of the present invention clearer, the present invention is further described below in conjunction with practical examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0054] Example

[0055] Federated learning is a privacy-preserving machine learning technology that enables participants to jointly train a global model without uploading local private data to the server. Assume that there are K participants with data {D1,D2,…,D K}, traditional centralized learning is to put all the data together and use D = D1 ∪ D2 ... D K to train the model.

[0056] In the federated learning process, we have a dataset D k The loss function of the kth participant is:

[0057]

[0058] The global goal of federated learning is to minimize the global loss function L(w):

[0059]

[0060] In formulas (1) and (2), k is the serial number of the participant, L k (w) is the loss function of the kth participant, l i (w) is the loss function on data sample i, n k is the dataset D of participant k k Size n k =|D k |, n is the total data sample size of K participants. The goal of federated learning is to minimize the participant loss function L k (w) to optimize the global loss function L(w). Federated learning is a collaborative process, such as Figure 1 shown.

[0061] In each round of training, each participant receives the global model w from the server t , and use local data to train the global model to obtain an updated local model The participants then send the updated local models to the server, which aggregates the models according to certain rules to obtain a new global model w t+1 , used for the next round of iterative training. The subscript t represents the communication round of federated learning. The third generation non-dominated sorting genetic algorithm (NSGA-III)

[0062] There are many research results on multi-objective optimization evolutionary algorithms based on genetic algorithms and Pareto optimal solutions, such as the second generation non-dominated sorting genetic algorithm (NSGA-II), multi-objective evolutionary algorithm based on decomposition (Multi-objective evolutionary algorithm based on decomposition, MOEA / D), SPEA2 (Strength Pareto Evolutionary Algorithm 2), and PASpareto archived evolution strategy). NSGA-II is a powerful and robust multi-objective evolutionary algorithm suitable for problems with two or three objectives. If the number of objectives is greater than 3, a newer evolutionary algorithm can be used, such as the third generation non-dominated sorting genetic algorithm (NSGA-III) based on reference points. NSGA-III performs better than NSGA-II in optimization problems with four or more objectives. The present invention defines federated learning as a three-objective optimization problem model. In order to ensure the scalability of the algorithm objectives, for example, when the objectives of federated learning are increased to four or more, the algorithm is still applicable. The present invention adopts the NSGA-III algorithm, and its basic steps can be summarized as follows:

[0063] Step 1: Initialize the reference point and parent population P t (The population size is N), the individuals in the population are non-dominated sorted, normalized, and associated to the reference point.

[0064] Step 2 In P t Use selection, crossover, and mutation operators to create a population P that is similar to the parent population t The offspring population Q of the same size t .

[0065] Step 3 Mix P t and Q t A new population R t , where R t The population size is 2N. Perform non-dominated sorting on the merged population and divide it into different non-dominated solution sets (F1, F2, ..., F s ), and normalize the individuals and relate them to the reference point.

[0066] Step 4: From the sorted population R t Select N solutions from the next generation of parent population P t+1 If the number of selected non-dominated frontier solutions is greater than N, the reference point-based selection method is used to select solutions from the last layer of non-dominated solutions to be selected.

[0067] Step 5: Go to step 2 and repeat the whole process until the preset stop condition is met and the Pareto optimal solution is output.

[0068] The present invention constructs a three-objective optimization model of federated learning based on a typical multi-objective optimization model, and describes its objectives, decision variables, variable encoding, etc. The three-objective optimization model of federated learning of the present invention is:

[0069]

[0070] (1) Objective function:

[0071] F(v) is the objective function of the model. This model has three minimization objectives, namely minimizing the global model test error rate f1, the global model accuracy distribution variance f2, and the communication cost f3.

[0072] In the present invention, the specific evaluation process of the three objectives of federated learning is to use the FedAvg algorithm combined with the SET algorithm to perform a certain number of communication rounds of training, test the trained global model w, and obtain the accuracy of each participant {a1, a2, ..., a K}. Calculate the average test accuracy of the global model Thus, the target f1 global model test error rate E=1-A can be calculated.

[0073] Target f2 is the global model accuracy distribution variance Variance can be regarded as one of the metrics of fairness. Taking fairness as the optimization goal can avoid the situation where the average accuracy is high but the accuracy of a single participant is not guaranteed. By reducing the variance of the global model accuracy distribution among participants, the accuracy distribution of the participants after aggregation is made more uniform and fairer.

[0074] The target f3 is the average communication cost of the participants. Assuming that the communication cost of each participant is only related to the size of the model parameters transmitted by it, since the neural network structure used in the present invention does not change during the federated learning training process, the model parameter size σ of each participant is the same and remains unchanged, then the target f3 can be expressed as K is the total number of participants, C is the proportion of participants participating in each round, and σ is the size of the model parameters.

[0075] (2) Model decision variables and their constraints

[0076] Since federated learning is a collaborative training process of machine learning models, in this invention, the decision variables and constraints are the parameters to be optimized and their ranges, represented by v. The parameters we set to be optimized include the hyperparameters of the neural network of federated learning, the connectivity parameter ε of the neural network, and the participation ratio C of each round.

[0077] The neural network of the present invention selects a multilayer perceptron (MLP) and a convolutional neural network (CNN), wherein the hyperparameters under the MLP model include the number of hidden layers L, the number of neurons N in each hidden layer, and the learning rate η; the hyperparameters of the CNN include the number of convolution layers Conv, the number of convolution kernels kc, the size of the convolution kernel ks, the number of fully connected layers L, the number of neurons N in the fully connected layer, and the learning rate η. That is, v = {Conv, kc, ks, L, N, ε, η, C}, and the value range of each variable is set in the experimental part.

[0078] The connectivity parameter ε of the neural network is based on the static SET algorithm proposed by Mocanu, which first initializes the sparse weight matrix between two fully connected layers using an ER random graph, and then keeps the network topology fixed. The number of connections between the two layers is determined by the parameter ε, and the total number of connections is n = ε (n k +n k-1 ), where n k and n k-1are the number of neurons in the k layer and the k-1 layer respectively. In the present invention, the static SET algorithm is used in the fully connected layers of MLP and CNN.

[0079] (3) Decision variable encoding

[0080] We use the FNSGA-III algorithm to optimize the neural network hyperparameters, connectivity parameter ε, and participation ratio C of each round of federated learning. Chromosomes are the main body of the algorithm operation. In the present invention, we have two types of decision variables, integers and real numbers, that need to be encoded, where all integers use binary encoding and real numbers use real value encoding. Then the number of convolutional layers, number of convolutional kernels, size of convolutional kernels, number of fully connected layers, number of neurons in each layer of fully connected layers, and SET parameter ε of MLP and CNN use binary encoding, and the learning rate η and participation ratio C of each round use real value encoding. The encoding examples of MLP and CNN are as follows: Figure 2 and Figure 3 shown.

[0081] During the decoding process, 1 is automatically added when decoding binary. For example, 000000 is decoded as 1. In the MLP example, N1 is encoded as 000111 and decoded as N1 = 8. For convenience, the convolution kernel size in CNN is only selected between 3 and 5, and the convolution output is always kept unchanged. In the neural network structure of CNN, only one pooling layer is added at the end.

[0082] In order to speed up the search performance, the present invention improves the random initialization in NSGA-III into a fast greedy initialization suitable for federated learning, and the improved NSGA-III algorithm is recorded as FNSGA-III algorithm. The fast greedy initialization process is briefly described as follows:

[0083] (1) Randomly generate a l-fold population initial solution;

[0084] (2) After all participants are randomly divided into groups of the same size, the federated learning training and evaluation process of the initial solution is carried out synchronously in each group. The number of participants in each round of federated learning, the number of local training rounds, and the number of global communication rounds are all reduced, so that the three goals after federated learning training can be obtained more quickly and the evaluation of all initial solutions can be completed;

[0085] (3) Then select the optimal double population solutions for the three objectives respectively;

[0086] (4) After removing duplicate solutions from the mixture, randomly select a specified population size solution from them.

[0087] FNSGA-III Algorithm Process

[0088] For the three-objective optimization model of federated learning, the present invention uses the FNSGA-III algorithm to solve and obtain a set of Pareto optimal solutions. The algorithm flow chart is as follows: Figure 4 shown.

[0089] FNSGA-III first uses fast greedy initialization to generate an initial population of size N, i.e., the first generation parent population, and encodes the corresponding variables into binary and real-valued chromosomes. Its iterative process mainly uses a binary tournament to select two parent individuals to generate two offspring individuals. The crossover and mutation algorithms use single-point crossover and flip mutation on binary chromosomes, and simulated binary crossover (SBX) and polynomial mutation on real-valued chromosomes. This process is repeated until N offspring individuals are generated.

[0090] Then, the offspring population is trained and evaluated by federated learning, and the three objectives of each individual are calculated. The parent population and offspring population are mixed, and the mixed population is non-dominated sorted. N individuals are selected as the parent population of the next generation. These steps are repeated until the iteration stop condition is met. Finally, a set of Pareto optimal solutions are obtained, which are analyzed in depth.

[0091] The specific evaluation process of federated learning is based on the FedAvg algorithm combined with the static SET algorithm. The pseudo code of the federated learning evaluation process under the FNSGA-III algorithm is shown in Algorithm 1.

[0092] In Algorithm 1, i is an individual in the population of the FNSGA-III algorithm, and P is the population size. After decoding individual i, the relevant federated learning neural network hyperparameters, the connectivity of the neural network, and the participation ratio C of participants in each round are obtained. i First, use the communication parameter ε i Initialize the static SET topology and use it as the global model used in the algorithm; in each round of training, use the mini-batch stochastic gradient descent method (Mini-batch SGD) to train local data. After a certain number of rounds, calculate the three objectives of the global model's test error rate, global model accuracy distribution variance, and communication cost.

[0093]

[0094]

[0095] In this section, we will introduce the experimental settings of the present invention. It mainly includes the following parts: (1) experimental environment and experimental data set; (2) neural network related parameters and sparse connectivity parameters used in the experiment; (3) federated learning parameters and data partitioning method; (4) FNSGA-III parameters.

[0096] The experimental environment is based on an Ubuntu system with an Intel(R) Core(TM) i9-9900KF CPU@3.60GHz×16. Each experiment is trained and tested on the MNIST dataset, which consists of 28×28 pixel handwritten digit images, with 60,000 training images and 10,000 test images.

[0097] We selected MLP and CNN as the neural network models for federated learning training, and set the standard MLP and CNN parameters of the present invention based on experience. Among them, there are 2 hidden layers in MLP, each with 200 neurons (the number of parameters is 199210), and the ReLu function is used as the activation function. The CNN model sets two 5×5 convolutional layers (the first has 32 channels and the second has 64 channels), followed by a 2×2 Max pooling layer, a fully connected layer with 128 neurons, using the ReLu activation function, and finally a 10-class softmax output layer (the number of parameters is 1659146). In MLP and CNN, the learning rate η of the Mini-batch SGD algorithm is 0.05, and the batch size B is 10. The static SET algorithm is used in the fully connected layers of MLP and CNN, and the present invention sets the network sparsity parameter ε=20. The above parameter settings are used as the standard neural network structure in the experiment of the present invention.

[0098] In federated learning, we set the total number of participants K to 100 and the participant participation ratio C to 1, that is, there are 100×1 participants in each round of communication. For participant local model training, the iteration round epoch is set to 5. Since the size and distribution of data between different participants are often different, we study the following two real-life scenarios: the first independent identical distribution (IID), first shuffle the MNIST data, then each of the 100 participants contains 600 samples; the second non-independent identical distribution (non-IID), first sort the labeled digital labels, divide them into 200 fragments according to 300 samples, and assign two fragments to 100 participants, then each participant has only two labels and the data sample size is the same. Because the present invention assumes that the communication environment of federated learning is unstable and there is loss in the transmission of model parameters, the loss rate Drop is set to 30%.

[0099] Next, we set the parameters of FNSGA-III, where the population size is set to 20 and the iterations are 20. The selection operator uses a two-round tournament, and the binary chromosome uses a single-point crossover with a probability of 0.9 and a bit flip mutation with a probability of 0.1; the real-valued chromosome uses a probability of 0.9 and n c = 2 simulated binary crossover with probability 0.1 and n m =20 polynomial variation.

[0100] The present invention uses the FNSGA-III algorithm to optimize the relevant parameters of federated learning to achieve the global model test error rate, global model accuracy distribution variance and communication cost in federated learning. We selected MLP and CNN as the neural network models for federated learning training, and set the standard MLP and CNN parameters of the present invention based on experience. Among them, there are 2 hidden layers in MLP, each with 200 neurons (the parameter amount is 199 210), and the ReLu function is used as the activation function. The CNN model sets two 5×5 convolutional layers (the first has 32 channels and the second has 64 channels), followed by a 2×2 Max pooling layer, a fully connected layer of 128 neurons, using the ReLu activation function, and finally a 10-class softmax output layer (the parameter amount is 1 659 146). In MLP and CNN, the learning rate η of the Mini-batch SGD algorithm is 0.05, and the batch size B is 10. The static SET algorithm is used in the fully connected layer of MLP and CNN, and the present invention sets the network sparsity parameter ε=20. The above parameter settings are used as the standard neural network structure in the experiments of the present invention.

[0101] In federated learning, we set the total number of participants K to 100 and the participant participation ratio C to 1, that is, there are 100×1 participants in each round of communication. For participant local model training, the iteration round epoch is set to 5. Since the size and distribution of data between different participants are often different, we study the following two real-life scenarios: the first independent identical distribution (IID), first shuffle the MNIST data, then each of the 100 participants contains 600 samples; the second non-independent identical distribution (non-IID), first sort the labeled digital labels, divide them into 200 fragments according to 300 samples, and assign two fragments to 100 participants, then each participant has only two labels and the data sample size is the same. Because the present invention assumes that the communication environment of federated learning is unstable and there is loss in the transmission of model parameters, the loss rate Drop is set to 30%.

[0102] Next, we set the parameters of FNSGA-III, where the population size is set to 20 and the iterations are 20. The selection operator uses a two-round tournament, and the binary chromosome uses a single-point crossover with a probability of 0.9 and a bit flip mutation with a probability of 0.1; the real-valued chromosome uses a probability of 0.9 and n c = 2 simulated binary crossover with probability 0.1 and n m=20 polynomial variation.

[0103] The present invention uses the FNSGA-III algorithm to optimize the relevant parameters of federated learning to achieve a balance between the global model test error rate, the global model accuracy distribution variance and the communication cost in federated learning. First, a comparative experiment is conducted between the FNSGA-III algorithm and the NSGA-III algorithm to explore the effectiveness of the proposed method. The relevant parameter settings for the multi-objective evolutionary algorithm experimental analysis are shown in Figure 2.

[0104] Table 1. Related parameter settings of multi-objective federated learning evolutionary algorithm

[0105] parameter MLP CNN Population size 20 20 Maximum number of iterations 20 20 Participation rate of federated learning 0.1-1 0.1-1 Learning Rate 0.01-0.2 0.01-0.2 SET parameters 1-128 1-128 Number of hidden layers in MLP 1-4 Number of neurons in the MLP hidden layer 1-256 Number of CNN convolutional layers 1-3 Number of CNN convolution kernel channels 1-64 CNN convolution kernel size 3 or 5 Number of CNN fully connected layers 1-3 Number of neurons in CNN fully connected layer 1-256

[0106] We set the population size to 20, the population iteration number to 20, and the number of communication rounds for each individual in the federated learning evaluation process to 5. The value range of the participant participation ratio parameter C is set to 0.1 to 1 to ensure that a certain number of participants participate in the training, and the learning rate is 0.01 to 0.2. Too high a learning rate will affect convergence.

[0107] In the parameter setting of the neural network, the maximum number of hidden layers of MLP is 4, and the maximum number of neurons in each layer is 256. For CNN, the maximum number of convolutional layers is set to 3, the maximum number of kernel channels is 64, the maximum number of fully connected layers is set to 3, the maximum number of neurons in the convolutional layer is 256, and the convolution kernel size is 3 or 5. The maximum value of the network sparsity parameter is set to 128.

[0108] The final Pareto solutions obtained by MLP and CNN on IID and non-IID data using FNSGA-III and NSGA-III algorithms are as follows: Figure 5-Figure 8 As shown, each point represents a solution corresponding to a specific structural parameter in federated learning. The light-colored points represent the Pareto optimal solutions obtained under the FNSGA-III algorithm of the present invention, and the dark-colored points represent the Pareto optimal solutions obtained under the randomly initialized NSGA-III algorithm.

[0109] The number of Pareto solutions obtained by FNSGA-III is more stable than that obtained by NSGA-III. The number of Pareto solutions of NSGA-III under CNN is relatively small. For example, the number of Pareto solutions of NSGA-III under CNN IID is only 3. Figure 5-Figure 8The light-colored solutions are superior to the dark-colored solutions, that is, the Pareto solutions obtained by the present invention are superior to the Pareto solutions of the NSGA-III algorithm, except that a dark-colored solution is superior to the light-colored solution under CNN IID, but the number of solutions under CNN IID is very small. Moreover, the dominant effect of the FNSGA-III algorithm under IID is more obvious, and the distance between the solutions of FNSGA-III and NSGA-III is quite different, while the distance between the solutions of the two algorithms under non-IID is relatively small. At the same time, we found that FNSGA-III will converge more to the inflection point. The solution characteristics at the inflection point are that each target value is small and the solution quality is higher. The Pareto solutions of FNSGA-III are concentrated at the inflection point, making its uniformity under MLP inferior to the uniformity of the Pareto solutions of NSGA-III.

[0110] In addition, Table 2 gives the relevant evaluation index results of the Pareto solutions finally obtained by FNSGA-III and NSGA-III, and analyzes the Pareto solutions of the two algorithms from multiple dimensions.

[0111] The minimum value of a single objective reflects the extreme value of each objective function and reflects the algorithm's optimization ability. As can be seen from Table 2, the minimum global model test error rate, minimum variance, and minimum communication cost obtained by FNSGA-III under a single objective are basically smaller than those of NSGA-III. For example, in terms of the minimum global model test error rate, the value of FNSGA-III is smaller than that of NSGA-III, and the difference is obvious under CNN non-IID, with a difference of 4.89%.

[0112] The number of Pareto non-dominated solutions obtained by the FNSGA-III algorithm is stable, and except for MLP IID, the other numbers are greater than those of NSGA-III. The number of solutions obtained by the two algorithms under MLP is greater than that under CNN on average. The number of MLP solutions of NSGA-III is significantly greater than that of CNN, and the change is obvious. Therefore, the FNSGA-III algorithm is more robust than the NSGA-III algorithm in terms of the number of solutions.

[0113] The comprehensive index Hypervolume (HV) calculates the sum of the hypervolume of the hypercube composed of all non-dominated solutions and the reference point. It is a comprehensive index for evaluating Pareto solutions. Generally speaking, the larger the HV value, the better the quality of the Pareto solution evaluated. As shown in Table 2, the HV value of the FNSGA-III algorithm is always better than that of NSGA-III, showing better quality.

[0114] Table 2 Analysis of various indicators of FNSGA-III algorithm and NSGA-III algorithm

[0115]

[0116] The coverage rate C(A,B) calculates the proportion of solutions in solution set B that are dominated by at least one solution in A, and measures the degree of overlap between the two solution sets. The larger the index C is, the better the quality of solution set A is compared with the quality of solution set B. In Table 2, F in C(F,N) represents the solution set of FNSGA-III, and N represents the solution set of NSGA-III. The metric values ​​of C(F,N) are generally greater than C(N,F). Under CNN IID, C(F,N) = 50% is less than C(N,F) = 53%, but the difference between the two is very small. In general, from the perspective of coverage rate C, the solution of FNSGA-III is better than that of NSGA-III algorithm.

[0117] In terms of running time, CNN consumes more time than MLP, non-IID consumes more time than IID, and NSGA-III consumes more time than FNSGA-III, that is, from the statistical results, FNSGA-III has the best time performance. Based on the above analysis of the single objective minimum, the number of non-dominated solutions, the HV index, the C index and the time performance, we can conclude that the Pareto optimal solution obtained by the proposed FNSGA-III algorithm has better quality and is better than the NSGA-III algorithm solution.

[0118] In addition, we compared the proposed algorithm FNSGA-III with NSGA-II and SPEA2 evolutionary multi-objective algorithms, and evaluated the advantages and disadvantages of the Pareto optimal solution obtained by the algorithm from multiple aspects such as HV comprehensive index, number of Pareto solutions, coverage C, time, and single-objective optimal solution. In the above experiment, the MLPnon-IID with the closest Pareto obtained by FNSGA-III and NSGA-III was selected for comparative experiments on multiple evolutionary algorithms. The experimental results are shown in Tables 3 and Figure 6 shown.

[0119] Table 3 Analysis of various indicators of FNSGA-III algorithm and other evolutionary algorithms under MLP non-IID experiment

[0120]

[0121] A simple analysis of Table 3 shows that the Pareto solutions of FNSGA-III are more concentrated at the inflection point, and are superior to the Pareto solutions of NSGA-II and SPEA2, that is, each of the three objectives is better than the NSGA-II and SPEA2 algorithms. The coverage rate is compared with FNSGA-III. The HV value, number of Pareto solutions, coverage rate, and minimum value under the three objectives of FNSGA-III are better than the relevant indicators of NSGA-II and SPEA2. In terms of time, SPEA2 has the shortest running time, but the running time of FNSGA-III is not much different from that of SPEA2. In summary, the proposed FNSGA-III algorithm changes random initialization to fast greedy initialization, which can improve the running efficiency while basically outperforming the NSGA-III, NSGA-II, and SPEA2 evolutionary algorithms, and the obtained Pareto has higher quality.

[0122] Since the number of communication rounds in the federated learning evaluation process of FNSGA-III is very small, its federated learning performance cannot be fully explored. Due to the limitation of computing resources, we only select the MLP non-IID with the worst accuracy for enhancement experiments. Four solutions are selected for the Pareto optimal solution of MLP non-IID obtained by the FNSGA-III algorithm, of which two are solutions with very small global test error rate and the other two are inflection point solutions. These four solutions are trained for federated learning and compared with the standard FedAvg algorithm, with the number of communication rounds set to 150. In addition to increasing the number of communication rounds, each solution is verified under IID and non-IID to study whether the non-dominated solution obtained on the IID dataset is still valid on the non-IID dataset, and vice versa. All verification results are listed in Table 4.

[0123] Table 4 Experimental data of MLP non-IID solution obtained by FNSGA-III algorithm

[0124] parameter Solution 1 Solution 2 Solution 3 Solution 4 Standard FedAvg Participation ratio C 0.62 0.473 3 0.281 2 0.753 4 1 Learning rate η 0.112 5 0.114 4 0.085 0.094 9 0.05 SET parameter ε 95 31 31 3 / Number of neurons in the MLP hidden layer 173 136 197 208 [200,200] communication cost 57 553 14 330 9 163 3 972 199 210 Non-IID test accuracy / % 96.70 96.08 94.81 91.60 95.69 Variance of test accuracy distribution under non-IID 4.71 5.79 12.78 22.50 9.86 Test accuracy under IID / % 97.98 97.58 97.45 95.39 97.35 Variance of test accuracy distribution under IID 1.67 2.12 2.18 3.94 2.24

[0125] From the results shown in Table 4, the evolution of the four Pareto solutions of the selected MLP non-IID can be observed as follows. In the experimental results of the four solutions obtained under MLP non-IID under non-IID data distribution, the sparsity parameter of solution 4 is 3. Although its accuracy curve iteration is stable, it is obviously lower than other cases, which may mean that excessively sparse neural networks will damage the accuracy of the model; solutions 1 and 2 are better than the results of standard federated learning in terms of communication cost, accuracy and variance, and the iteration curves are stable, which verifies that there are high-quality solutions in the Pareto solutions obtained by the algorithm FNSGA-III proposed in the present invention. In the experimental results under IID data distribution, only the iteration curve of solution 4 is inferior to that of standard federated learning. It can be said that the solution under MLP non-IID is effective under non-IID, and it can still have a good operating effect when extended to IID.

[0126] The beneficial effects of the present invention are as follows:

[0127] The present invention proposes the FNSGA-III algorithm to solve the problem of multi-objective federated learning model, and conducts experimental verification under unstable communication. We first construct a three-objective template for federated learning, set the optimization goal to minimize the global model test error rate, communication cost, and global model accuracy distribution variance, and the decision variables are the hyperparameters of the neural network and the federated learning parameters. The NSGA-III algorithm is introduced to solve the federated learning multi-objective model, and the initialization of NSGA-III is changed. The experimental results show that the improved FNSGA-III algorithm is better than the original NSGA-III algorithm. And the Pareto optimal solution optimized by the FNSGA-III algorithm is compared with the benchmark federated average algorithm, which effectively improves the global model accuracy, reduces the global model accuracy distribution variance and communication cost.

[0128] The word "preferred" as used herein is intended to be used as an example, instance or illustration. Any aspect or design of the present invention described as "preferred" is not necessarily to be construed as being more advantageous than other aspects or designs. On the contrary, the use of the word "preferred" is intended to present concepts in a specific way. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X uses A or B" means any one of the naturally included arrangements. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the aforementioned examples.

[0129] Moreover, although the present disclosure has been shown and described with respect to one or implementations, those skilled in the art will think of equivalent variations and modifications based on the reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations, and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if the structure is not equivalent to the disclosed structure of the function in the exemplary implementation of the present disclosure shown in the present invention. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that may be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0130] The functional units in the embodiments of the present invention may be integrated into a processing module, or each unit may exist physically separately, or multiple or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc. The above-mentioned devices or systems may execute the storage method in the corresponding method embodiment.

[0131] To sum up, the above embodiment is an implementation mode of the present invention, but the implementation mode of the present invention is not limited by the embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A multi-objective federated learning evolutionary method based on the improved NSGA-III, applied to the server and multiple participants, characterized by: The following steps are involved: Acquire learning data, where the learning data is used for labeling; Constructing a federated learning multi-objective optimization model, wherein the multi-objective optimization model includes three objectives: maximizing global model accuracy, minimizing global model accuracy distribution variance, and communication cost; Fast Greedy Initialization P t =0, use FedAvg algorithm for multi-objective evaluation; P t Perform non-dominant sorting; Iterate, and each iteration performs the following operations: selection, crossover, and mutation operator operations to generate Q t , conduct federated learning training and evaluation on the offspring population, and calculate the three objectives of each individual; mix the parent and offspring populations R t =Q t +P t ; for R t Non-dominant sorting, and selection of P based on reference point t+1 ; Find the Pareto optimal solution and output the annotation results corresponding to the Pareto optimal solution; The process of training and evaluating the federated learning of the offspring population is implemented by the FedAvg algorithm based on static SET, which specifically includes the following steps: i is an individual in the population of the FNSGA-III algorithm, P is the population size, and after decoding individual i, the relevant federated learning neural network hyperparameters, the connectivity of the neural network, and the participation ratio C of participants in each round are obtained. i ; Use the communication parameter ε i Initialize a static SET topology and use the static SET topology as a global model used in the algorithm; In each round of training, the local data is trained using the mini-batch stochastic gradient descent method; After a certain number of rounds, the three objectives of the global model’s test error rate, global model accuracy distribution variance, and communication cost are calculated.

2. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 1, characterized in that: In the evolution of federated learning, the dataset D k The loss function of the kth participant is: The global goal of the federated learning evolutionary method is to minimize the following global loss function L(w): Among them, k is the serial number of the participant, L k (w) is the loss function of the kth participant, l i (w) is the loss function on data sample i, n k is the dataset D of participant k k Size n k =|D k |, n is the total data sample size of K participants.

3. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 1, characterized in that: In each round of training in the federated learning process, each participant receives the global model w from the server t , and use local data to train the global model to obtain an updated local model The participants then send the updated local models to the server, which aggregates the models according to certain rules to obtain a new global model w t+1 , used for the next round of iterative training, and the subscript t represents the communication round of federated learning.

4. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 1, characterized in that: The three-objective optimization model of federated learning evolution is: Among them, F(v) is the objective function of the three-objective optimization model. This model has three minimization objectives, namely minimizing the global model test error rate f1, the global model accuracy distribution variance f2, and the communication cost f3. Conv is the number of convolutional layers, kc is the number of convolutional kernels, ks is the size of the convolutional kernel, L is the number of fully connected layers, N is the number of neurons in the fully connected layer, η is the learning rate, and ε is the connectivity parameter of the neural network. The number of connections between two fully connected layers is determined by the connectivity parameter ε. The total number of connections is n=ε(n k +n k -1 ), where n k and n k-1 are the number of neurons in the k layer and k-1 layer respectively.

5. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 1, characterized in that: The global model test error rate of target f1 is E=1-A, where A is the average test accuracy of the global model {a1,a2,...,a K } is the accuracy of each participant.

6. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 5, characterized in that: Target f2 is the global model accuracy distribution variance 7. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 5, characterized in that: The target f3 can be expressed as K is the total number of participants, C is the proportion of participants participating in each round, and σ is the size of the model parameters.

8. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 1, characterized in that: The number of convolutional layers, number of convolutional kernels, convolutional kernel size, number of fully connected layers, number of neurons in each layer of fully connected layers, and SET parameter ε of MLP and CNN are encoded in binary format, while the learning rate η and the participation ratio C of each round are encoded in real value format.

9. The multi-objective federated learning evolutionary method based on improved NSGA-III according to claim 1, characterized in that: The learning data is the MNIST data set.

Images