Federated learning system, model learning device, federated learning method, model learning program

Abstract

This associative training system has a plurality of model training devices, each of which is connected to any one of other model training devices over a network. The model training device comprises a mini batch extraction unit, a gradient calculation unit, a dual variable limiting unit, a model parameter update unit, a dual variable calculation / transmission unit, a dual variable reception unit, and a dual variable setting unit. The model parameter update unit uses model parameters, limited dual variables, a training rate, a dual variable training rate, the degree of contribution, the gradient of a cost function, constraint parameters, and the number of other model training devices connected to the model training device to perform training, and updates the model parameters. The dual variable calculation / transmission unit uses, for each of the other model training devices connected to the model training device, the updated model parameters, the dual variable training rate, the limited dual variables, the degree of contribution, and the constraint parameters to calculate and output the dual variables.

Description

[Technical Field]

[0001] The present disclosure relates to a federated learning system having a plurality of model learning devices, each of which is connected to one of the other model learning devices via a network, a model learning device, a federated learning method, and a model learning program. [Background technology]

[0002] Utilizing emails, purchase histories, corporate research materials, IoT information, and hospital diagnostic information stored on personal devices promises many benefits, including matching, automatic control, and AI-based medical treatment. However, concerns about information security and leaks have prevented this technology from being fully utilized. Federated learning, which allows for distributed learning, is a well-known solution to this problem. For example, in Non-Patent Document 1, in order to solve the drift problem in distributed training of machine learning models, the authors restructure the update process of edge-consensus learning (ECL) to implicitly increase the gradient correction of stochastic variance reduction (SVR) by optimally selecting the constraint-strength control parameter.

[0003] In federated learning, if there is an attacker among the users who does not follow the correct learning protocol, problems can occur in which the accuracy of the model being learned does not improve or the wrong model is learned. To address this, methods have been proposed for federated learning that exclude attackers and that reduce the impact of attacks on the model being learned, in a basic federated learning method called FedAvg, which consists of a server and a client (see, for example, Non-Patent Document 2). Figure 1 shows Algorithm 2 and Equations 7, 8, 10, and 11 shown in Non-Patent Document 2. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Niwa, K., Zhang, G., Kleijn, WB, Harada, N., Sawada, H., Fujino, A.. (2021). Asynchronous Decentralized Optimization With Implicit Stochastic Variance Reduction. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:8195-8204. [Non-patent document 2] Liping Li, Wei Xu, Tianyi Chen, Georgios B. Giannakis, and Qing Ling. 2019. RSA: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI'19 / IAAI'19 / EAAI'19). AAAI Press, Article 190, 1544-1551. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method of Non-Patent Document 2 described above only supports basic network configurations consisting of a server and a client, and has the problem of not supporting complex network configurations where the number of other connected terminals varies from terminal to terminal. The present disclosure aims to provide a federated learning technology that can prevent degradation of learning accuracy not only in basic network configurations but also in other complex network configurations, even when attacked by an attacker. [Means for solving the problem]

[0006] The federated learning system disclosed herein includes multiple model learning devices, each connected to one of the other model learning devices via a network. The model parameters, dual variables, learning rate, learning rate of the dual variables, and contribution are set to predetermined initial values. Each model learning device includes a mini-batch extraction unit, a gradient calculation unit, a dual variable restriction unit, a model parameter update unit, a dual variable calculation and transmission unit, a dual variable reception unit, and a dual variable setting unit. The mini-batch extraction unit extracts a predetermined amount of data as a mini-batch from the model training data. The gradient calculation unit calculates the gradient of the cost function from the model parameters and the mini-batch of the model training data. The dual variable restriction unit restricts the magnitude of the dual variables based on a predetermined condition. The model parameter update unit performs learning using the model parameters, the restricted dual variables, the learning rate, the learning rate of the dual variables, the contribution, the gradient of the cost function, the constraint parameters, and the number of other model learning devices connected to the model learning device, and updates the model parameters. The dual variable calculation and transmission unit calculates and transmits dual variables for each of the other model learning devices connected to the model learning device using the updated model parameters, the learning rate of the dual variables, the restricted dual variables, the contributions, and the constraint parameters. The dual variable reception unit receives dual variables from the other model learning devices connected to the model learning device. The dual variable setting unit sets the received dual variables as dual variables to be used in the next learning. [Effects of the Invention]

[0007] According to the federated learning system of the present disclosure, the values ​​of the dual variables used when updating model parameters are restricted by the dual variable restriction unit 140. This reduces the impact on model learning even if an attacker transmits dual variables with large values. Therefore, it is possible to prevent degradation of learning accuracy due to attacks not only in basic network configurations but also in other complex network configurations. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing Algorithm 2 and Equations 7, 8, 10, and 11 shown in Non-Patent Document 2. [Figure 2] FIG. 2 is a diagram showing an example of the functional configuration of the federated learning system according to this embodiment. [Figure 3] FIG. 3 is a diagram showing an example in which model learning devices according to this embodiment are connected in a ring shape. [Figure 4] FIG. 4 is a diagram showing an example in which model learning devices according to this embodiment are connected randomly. [Figure 5] FIG. 5 is a diagram showing an example of a processing flow of the federated learning system according to this embodiment. [Figure 6] FIG. 6 is a diagram showing the algorithm of the federated learning system according to this embodiment in the same format as Non-Patent Document 2. [Figure 7] FIG. 7 is a diagram illustrating an example of the functional configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail. Note that components having the same functions are assigned the same numbers, and redundant explanations will be omitted.

[0010] FIG. 2 shows an example of the functional configuration of a federated learning system according to this embodiment. FIG. 3 shows an example of model learning devices according to this embodiment connected in a ring configuration, and FIG. 4 shows an example of model learning devices according to this embodiment connected randomly. FIG. 5 is a diagram showing an example of the processing flow of the federated learning system according to this embodiment. FIG. 6 is a diagram showing the algorithm of the federated learning system according to this embodiment in the same format as in Non-Patent Document 2. The federated learning system 10 of this embodiment comprises N model learning devices 1001, ..., 100 N Each model learning device 100 i Other model learning device 100 j Here, N is an integer of 2 or more, i is an integer of 1 or more and N or less, and j is an integer of 1 or more and N or less other than i. Each model learning device 100 i The system includes an initial setting unit 110, a mini-batch extraction unit 120, a gradient calculation unit 130, a dual variable restriction unit 140, a model parameter update unit 150, a dual variable calculation and transmission unit 160, a dual variable reception unit 170, and a dual variable setting unit 180. Note that Figs. 3 and 4 are merely examples, and the network configuration, the number of model learning devices N, and the number of model learning devices 100 may vary. i and the number of other model learning devices connected E i can be determined arbitrarily.

[0011] Each model learning device 1001, ..., 100 N In the initial setting section 110, the model parameters w i , the dual variable λ i|j , dual variable z i|j , the learning rate μ of the learning model, and the learning rate η of the dual variables i , contribution γ i are set to predetermined initial values ​​(S110). For example, w1,...,w N are set to the same value, and λ i|j =z i|j = 0. μ and γ i is set to an arbitrary value. As will be explained later, η i is 1 / (μE i K) and E iThe model learning device (100 i ) and other model learning devices (100 j ), and K is the number of iterations in the inner loop.

[0012] Each model learning device 1001, ..., 100 N The mini-batch extraction unit 120 extracts the model training data x i A predetermined amount of data is divided into mini-batches ξ i r,k For example, when there are 10,000 pieces of learning data, 500 or 1,000 mini-patches can be arbitrarily extracted. The i at the bottom right of ξ indicates the number of the model learning device. The k at the top right of ξ indicates the number of each model learning device 1001, ..., 100 N is an integer indicating the number of iterations of the iterative process (inner loop process) performed by ξ. The inner loop process is iterated K times. K is an integer greater than or equal to 2, and k is an integer greater than or equal to 1 and less than or equal to K. The r in the upper right corner of ξ is an integer indicating the number of iterations of the iterative process of the inner loop process (outer loop process) for the entire associative learning system. The outer loop process is iterated R times. R is an integer greater than or equal to 2, and r is an integer greater than or equal to 1 and less than or equal to R. In the following explanation, the i and j in the lower right corner of the symbol indicate the number of the model learning device, the r in the upper right corner of the symbol indicates the number of iterations in the outer loop process, and the k in the upper right corner indicates the number of iterations in the inner loop process. Note that in the text of the specification, the symbols in the lower right and the upper right cannot be written in the same horizontal position, so they are written offset from each other. On the other hand, in the formulas and figures in the specification, they are written in the same horizontal position because it is possible to write them.

[0013] Each model learning device 1001, ..., 100 N The gradient calculation unit 130 calculates the model parameters w i r,k and a mini-batch of data for model training, ξ i r,k From the gradient of the cost function g i (w i r,k) (step S130). Specifically, the gradient calculation unit 130 calculates the gradient g of the cost function of the learning model as follows: i (w i r,k ) is calculated, where f i is the cost function, and ∇ indicates the gradient. The gradient of the calculated cost function of the learning model, g i (w i r,k ) is the parameter w of the model described later. i The arrow in the formula indicates substitution.

number

[0014] Each model learning device 1001, ..., 100 N The dual variable restriction unit 140 restricts the magnitude of the dual variables based on a predetermined condition (step S140). Specifically, the dual variable restriction unit 140 restricts the magnitude of the dual variables ^z before the restriction process is performed by the dual variable restriction unit 140 (hereinafter also referred to as "before restriction") as follows: i|j r The dual variable z after the restriction process by the dual variable restriction unit 140 (hereinafter also referred to as "after restriction") is i|j r The symbol "^" (superscript hat) to the left of z indicates that it is before the restriction. Note that in the text of the specification, it cannot be written above z, so it is written shifted. On the other hand, in the formulas and figures in the specification, it can be written, so it is written above z.

[0015] If the outer loop process is r=0, the restricted dual variable z is obtained from the following equation: i|j r Calculate.

number

[0016] If the outer loop process is r>0, the restricted dual variable z is obtained from the following equation: i|j r Calculate.

number

[0017] Dual variable z when r>0 i|j r There are two methods for calculating the ^z before the restriction: "clipping" and "shrinking." i|j r This is a method of limiting to an arbitrary value (determining an upper limit and replacing it). i|j r and, y i|j r-1,k and ^z i|j r The element of y i|j r-1,k If it exceeds z i|j r The elements of i|j r-1,k The reduction is done by substituting the elements of the received dual variable ^z i|j r This is a method to reduce the value to an arbitrary value or less. i|j r The value of the L2 norm of y i|j r-1,k If the L2 norm of z exceeds the i|j r ^z i|j ry i|j r-1,k ||2 / ||z i|j r The dual variable calculation and transmission unit 160 uses either the clipping or reduction described above to limit the dual variable z i|j Limit the magnitude of the value of

[0018] The dual variable restriction unit 140 receives the unrestricted dual variable ^z i|j r y i|j r-1,k However, it can be realized by the following method. For example, the received unrestricted dual variable ^z i|j r is the value of the local sending variable (y i|j r-1,K-1 ) may be used to clip or reduce the received unconstrained dual variable ^z i|j r The value of the local sending variable (y i|j r,k-1 ) may be used for clipping or shrinking. Alternatively, the model learning device 100 i All model learning devices 100 with which the j The dual variable (y j|i r-1 ) may be clipped or scaled down using the mean or median value of

[0019] Each model learning device 1001, ..., 100 N The model parameter update unit 150 updates the model parameters w i r,k , the restricted dual variable z i|j r , learning rate μ, learning rate η of dual variables i , contribution γ, gradient of cost function g i (w ir,k ), constraint parameter A i|j , the model learning device (100 i ) and other model learning devices (100 j ) number E i Specifically, the model parameter update unit 150 updates the model parameters as shown in the following equation, and updates the model parameters w to be used in the next process (the (k+1)th process of the inner loop process). i r,k+1 Ask for.

number

[0020] In addition, the model parameter w i r,k , dual variable z i|j r , learning rate μ, learning rate η of dual variables i , constraint parameter A i|j , the model learning device (100 i ) and other model learning devices (100j ) number E i For details, please refer to the above-mentioned Non-Patent Document 1.

[0021] Each model learning device 1001, ..., 100 N The dual variable calculation and transmission unit 160 of the model learning device 100 i and other model learning devices 100 connected to it. j For each parameter w, the updated model i r,k+1 and the learning rate η of the dual variables i and the restricted dual variable z i|j r and contribution γ i and the constraint parameter A i|j Using the dual variable y i|j r,k+1 and other model learning device 100 j Specifically, the dual variable calculation and transmission unit 160 calculates the dual variable y i|j In addition, other model learning devices 100 j There must be at least one, but there may be two or more.

number

number

[0022] Each model learning device 1001, ..., 100 N The dual variable receiving unit 170 of the model learning device 100 i and other model learning devices 100 connected to it. j From the dual variable y j|i r,k+1 The dual variable y received by the dual variable receiving unit 170 is received (S170). j|i r,k+1The model learning device 100 j Since the dual variable calculation and transmission unit 160 calculates and transmits the dual variable y i|j r,k+1 The positions of "j" and "i" are reversed.

[0023] Each model learning device 1001, ..., 100 N The dual variable setting unit 180 sets the received dual variable y j|i r,k+1 The dual variable ^z used in the next training i|j r The dual variable setting unit 180 sets the received dual variable y j|i r,k+1 is used in the processing in step S140 in the next iteration of the same inner loop process. i|j r This can be done as follows.

number

[0024] Each model learning device 1001, ..., 100 N If the inner loop process has not been completed (if No), the process continues, and if the inner loop process has been completed (if Yes), the process proceeds to check the outer loop process (S190). If the result of step S190 is Yes, each model learning device 1001, ..., 100 N checks whether the processing of the outer loop step has ended, and if it has not ended (if No), continues the repeated processing, and if it has ended (if Yes), ends the processing (S195).

[0025] According to the federated learning system 10 of this embodiment, model learning devices 1001, . . . , 100 N is the dual variable z used when updating the model parameters in step S150. i|jThe value of is limited in advance by the dual variable limiting unit 140 based on predetermined conditions. Therefore, even if an attacker transmits a dual variable with a large value, the impact on model learning can be reduced. Therefore, even if an attacker attacks, not only in a basic network configuration consisting of a server and a client, but also in other complex network configurations such as those shown in FIG. 3 or FIG. 4, it is possible to prevent degradation of learning accuracy.

[0026] [Programs, recording media] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 7, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

[0027] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0028] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0029] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the received program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to the computer but has properties that define computer processing).

[0030] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware. [Explanation of symbols]

[0031] 10 Federated Learning Systems 100 Model Learning Devices 110 Initial setting section 120 Mini-batch extraction part 130 Gradient calculation unit 140 Dual variable restriction section 150 Model parameter update unit 160 Dual variable calculation and transmission unit 170 Dual variable receiver 180 Dual variable setting section 900 Network A i|j Constraint parameters C Clipping variable w i Model parameters x i Model training data z i|j Dual Variable α hyperparameter γ Contribution η learning rate for dual variables μ learning rate

Claims

1. A federated learning system having a plurality of model learning devices, each of which is connected to any of the other model learning devices via a network, The model parameters, dual variables, learning rate, learning rate of dual variables, and contribution are set to predetermined initial values. Each model learning device is a mini-batch extraction unit that extracts a predetermined amount of data as a mini-batch from the model training data; a gradient calculation unit that calculates the gradient of the cost function from the model parameters and mini-batches of model training data; a dual variable restriction unit that restricts the magnitude of the dual variable based on a predetermined condition; a model parameter update unit that performs learning using the model parameters, the restricted dual variables, the learning rate, the learning rate of the dual variables, the contribution, the gradient of the cost function, the constraint parameters, and the number of other model learning devices connected to the model learning device, and updates the model parameters; a dual variable calculation and transmission unit that calculates and transmits dual variables for each of the other model learning devices connected to the model learning device using the updated model parameters, the learning rate of the dual variables, the restricted dual variables, the contributions, and the constraint parameters; a dual variable receiving unit that receives dual variables from another model learning device connected to the model learning device; a dual variable setting unit that sets the received dual variables as dual variables to be used in the next learning; Equipped with The model parameter update unit [0013] where w i r,k+1 is a model parameter, r is the number of learning iterations in the entire federated learning system, k is the number of learning iterations in the model learning device, i and j are symbols indicating the model learning device, γ i is the contribution, η i is the learning rate of the dual variable, E i and ε i are both the number of other model learning devices connected to the model learning device, A i|j is a constraint parameter, and z i|j r is a dual variable Update the model parameters w i r,k+1 as follows: Federated learning system.

2. A model learning device that configures a federated learning system having a plurality of model learning devices, each of which is connected to any of the other model learning devices via a network, The model parameters, dual variables, learning rate, learning rate of dual variables, and contribution are set to predetermined initial values. a mini-batch extraction unit that extracts a predetermined amount of data as a mini-batch from the model training data; a gradient calculation unit that calculates the gradient of the cost function from the model parameters and mini-batches of model training data; a dual variable restriction unit that restricts the magnitude of the dual variable based on a predetermined condition; a model parameter update unit that performs learning using the model parameters, the restricted dual variables, the learning rate, the learning rate of the dual variables, the contribution, the gradient of the cost function, the constraint parameters, and the number of other model learning devices connected to the model learning device, and updates the model parameters; a dual variable calculation and transmission unit that calculates and transmits dual variables for each of the other model learning devices connected to the model learning device using the updated model parameters, the learning rate of the dual variables, the restricted dual variables, the contributions, and the constraint parameters; a dual variable receiving unit that receives dual variables from another model learning device connected to the model learning device; a dual variable setting unit that sets the received dual variables as dual variables to be used in the next learning; Equipped with The model parameter update unit [0014] where w i r,k+1 is a model parameter, r is the number of learning iterations in the entire federated learning system, k is the number of learning iterations in the model learning device, i and j are symbols indicating the model learning device, γ i is the contribution, η i is the learning rate of the dual variable, E i and ε i are both the number of other model learning devices connected to the model learning device, A i|j is a constraint parameter, and z i|j r is a dual variable Update the model parameters w i r,k+1 as follows: Model learning device.

3. 3. The model learning device according to claim 2, The dual variable restriction unit When r = 0 [Equation 9] When r>0 [Equation 10] However, ^z ｉ｜ｊ ｒ is the dual variable before restriction, C is a predetermined value, y ｉ｜ｊ ｒ－１，ｋ is the dual variable calculated by the dual variable calculation and transmission unit, As shown, the dual variable ^z ｉ｜ｊ ｒ Limit Model learning device.

4. 3. The model learning device according to claim 2, The dual variable calculation and transmission unit [0011] [0012] However, y ｉ｜ｊ ｒ，ｋ＋１ is the dual variable Calculate the dual variables as follows: Model learning device.

5. A federated learning method using a federated learning system having a plurality of model learning devices, each of which is connected to any of the other model learning devices via a network, comprising: The model parameters, dual variables, learning rate, learning rate of dual variables, and contribution are set to predetermined initial values. Each model learning device is a mini-batch extraction step of extracting a predetermined amount of data as a mini-batch from the model training data; a gradient calculation step of calculating the gradient of the cost function from the model parameters and mini-batches of model training data; a dual variable restriction step of restricting the magnitude of the dual variables based on a predetermined condition; a model parameter update step of performing learning using the model parameters, the restricted dual variables, the learning rate, the learning rate of the dual variables, the contribution, the gradient of the cost function, the constraint parameters, and the number of other model learning devices connected to the model learning device, and updating the model parameters; a dual variable calculation and transmission step of calculating and transmitting dual variables for each of the other model learning devices connected to the model learning device using the updated model parameters, the learning rates of the dual variables, the restricted dual variables, the contributions, and the constraint parameters; a dual variable receiving step of receiving dual variables from another model learning device connected to the model learning device; a dual variable setting step of setting the received dual variables as dual variables to be used in the next learning; an inner loop process for executing the above a first predetermined number of times; an outer loop step of executing the inner loop step a second predetermined number of times; The model parameter updating step includes: [Equation 15] where w i r,k+1 is a model parameter, r is the number of learning iterations in the entire federated learning system, k is the number of learning iterations in the model learning device, i and j are symbols indicating the model learning device, γ i is the contribution, η i is the learning rate of the dual variable, E i and ε i are both the number of other model learning devices connected to the model learning device, A i|j is a constraint parameter, and z i|j r is a dual variable, updating the model parameter w i r,k+1. Associative learning methods.

6. A model learning program that causes a computer to function as the model learning device according to any one of claims 2 to 4.

Images