Federated learning method for reducing communication overhead

Abstract

A federated learning method for reducing communication overhead is disclosed. The disclosed method is a federated learning method performed by a federated learning server of a federated learning system including a plurality of user terminals and the federated learning server, the method comprising the steps of: (a) allocating the plurality of user terminals to a plurality of subgroups; (b) receiving an encrypted local sign gradient calculated by the user terminal belonging to each subgroup; (c) summing the local sign gradients encrypted for each subgroup so as to calculate a sub-global sign gradient of each subgroup; and (d) summing the sub-global sign gradients of respective subgroups and sign-encoding the summed sub-global sign gradients so as to calculate a global sign gradient, and broadcasting the calculated global sign gradient to the plurality of user terminals, wherein the local sign gradient in step (b) is encrypted using an encryption polynomial, a plurality of sub-rounds are performed for encryption of higher-order terms, and the order of the encryption polynomial and the number of the sub-rounds are determined on the basis of the number of user terminals allocated to the subgroups. According to the disclosed method, communication overhead can be minimized without exposing local gradient information of a local user to a server.

Description

Federated learning method for reducing communication overhead

[0001] The present invention relates to a federated learning method, and more specifically, to a federated learning method for reducing communication overhead.

[0002]

[0003] Federated learning is a method in which each user maintains their own data locally and trains a global model provided by a server by transmitting model information updated by the user with their local data. It is known to guarantee the privacy of local users. However, privacy can be threatened as model parameter information updated by local users can be exposed through model inversion attacks. Furthermore, methods that transmit gradient information instead of model parameters, as well as the sign gradient method that transmits only the sign of the gradient, pose a risk of exposing the local user's actual data through gradient inversion attacks. To address these issues, techniques such as masking, differential privacy (DP), and homomorphic encryption (HE) have been proposed.

[0004] However, existing techniques have the disadvantage of generating very large communication overhead. In particular, in the case of HE, thousands of bits of additional overhead are incurred to transmit a single bit. Furthermore, masking and DP techniques still carry the risk of privacy exposure. Therefore, a method is needed that can guarantee the privacy of local users even with low traffic volume.

[0005]

[0006] The present invention proposes a federated learning method that can minimize communication overhead without exposing local user's local gradient information to the server.

[0007]

[0008] According to one aspect of the present invention, a federated learning method performed on a federated learning server of a federated learning system comprising a plurality of user terminals and a federated learning server is provided, comprising the steps of: assigning a plurality of user terminals to a plurality of subgroups (a); receiving an encrypted local sign gradient calculated by a user terminal belonging to each subgroup (b); summing the encrypted local sign gradients for each subgroup to calculate a sub-global sign gradient for each subgroup (c); summing the sub-global sign gradients of each subgroup, sine-coding the summed sub-global sign gradient to calculate a global sign gradient, and broadcasting the calculated global sign gradient to the plurality of user terminals (d), wherein the local sign gradient of step (b) is encrypted using an encryption polynomial, a plurality of sub-rounds are performed for encryption of higher-order terms, and the degree of the encryption polynomial and the number of sub-rounds are determined based on the number of user terminals assigned to the subgroups.

[0009] A federated learning method is provided, wherein step (a) comprises: determining a minimum number of users n to belong to a subgroup; determining the total number of subgroups by calculating [N / n] (where N is the total number of user terminals); determining user terminals belonging to each subgroup; and determining a modular constant p to be applied to the cryptographic polynomial of the subgroup, wherein the modular constant p is a minimum prime number greater than the number of user terminals assigned to each subgroup.

[0010] If N is not divisible by n, the difference in the number of user terminals between subgroups is (here, p m is the largest modular constant among all subgroups, and p sUser terminals are assigned to each subgroup so that it becomes the smallest modular constant among all subgroups' modular constants.

[0011] The above cryptographic polynomial is as follows:

[0012]

[0013] In the above mathematical formula, and, x i is the local gradient of each user terminal, i is an index representing the user terminal, and m i is a variable representing all values ​​that each user terminal can have ( ), n is the number of user terminals for each subgroup, and p is a modular constant.

[0014] The above multiple user terminals receive and store encrypted variables from an encryption server before the start of collaborative learning or during idle time.

[0015] The above user terminals perform sub-rounds based on the degree of the cryptographic polynomial for each subgroup to encrypt the local sign gradient using the above cryptographic polynomial.

[0016] The above global sine gradient is calculated as follows:

[0017]

[0018] In the mathematical formula above, F(x j ) is the subglobal sign gradient, and j is the index representing the subgroup.

[0019] According to another aspect of the present invention, a federated learning method performed at a user terminal of a federated learning system comprising a plurality of user terminals and a federated learning server is provided, comprising the steps of: (a) receiving information on an assigned subgroup and information on the number of users of the assigned subgroup; (b) calculating an encrypted local sign gradient based on the number of users of the assigned subgroup; (c) transmitting the encrypted local sign gradient to the federated learning server; and (d) receiving a global sign gradient from the federated learning server, wherein the federated learning server calculates a sub-global sign gradient of each subgroup by summing the encrypted local sign gradients for each subgroup, and calculates a global sign gradient by summing the sub-global sign gradients of each subgroup and sine-coding the summed sub-global sign gradients, and wherein step (b) calculates the encrypted local sign gradient using an encryption polynomial, and a plurality of sub-rounds are performed for encryption of higher-order terms, and the degree of the encryption polynomial and the number of sub-rounds are determined based on the number of user terminals assigned to the subgroup.

[0020] According to another aspect of the present invention, a federated learning method is provided comprising the steps of: receiving encrypted local sign gradients from a plurality of user terminals, wherein each of the plurality of user terminals calculates a local sign gradient and encrypts the local sign gradient using a cryptographic polynomial; (a) calculating a global update value by summing all the encrypted local sign gradients received from the user terminals; and (b) broadcasting the calculated global update value to the plurality of user terminals, wherein the cryptographic polynomial is as follows:

[0021]

[0022] In the above mathematical formula, and, x i is the local gradient of each user terminal, i is an index representing the user terminal, and m i is a variable representing all values ​​that each user terminal can have ( ), n is the number of user terminals for each subgroup, and p is a modular constant.

[0023] According to another aspect of the present invention, a beaver triple variable (a) from a beaver triple server i , b i , c i A step of receiving and storing ) (a) - where i is an index representing the user terminal -; a step of computing a local sine gradient by quantizing the local gradient obtained through local learning (b); and a higher-order value (x) of a preset cryptographic polynomial through a plurality of sub-round processes. k Step (c) of computing , where x is the local sign gradient; and higher-order value (x) of the computed cryptographic polynomial. k A federated learning method is provided, comprising the steps of: (d) encrypting the local sign gradient using the cryptographic polynomial; (e) transmitting the local sign gradient encrypted in step (d) to the federated learning server; and (f) receiving a global update value from the federated learning server and performing local learning.

[0024]

[0025] According to the present invention, communication overhead can be minimized without exposing the local user's local gradient information to the server, and there is an advantage of privacy protection.

[0026]

[0027] FIG. 1 is a diagram showing a system structure to which a federated learning method for reducing communication volume according to one embodiment of the present invention is applied.

[0028] FIG. 2 is a flowchart showing the overall flow of an associative learning method according to one embodiment of the present invention.

[0029] FIG. 3 is a flowchart illustrating a method for subgrouping user terminals participating in federated learning according to an embodiment of the present invention.

[0030] FIG. 4 is a diagram showing a structure that generates a beaver triple variable and transmits it to a user terminal according to an embodiment of the present invention.

[0031] FIG. 5 is a diagram showing an example of a beaver triple variable transmitted to each user terminal according to an embodiment of the present invention.

[0032] FIG. 6 is a diagram illustrating a sub-round procedure performed in a subgroup according to an embodiment of the present invention.

[0033]

[0034] Hereinafter, specific embodiments according to embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, this is merely illustrative and the present invention is not limited thereto.

[0035] In describing the embodiments of the present invention, if it is determined that a detailed description of known technology related to the present invention may unnecessarily obscure the essence of the embodiments, such detailed description will be omitted. Furthermore, the terms described below are defined in consideration of their functions in the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe specific embodiments and should not be limiting. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as “include” or “comprising” are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.

[0036] FIG. 1 is a diagram showing a system structure to which a federated learning method for reducing communication volume according to one embodiment of the present invention is applied.

[0037] Referring to FIG. 1, a system for a federated learning method according to one embodiment of the present invention includes a federated learning server (100), an encryption server (200), and a plurality of user terminals divided into a plurality of subgroups.

[0038] Federated learning is performed by transmitting local learning results from multiple user terminals (300) to a federated learning server (100), receiving local learning data from the federated learning server (100), performing a global update, and then transmitting the global update results (global sign gradient) to each user terminal (300).

[0039] In an embodiment of the present invention, it is assumed that each user terminal (300) transmits the learning result as a local sign gradient. That is, each user terminal (300) transmits a local sign gradient of -1 (or 0) or 1 to a federated learning server (100) as the learning result of performing local learning.

[0040] Since the user terminal (300) transmits a local sign gradient of 1 or -1, the global update of the federated learning server (100) becomes a majority operation. That is, in a situation where there are user terminals that transmit 1 as the local sign gradient and user terminals that transmit -1, the global sign gradient for the global update is calculated by determining which sign code (1 or -1) is transmitted more frequently.

[0041] For example, if there are many user terminals that have sent 1 as a local sign gradient, the federated learning server (100) broadcasts 1 as a global sign gradient to all user terminals. Conversely, if there are many user terminals that have sent -1 as a sign gradient, the federated learning server (100) sends -1 as a global sign gradient to all user terminals.

[0042] As discussed in the background technology section, conventional federated learning methods may infringe on the privacy of user terminals because the local learning results (local sign gradients) of the user terminals are exposed to the federated learning server. While local sign gradients can be encrypted to address this issue, such encryption requires significant computing resources and incurs communication overhead.

[0043] To solve such problems, the present invention proposes a method to reduce communication overhead that occurs when encrypting the sign gradient of each user terminal and transmitting it to a federated learning server.

[0044] In addition, since the user terminal encrypts the local sign gradient and transmits it to the federated learning server (100), the federated learning server (100) performs a majority operation (sum operation) on the original data (original local sign gradient) that has been changed, so the accuracy of the result may not be guaranteed.

[0045] An embodiment of the present invention also proposes an encryption method in which, even if the sign gradient is encrypted and transmitted from a user terminal (300), the same result as the original sign gradient being summed and sine encoded can be obtained when the summation operation is performed on the combined learning server.

[0046] In this embodiment, each user terminal (300) transmits an encrypted local sign gradient based on the cryptographic polynomial proposed in this embodiment, and the federated learning server (100) calculates a global sign gradient by performing a majority operation (sum operation) on the encrypted local sign gradient and transmits the global sign gradient to the user terminal (300).

[0047] According to the present invention, the federated learning server cannot verify the local sign gradient of individual user terminals but can calculate the accurate global sign gradient, so federated learning is performed while protecting the privacy of the user terminal (300).

[0048] The cryptographic polynomial for encrypting the local sign gradient of each user terminal is shared with all user terminals in advance before the start of federated learning.

[0049] However, cryptographic polynomials alone cannot perfectly guarantee the protection of the privacy of the user terminal. If there is a possibility that the cryptographic polynomials may be exposed to the federated learning server due to communication between the federated learning server (100) and a specific malicious user terminal, and if the cryptographic polynomials are exposed to the federated learning server (100), the privacy of the user terminal cannot be guaranteed.

[0050] The present invention performs encryption of the local sign gradient of a user terminal using an encryption server (200). The encryption server (200) is isolated from the federated learning server (100) and generates an encryption variable and transmits it to each user terminal (300). The encryption server (200) may transmit the encryption variable during an idle period or before the start of federated learning, and each user terminal performs encryption of the local sign gradient using the encryption variable provided by the encryption server (200).

[0051] According to one embodiment of the present invention, encryption methods such as Beaver Triple, DN Multiplication, and ATLAS may be used for the encryption of local sign gradients. To encrypt an encryption polynomial, higher-order terms of the polynomial must be computed, and the computation of higher-order terms is performed by conducting multiple sub-rounds. The degree of the encryption polynomial is based on the number of users participating in federated learning. As the number of users participating in federated learning increases, the degree of the encryption polynomial increases, and multiple sub-rounds must be conducted to compute the encrypted value of the higher-order terms. For example, for the computation of the encryption polynomial, x 2 , x 4 , x 6 The value of a higher-order term must be calculated, and multiple sub-rounds must be performed based on the higher-order term to calculate the value of the higher-order term. The higher the degree of the term, the more sub-rounds must be performed. This increase in sub-rounds not only increases the amount of communication but also causes the problem of increasing the time required for encryption.

[0052] To minimize the number of sub-rounds for encrypting local sign gradients, the present invention performs federated learning by dividing users into multiple subgroups. Referring to FIG. 1, an example of division into multiple subgroups (L) is illustrated, and the operation of encrypted local gradients and the operation of sub-global gradients for each subgroup are performed independently for each subgroup. Since the users participating in federated learning are divided into multiple subgroups, the number of users in each subgroup is smaller than the total number of users, and accordingly, the subgroups proceed with fewer sub-rounds. Since the sub-global gradients for each subgroup are calculated with a smaller number of sub-rounds for each subgroup, a reduction in communication volume and encryption operation time can be achieved.

[0053] The federated learning server (100) calculates the sub-global gradient for each subgroup and then calculates the global gradient by summing the sub-global gradients.

[0054] The method for dividing subgroups is explained in detail with reference to a separate drawing.

[0055] FIG. 2 is a flowchart showing the overall flow of an associative learning method according to one embodiment of the present invention.

[0056] Referring to FIG. 2, user terminals participating in federated learning are grouped into multiple subgroups (step 200). As previously explained, the user terminals are divided into multiple subgroups to reduce the number of sub-rounds for higher-order term encryption.

[0057] FIG. 3 is a flowchart illustrating a method for subgrouping user terminals participating in federated learning according to an embodiment of the present invention.

[0058] Let N be the total number of user terminals participating in federated learning, and determine n, which is the minimum number of user terminals to belong to each subgroup (step 300). The minimum number of user terminals is pre-set. According to one embodiment of the present invention, it is preferable that the minimum number of user terminals belonging to each subgroup be greater than 3.

[0059] Once the minimum number of user terminals to be included in a group is determined, the total number of subgroups is calculated (step 310). The total number of subgroups is calculated based on the minimum number of user terminals to be included in a subgroup that is pre-set, and specifically, the integer value of the value obtained by dividing the total number of user terminals (N) by the minimum number of user terminals is calculated as the total number of subgroups.

[0060] Specifically, [N / n] can be calculated as the total number of subgroups. For example, if the total number of users N=16 and the minimum number of users to belong to a subgroup is 3, the total number of subgroups is calculated as 5.

[0061] When the total number of subgroups is calculated, the user terminals to be assigned to each subgroup are determined (step 320). If the total number of users is divided by a preset minimum number of users, an equal number of user terminals are assigned to each subgroup. If the total number of user terminals is not divided by a preset minimum number of user terminals, the user terminals are assigned to each subgroup as evenly as possible.

[0062] When assigning user terminals to subgroups, the terminals are assigned based on a prime number p corresponding to the number of user terminals in each subgroup. Here, the prime number p corresponding to the number of user terminals in a subgroup is the largest prime number among those larger than the number of user terminals in the subgroup. For example, if the number of users assigned to a specific subgroup is 4, p becomes 5. If the total number of user terminals is not divisible by a preset minimum number of user terminals, the number of users in each group will inevitably differ; in this case, the difference between the group with the smallest number of user terminals (m) and the group with the largest number of user terminals is set as α. Here, α is (here, p m is the largest modular constant among all subgroups, and p s is selected as one of the values ​​satisfying (meaning the smallest modular constant among all subgroups).

[0063] Once the user terminals assigned to each subgroup are determined, a modular constant p to be applied to the cryptographic polynomial is determined (step 330). The modular constant p is determined as the largest prime number among the prime numbers corresponding to the number of user terminals in all subgroups. For example, the modular constant p is It can be determined as such, and the subscript is an index representing a subgroup.

[0064] Referring again to FIG. 2, after the subgrouping of user terminals is completed, the local sign gradient encrypted for each user terminal is calculated for each subgroup (step 210).

[0065] According to one embodiment of the present invention, an encrypted local sign gradient is computed using a preset cryptographic polynomial.

[0066] The present invention proposes a cryptographic polynomial, and the cryptographic polynomial proposed in the present invention is determined as shown in the following mathematical formula 1 by n, which is the number of user terminals belonging to a subgroup, and p (a minimum prime number greater than n) determined in each subgroup during the subgrouping process.

[0067]

[0068] In the above mathematical formula, and, x i is the local gradient of each user terminal, i is an index representing the user terminal, and m i is a variable representing all values ​​that each user terminal can have ( ), n is the number of user terminals for each subgroup, and p is the minimum value among prime numbers greater than n.

[0069] The encryption polynomial can be set differently depending on the setting value of sign(0). For example, if the number of user terminals n=4, p is set to 5, which is the smallest prime number greater than n. If sign(0) is set to -1, the encryption polynomial is set as shown in Equation 2 below.

[0070]

[0071] When sign(0) is set to 0 under the same conditions, the cryptographic polynomial is set as shown in the following mathematical formula 3.

[0072]

[0073] If sign(0) is set to -1 (or 1), the highest degree of the cryptographic polynomial increases. This may increase the number of sub-rounds, which will be explained later. However, the global update value transmitted from the server to the user terminal can be sent with only 1 bit. If sign(0) is set to 0, the highest degree decreases, which may reduce the number of sub-rounds, but the global update value transmitted from the server to the user terminal increases to 2 bits. Ultimately, since the setting of sign(0) involves a trade-off, the designer can make an appropriate choice depending on the environment.

[0074] In this embodiment, the encrypted local sign gradient of the terminal of subgroup j Defined as, where x represents the sine gradient and F() represents the cryptographic polynomial, i is an index representing the terminal, and t is an index representing the number of main rounds of federated learning.

[0075] encrypted local sign gradient Separate encryption is required when performing the calculation. This is because if the cryptographic polynomial is exposed to the federated learning server, the privacy of the local sign gradient of each user terminal cannot be guaranteed.

[0076] As previously discussed, encryption methods such as Beaver Triple, DN Multiplication, and ATLAS can be used to encrypt local sign gradients. Cryptographic polynomials consist of multiple high-order terms, and the values ​​of these high-order terms must be computed to obtain the final local sign gradient. It becomes possible to compute. For example, the cryptographic polynomial In the case of x 4 , x 3Higher-order values ​​of x (sine gradient) such as must be calculated, and sub-rounds are performed to calculate the encrypted higher-order values. To reduce the number of such sub-rounds, the present invention performs sub-grouping.

[0077] The following describes an example of encrypting a local sign gradient using a beaver triple.

[0078] FIG. 4 is a diagram showing a structure that generates a beaver triple variable and transmits it to a user terminal according to an embodiment of the present invention.

[0079] Referring to Fig. 4, the Beaver Triple server (server 2) sends the Beaver Triple variable a to each user terminal i , b i , c i It generates and transmits. As previously explained, the Beaver Triple Server is disconnected from the Federated Learning Server. For each user terminal, the Beaver Triple Server uses a for each sub-round. i , b i , c i Creates and transmits.

[0080] Beaver triple variables a, b, and c transmitted from the beaver triple server are generated to satisfy the conditions of the following mathematical formula 5.

[0081]

[0082] Here, and a, b, c are F p It is randomly selected to satisfy the above conditions. When the beaver triple is applied, decryption is possible by combining the shares (encrypted sign gradient in the present invention) of all user terminals (n).

[0083] FIG. 5 is a diagram showing an example of a beaver triple variable transmitted to each user terminal according to an embodiment of the present invention.

[0084] In FIG. 5, t represents the number of sub-rounds, and the beaver triple server (200) transmits beaver triple variables a, b, and c to be used for each sub-round and each user terminal, and FIG. 5 illustrates an example of beaver triple variables a, b, and c assigned to each terminal. The values ​​of a, b, and c shown in FIG. 5 are examples, and it will be obvious to those skilled in the art that various values ​​can be assigned while satisfying the conditions of Equation 4.

[0085] Ultimately, the Beaver triple variable is It is defined as such, where t is an index representing a sub-round and i is an index representing a user terminal.

[0086] The user terminal performs the sub-round process by repeating the process of sending to the server the value obtained by subtracting a and the value obtained by subtracting b from the value based on the sine gradient, using the a and b values ​​assigned in each sub-round.

[0087] FIG. 6 is a diagram showing a sub-round procedure performed in a subgroup according to one embodiment of the present invention.

[0088] Referring to Fig. 6, first, Sub-Round 0 is executed for each sub-group, and in Sub-Round 0, the user terminal of each sub-group [x j ] i -[a j ] i and [x j ] i -[b j ] i Calculate and send to the federated learning server. Here, i is an index representing the user terminal, and j is an index representing the subgroup.

[0089] The federated learning server [x transmitted by each user terminal j ] i -[a j ] i The sum of all values ​​for each subgroup (x j -a j) and [x1] i -[b1] i The sum of all (x j -b j It computes ). In sub-round 1, the federated learning server (x j -a j ) and (x j -a j ) is transmitted to the user terminals of each subgroup (j). The user terminals of each subgroup receive (x) from the federated learning server in sub-round 1. j -a j ) and (x j -b j Using ) [x j 2 ] i It can perform operations.

[0090] [x 2 ] i It is calculated as shown in the following mathematical formula 5.

[0091]

[0092] In Sub-Round 1, the user terminals of each sub-group [x j 2 ] i It can perform operations up to, and [x j 2 ] i Higher orders are provided by the federated learning server in sub-round 1 (x j -a j ) and (x j -b j It cannot be operated on with ) alone, and [x j 2 ] i Higher order [x k ] i To compute (k>2), additional sub-rounds must be performed.

[0093] In sub-round 2, each user terminal is [x j 2 ] i -[a] i and [x j2 ] i -[b] i Calculate and the calculated [x j 2 ] i -[a j ] i and [x j 2 ] i -[b j ] i It sends to the federated learning server. The federated learning server receives the [x j 2 ] i -[a j ] i The sum of (x j 2 -a j ) and received [x j 2 ] i -[b j ] i The sum of (x j 2 -b j The federated learning server computes the computed (x j 2 -a j ) and (x j 2 -b j Broadcasts ) to user terminals of each subgroup.

[0094] (x j 2 -a j ) and (x j 2 -b j The user terminal of each subgroup that received ) receives the received (x j 2 -a j ) and (x j 2 -b j Using ) [x j 4 ] i or [x j 3 ] i can be calculated. If, x 4If a higher degree is required for the cryptographic polynomial operation, perform the next sub-round.

[0095] Subsequent sub-rounds proceed in the same manner. Each user terminal uses the information received from the server in the previous sub-round (k-1) in sub-round k to [x j k ] i -[a j ] i and [x j k ] i -[b j ] i Sends to the federated learning server and the federated learning server (x j k -a) and (x j k Broadcast -b) to the user terminals of each subgroup.

[0096] [x that each user terminal calculates in each sub-round j mk ] i It can be generally defined as shown in the following mathematical formula 6.

[0097]

[0098] Here, is a constant value independent of user terminal i. Accordingly, Only one pre-designated user terminal performs the operation, and the remaining terminals It is desirable to calculate only and send it to the federated learning server.

[0099] In mathematical equation 6, it is expressed as mk because the values ​​of m and k can be flexibly set according to the required order of the encryption function.

[0100] Referring again to FIG. 2, through a sub-round, the user terminal of each sub-group transmits the calculated local sign gradient to the federated learning server, and the federated learning server receives the local sign gradient for each sub-group (step 220).

[0101] The federated learning server calculates the subglobal sign gradient for each subgroup (step 230). The federated learning server calculates the subglobal sign gradient by summing the local sign gradients transmitted by the user terminals of each subgroup. In this embodiment, the subglobal sign gradient is [F(x j )] i Defined as such, and when the sub-global sign gradient is computed, the sub-global gradients of each sub-group are summed, the summed sub-global gradients are sine-coded to compute the global sign gradient, and the computed global sign gradient is broadcast to the user terminals of each sub-group (step 240).

[0102] According to one embodiment of the present invention, the global sine gradient calculated based on the sub-global sine gradient is defined as shown in the following mathematical formula 7.

[0103]

[0104] The method of the present invention described above may be performed in a computing device including a processor and memory.

[0105] The present invention has been described with reference to embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the technical spirit of the appended claims.

Claims

1. A federated learning method performed on a federated learning server of a federated learning system comprising multiple user terminals and a federated learning server, wherein A step of assigning multiple user terminals to multiple subgroups (a); Step (b) of receiving an encrypted local sign gradient calculated by a user terminal belonging to each subgroup; A step (c) of calculating the sub-global sign gradient of each sub-group by summing the encrypted local sign gradients for each sub-group; The method includes the step (d) of summing the sub-global sign gradients of each of the above sub-groups, sine-coding the summed sub-global sign gradients to calculate the global sign gradient, and broadcasting the calculated global sign gradient to the plurality of user terminals. The local sign gradient of step (b) above is encrypted using a cryptographic polynomial, and multiple sub-rounds are performed for the encryption of higher-order terms, and A federated learning method in which the degree of the above cryptographic polynomial and the number of the above sub-rounds are determined based on the number of user terminals assigned to the above sub-group.

2. In Paragraph 1, The above step (a) is a step of determining the minimum number of users n to belong to a subgroup; A step of determining the total number of subgroups by calculating [N / n] (where N is the total number of user terminals); A step of determining user terminals belonging to each subgroup; The step of determining a modular constant p to be applied to the cryptographic polynomial of the above subgroup, comprising: A federated learning method in which the above modular constant p is a minimum prime number greater than the number of user terminals assigned to each subgroup.

3. In Paragraph 2, If N is not divisible by n, the difference in the number of user terminals between subgroups is (here, p m is the largest modular constant among all subgroups, and p s A federated learning method in which user terminals are assigned to each subgroup such that (means the smallest modular constant among all subgroups).

4. In Paragraph 3, The above cryptographic polynomial is a federated learning method such as the following mathematical formula. In the above mathematical formula, and x i is the local gradient of each user terminal, i is an index representing the user terminal, and m i is a variable representing all values ​​that each user terminal can have ( ), n is the number of user terminals for each subgroup, and p is a modular constant.

5. In Paragraph 4, A federated learning method for privacy protection in which the above-mentioned plurality of user terminals receive and store encrypted variables from an encryption server before the start of federated learning or during idle time.

6. In Paragraph 5, A federated learning method in which the above user terminals perform encryption by performing sub-rounds based on the degree of the cryptographic polynomial for each subgroup to encrypt the local sign gradient using the above cryptographic polynomial.

7. In Paragraph 1, The above global sine gradient is a federated learning method calculated as follows: In the mathematical formula above, F(x j ) is the subglobal sign gradient, and j is an index representing the subgroup.

8. A federated learning method performed on a user terminal of a federated learning system comprising a plurality of user terminals and a federated learning server, wherein Step (a) in which a user terminal receives information about an assigned subgroup and information about the number of users in the assigned subgroup; Step (b) of calculating an encrypted local sign gradient based on the number of users in the assigned subgroup; Step (c) of transmitting the above encrypted local sign gradient to the federated learning server; and The method includes the step (d) of receiving a global sign gradient from the federated learning server, wherein The federated learning server calculates the sub-global sign gradient of each sub-group by summing the encrypted local sign gradients for each sub-group, and calculates the global sign gradient by summing the sub-global sign gradients of each sub-group and sine-coding the summed sub-global sign gradient. The above step (b) computes the encrypted local sign gradient using an encryption polynomial, and multiple sub-rounds are performed for encryption of higher-order terms, and A federated learning method in which the degree of the above cryptographic polynomial and the number of the above sub-rounds are determined based on the number of user terminals assigned to the above sub-group.

9. In Paragraph 8, For the allocation of the above subgroups, the federated learning server Step of determining the minimum number of users n to belong to a group; A step of determining the total number of subgroups by calculating [N / n] (where N is the total number of user terminals); A step of determining user terminals belonging to each subgroup; Perform the step of determining a modular constant p to be applied to the cryptographic polynomial of the above subgroup, A federated learning method in which the above modular constant p is a minimum prime number greater than the number of user terminals assigned to each subgroup.

10. In Paragraph 10, If N is not divisible by n, the difference in the number of user terminals between subgroups is (here, p m is the largest modular constant among all subgroups, and p s A federated learning method in which user terminals are assigned to each subgroup such that (means the smallest modular constant among all subgroups).

11. In Paragraph 10, The above cryptographic polynomial is a federated learning method such as the following mathematical formula. In the above mathematical formula, and x i is the local gradient of each user terminal, i is an index representing the user terminal, and m i is a variable representing all values ​​that each user terminal can have ( ), n is the number of user terminals for each subgroup, and p is a modular constant.

12. In Paragraph 8, A federated learning method that further includes the step of receiving and storing encrypted variables from an encryption server before the start of federated learning or during idle time.

13. In Paragraph 12, The above step (b) is a federated learning method that performs encryption by performing sub-rounds based on the degree of the cryptographic polynomial for each subgroup to encrypt the local sign gradient using the cryptographic polynomial.

14. In Paragraph 13, The above global sine gradient is a federated learning method calculated as follows: In the mathematical formula above, F(x j ) is the subglobal gradient, and j is the index representing the subgroup.

15. A federated learning method performed on a federated learning server of a federated learning system comprising multiple user terminals and a federated learning server, wherein A step of receiving encrypted local sign gradients from multiple user terminals, wherein each of the multiple user terminals calculates a local sign gradient and encrypts the local sign gradient using a cryptographic polynomial (a); Step (b) of calculating a global update value by summing all encrypted local sign gradients received from the above user terminals; The method includes the step of broadcasting the calculated global update value to the plurality of user terminals, The above cryptographic polynomial is a federated learning method such as the following mathematical formula. In the above mathematical formula, and x i is the local gradient of each user terminal, i is an index representing the user terminal, and m i is a variable representing all values ​​that each user terminal can have ( ), n is the number of user terminals for each subgroup, and p is a modular constant.

16. A federated learning method performed on a user terminal of a federated learning system comprising a plurality of user terminals and a federated learning server, wherein Beaver Triple variable from Beaver Triple server (a i , b i , c i Step (a) of receiving and storing ) - where i is an index representing the user terminal -; Step (b) of quantizing the local gradient obtained through local learning to compute the local sine gradient; Higher-order values ​​of a pre-set cryptographic polynomial (x) through multiple sub-round processes k , step (c) of calculating x) where x is the local sine gradient; Higher-order value of the above-calculated cryptographic polynomial (x k ) and the step (d) of encrypting the local sine gradient using the cryptographic polynomial; Step (e) of transmitting the encrypted local sign gradient from step (d) above to the federated learning server; A federated learning method comprising the step (f) of receiving a global update value from the federated learning server and performing local learning.

17. In the sub-round of step (c) above, the user terminals receive a preset higher-order value (x) of the local sine gradient k A federated learning method that encrypts , x (where x is a local sign gradient) using the beaver triple variable and transmits it to the federated learning server, and the federated learning server performs a procedure of broadcasting the values ​​encrypted by the beaver triple variable received from each user terminal to all user terminals.

Images