Multi - center Federated Learning Method and Computer Device with Enhanced Privacy Protection

Abstract

The present invention discloses a multi - center federated learning method for enhancing privacy protection. The method includes: the client receives the public key {n, g} in the Paillier algorithm key {n, g, λ} generated by the server; the client receives the global parameter model established by the server, classifies the clients, and encrypts and sends the classification information and model parameters to the server for calculation and decoding; the clients are grouped, the clients receive the model parameters and adjust the classification, then encrypt the classification information and gradient parameters or model parameters, and perform preliminary aggregation on the ciphertext within the group and send it to the server, and the server decrypts it for further aggregation to obtain the new round of model parameters. This method extends the Paillier encryption method and the secure Euclidean distance calculation method to the real number range, and enhances privacy protection by encrypting the model parameters or gradient parameters and category information and performing calculations on the ciphertext.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and in particular, to a multi-center federated learning method for enhancing privacy protection and a computer device. Background Art

[0002] With the rapid popularization of intelligent terminal devices, a large amount of personal information data has been collected by various network platforms. In real life, the leakage of personal information has become quite common. In recent years, not only individuals have paid more attention to privacy data, but government departments have also successively introduced laws and regulations on network personal privacy protection and network security. Traditional machine learning requires collecting all information into a central server, and the privacy data of clients is easily leaked. As a distributed machine learning model, federated learning does not require data to leave local clients, thereby protecting the privacy of data. In federated learning, due to the fact that data comes from various terminal devices, the problem of data heterogeneity will naturally occur, that is, these data are non-independent and identically distributed (non-IID). In applications such as recommendation systems and personalized advertising, leveraging data heterogeneity is particularly crucial, which is beneficial to both users and enterprises. In relatively early federated learning methods, all user data is brought into the same global model for training, and after training is completed, all users also use the same model for prediction. This approach treats non-IID data as independent and identically distributed data, ignoring the differences between user data, and thus cannot generate some personalized predictions.

[0003] Many scholars adopt the method of establishing multiple global models to conduct federated learning on non-IID user data, and this method is called multi-center federated learning. Lu Yu et al. first cluster users using some statistical features of user data, regard the user data belonging to the same category as independently and identically distributed, and establish a global model for training within each category. Felix Sattler et al. use the geometric properties of the federated loss surface to cluster user groups and conduct federated learning within each category. Lu Yu, Felix Sattler et al. only cluster each client once. If the clustering is inappropriate, there is no corresponding correction mechanism subsequently, which leads to bias in the final trained model. Ming Xie et al. first establish multiple models (each model represents a category). After each round of training, according to the distance between the user model parameters and the parameters of each category model, the users are classified, and the model parameters of the users belonging to the same category are aggregated. However, in a neural network, for two similar models, their parameters may be very different, so this classification method is not necessarily reasonable. Avishek Ghosh et al. also first establish multiple models (each model represents a category). During each round of training, the user brings the data into these models to obtain multiple local loss functions, and this user is classified into the category of the model with the smallest loss function. And the author also gives a proof of the algorithm convergence under some conditions. However, Le Trieu Phong et al. pointed out that sharing local gradients or model data will lead to the leakage of privacy, and all the above multi-center federated learning algorithms directly send the user's gradient parameters or model parameters to the server, and the privacy information is easily obtained by other users through eavesdropping. And in the above algorithms, the classification situation of the clients is not protected either.

[0004] A commonly used means to protect data privacy is data encryption technology. Based on Avishek Ghosh et al., encryption technology is used to protect user privacy information (including the classification situation of users). In federated learning, after the model parameters are uploaded to the server, they need to be aggregated, and the main operation used for aggregation is addition operation. Therefore, the Paillier encryption method that satisfies additive homomorphism is selected to encrypt user privacy information (including model parameters and the category to which they belong), and then some parameter transfers and calculations are carried out, so as to achieve the purpose of protecting user privacy information. Summary of the Invention

[0005] In view of this, an object of an embodiment of the present invention is to propose a multi - center federated learning method based on enhanced privacy protection. This method groups clients, and uses the Paillier algorithm that satisfies additive homomorphism to encrypt model parameters, gradient parameters, and class information, then performs preliminary calculations on the ciphertext within each group, and finally aggregates the results to the server, where the server performs overall aggregation.

[0006] Based on the above object, on the one hand, an embodiment of the present invention provides a multi - center federated learning method for enhanced privacy protection. The method includes: the client receives the public key {n, g} in the Paillier algorithm key {n, g, λ} generated by the server; the client receives the global parameter model established by the server, classifies the client according to the local training result, encrypts the classification information and model parameters and sends them to the server, and the server performs calculations and decoding to obtain the initial value of the multi - center federated learning algorithm model parameters; and groups the clients, the client receives the model parameters, trains the corresponding model according to the category it belongs to, adjusts the classification, then encrypts the classification information and gradient parameters or model parameters, and performs preliminary aggregation on the ciphertext using the method of gradient averaging or model averaging within the group, and then sends the result to the server, and the server decrypts the preliminary aggregation result and performs further aggregation to obtain the model parameters of a new round.

[0007] In some embodiments, the client receives the global parameter model established by the server, classifies the client according to the local training result, and encrypts the classification information and model parameters and sends them to the server, and the server performs calculations and decoding to obtain the initial value of the multi - center federated learning algorithm model parameters, including: sending the global parameter model established by the server to the client, and the model parameters of the global parameter model are

[0008] Model training is performed on each client to obtain the preliminary model parameters of each client

[0009] Randomly select k clients from all clients, set the indexes of the k clients as i1, …, i k , and classify the clients according to the secure Euclidean distance between the model parameters of other clients and the model parameters of the k clients;

[0010] Select one client from other clients, and use the secure Euclidean distance calculation method to calculate the distance between the selected client and the model parameters of the i1, …, i k clients. If the distance between the selected client and the model parameters of the i t th client is the smallest, then classify the selected client into the t - th category, and send the category information it belongs to and the i tSend the distance of the client model parameters to the i-th t client;

[0011] Use the same method to complete the classification for the remaining clients among other clients;

[0012] Take t = 1, …, k respectively and perform the following operations:

[0013] The i-th t client counts the number M of clients belonging to the t-th class t , sends it to the clients belonging to the t-th class, and records the maximum distance dis t between the i-th t client and other clients within the t-th class, where the index set of the clients in the t-th class is L t ;

[0014] Each client in the t-th class divides the model parameter by the number M of clients in the corresponding class t , and sends the encrypted result to the i-th t client;

[0015] The i-th t client multiplies all the encrypted results within the t-th class to obtain

[0016] Send and dis t to the server for decryption to obtain

[0017] Obtain the maximum value dist among dis t , t = 1, …, k in this round;

[0018] Repeat the above steps, and select the result of the round with the smallest dist as the initial value of the multi-center federated learning algorithm.

[0019] In some embodiments, the encryption process of the Paillier algorithm includes:

[0020] Select two prime numbers p and q, set n = pq, λ = lcm(p - 1, q - 1), and satisfy gcd(λ, n) = 1, where lcm(p - 1, q - 1) represents the least common multiple of p - 1 and q - 1, and gcd(λ, n) represents the greatest common divisor of λ and n;

[0021] Select g such that it satisfies gcd(L(g λ mod n 2 ), n) = 1, and take (n, g) as the public key and λ as the private key;

[0022] For any Select a random number Perform encryption to obtain the ciphertext: c = g m r n mod n 2 , where represents the set {0, 1, …, n - 1}, represents the set of elements relatively prime to n in the set;

[0023] For any Perform decryption to obtain the plaintext: where, for the set S n = {S < n 2 | u = 1 mod n}, define the function L on S n as

[0024]

[0025] E(m) represents the encryption of the plaintext m, and D(c) represents the decryption of the ciphertext c;

[0026] According to the homomorphism principle, if then D(E(m1) … E(m k )) = m1 + … + m k .

[0027] In some embodiments, the method further includes extending the application scope of the encrypted Paillier algorithm from positive integers to real numbers. Extending the application scope of the encrypted Paillier algorithm from positive integers to real numbers includes:

[0028] Multiply the data by a certain multiple S uniformly before encryption to convert the data into integers;

[0029] Select a sufficiently large n such that the absolute value of the sum of the data after conversion into integers does not exceed n / 2;

[0030] Add n to the data less than 0 to convert all the data into natural numbers, and encrypt using the said Paillier algorithm;

[0031] After decrypting with the Paillier algorithm, if the data is greater than n / 2, subtract n from it and then divide by S. If it is less than n / 2, directly divide by S to obtain the final decryption result, where E0(m) represents the encryption of the plaintext m extended to the real number range, and D0(c) represents the decryption of the ciphertext c extended to the real number range.

[0032] In some embodiments, the method for calculating the secure Euclidean distance includes:

[0033] Input the confidential positive integer vector X = (x1, …, x n ) into the first client and input the confidential positive integer vector Y = (y1, …, y n ) into the second client to output f1(X, Y) = f2(X, Y) = |x1 - y1| + … + |x n -y n |;

[0034] Send the public key {N, g} in the Paillier key {N, g, λ} generated by the second client to the first client;

[0035] Send the data E(x1), …, E(x n encrypted by the second client for x1, …, x n ) to the first client;

[0036] Calculate E(x1)E(y1) -1 , …, E(x n )E(y n ) -1 by the first client and send them to the server after permuting the order, where E(y i ) -1 refers to the multiplicative inverse of E(y i ) in the multiplicative group ;

[0037] Decrypt each item sent by the client through the server. If the obtained results are z1, …, z n , if z i > N / 2, then z′ i = N - z i , otherwise z′ i = z i . Then, find the arithmetic square root of the sum of the squares of all z′ i to obtain the Euclidean distance between X and Y.

[0038] In some embodiments, the method further includes extending the application range of the confidential Euclidean distance from positive integer vectors to real number vectors. Extending the application range of the confidential Euclidean distance from positive integer vectors to real number vectors includes:

[0039] Multiply the data by a certain multiple S uniformly before applying the calculation method to convert the data into integers;

[0040] Select a sufficiently large n such that the absolute value of the data after being converted into integers does not exceed n / 8;

[0041] Add n / 8 to all the data to convert it into natural numbers within the range of 0 to n / 4;

[0042] After calculating the Euclidean distance of the transformed data using the above algorithm and then dividing it by S, the result obtained is the Euclidean distance between the original two real number vectors.

[0043] In some embodiments, the calculation results are aggregated to the server, and the overall aggregation by the server includes:

[0044] The client receives the initial value θ′ of the model parameters sent from the server j , j ∈ 1, …, k;

[0045] Randomly select m clients participating in this training, and randomly divide these m clients into P groups, and randomly select a client in each group as the central client. Let the set of indices of the clients belonging to the t-th group be L t , and the index of the central client of the t-th group is set as

[0046] Calculate the category to which the client belongs: where F i (θ′ j ) represents the result of substituting the data of the i-th client into the loss function corresponding to the j-th model parameter, where [k] = {1, …, k}, and define the one-hot encoding vector When then s ij = 1, when then s ij = 0, and encrypt it to obtain

[0047] If gradient averaging is selected, then: calculate the gradient Multiply each component of g i by s i and encrypt it to obtain According to the t-th group to which it belongs, this client sends s′ i , g′ i to the -th client; at the -th client (t = 1, …, P), calculate the product of the encrypted results of the intra-group gradient parameters and the product of the encrypted results of the category parameters: and Send the result to the server;

[0048] If it is model averaging, then: at the i-th client (i = 1, …, m), update the local model parameters Multiply each component of by s i and encrypt to obtain According to the group it belongs to (assumed to be the t-th group), this client will send it to the th client. The product of the encrypted results of the intra-group model parameters and the product of the encrypted results of the class parameters are calculated at the th client (t = 1,..., P), that is and Then the result is sent to the server;

[0049] The number of clients in the j-th class where j ∈ [k], and the following alternative model parameters are calculated by the server:

[0050] For gradient averaging:

[0051]

[0052] For model averaging:

[0053]

[0054] Repeat the above steps T times and output θ′ j , j ∈ [k].

[0055] In some embodiments, updating the local model parameters at the i-th client (i = 1,..., m) includes:

[0056] Calculating the gradient of F i (θ′ j ) with respect to θ′ j using a numerical solution method or the error backpropagation method

[0057] Update the local model parameter θ′ j = θ′ j - γg i , and use the value of θ′ j - γg i as the result of the new round of θ′ j , where γ is the step size;

[0058] Repeat the above process until the number of iterations reaches the set value τ, and output the final result of θ′ j , denoted as

[0059] In some embodiments, the encryption aggregation method in the process of multi-center federated learning further includes: a distributed privacy-preserving k-means clustering method (where each client has a piece of confidential data includes:

[0060] I. Randomly select k client indices as i1, …, i k , and classify these clients according to the Euclidean distance between the data of other clients and the data of the k clients;

[0061] II. The i t -th client counts the number M of clients belonging to the t-th class t , and then sends it to the clients belonging to the t-th class, where t ∈ [k]. Let the set of client indices of the t-th class be L t ;

[0062] III. Each client of the t-th class divides θ i by the number M of clients in the corresponding class t , and then sends the encrypted result E0(θ i / M t ) to the i t -th client;

[0063] IV. The i t -th client multiplies all the encrypted results within the t-th class to obtain

[0064] V. Send to the server for decryption to obtain the result

[0065] VI. Calculate the secure Euclidean distance between each client's data θ i and θ′ i in the server to re-classify the clients. Randomly select one client in each class, and update the indices to i1, …, i k ;

[0066] Repeat steps II - VI until the obtained θ′ i , i ∈ [k] are consistent, and then output the clustering center point θ′ i , i ∈ [k].

[0067] In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor. When the instructions are executed by the processor, the steps of the method include: the client receives the public key {n, g} in the Paillier algorithm key {n, g, λ} generated by the server; the client receives the global parameter model established by the server, classifies the client according to the local training result, encrypts the classification information and model parameters and sends them to the server, and the server calculates and decodes them to obtain the initial values of the multi-center federated learning algorithm model parameters; and groups the clients. The client receives the model parameters, trains the corresponding model according to the category it belongs to, adjusts the classification, then encrypts the classification information and gradient parameters or model parameters, and performs preliminary aggregation on the ciphertext using the method of gradient averaging or model averaging within the group, and then sends the result to the server. The server decrypts the preliminary aggregation result and performs further aggregation to obtain the new round of model parameters.

[0068] In another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above method steps.

[0069] The present invention has at least the following beneficial technical effects:

[0070] In the process of multi-center federated learning, the present invention adds an encryption and grouping process. First, the clients are grouped, and the Paillier algorithm that satisfies additive homomorphism is used to encrypt the model parameters or gradient parameters and category information. Then, preliminary calculations are performed on the ciphertext within each group. Finally, the results are summarized to the server, and the server performs overall aggregation. Since the calculations between clients are performed on the ciphertext, the data between different users is mutually confidential. This method also enables the server not to directly obtain the model parameters of users. Even if the server is malicious, this method can still protect the privacy information of clients. And the present invention modifies a calculation method of a confidential Manhattan distance into a calculation method of a confidential Euclidean distance, and uses it to give a way to select the initial value of the multi-center federated learning algorithm. This way also protects the privacy of users. Moreover, according to the way of selecting the initial value, a distributed privacy-preserving k-means clustering method is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments can be obtained based on these drawings.

[0072] Figure 1 Schematic diagram of an embodiment of the multi - center federated learning method for enhancing privacy protection provided by the present invention;

[0073] Figure 2 Schematic diagram of an embodiment of the computer device provided by the present invention;

[0074] Figure 3 Schematic diagram of an embodiment of the computer - readable storage medium provided by the present invention. Detailed implementation manners

[0075] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the following further details the embodiments of the present invention in conjunction with specific embodiments and with reference to the accompanying drawings.

[0076] It should be noted that all the expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two entities or parameters with the same name but different, so "first" and "second" are only for the convenience of expression and should not be construed as a limitation on the embodiments of the present invention. This will not be elaborated in subsequent embodiments one by one.

[0077] Federated Learning: Federated Learning was first proposed by Google in 2016 and was originally used to solve the problem of local model updating for Android mobile phone terminal users. Federated Learning is essentially a distributed machine learning technology, and its goal is to achieve joint modeling on the basis of ensuring data privacy security and compliance with laws and regulations.

[0078] Loss Function: It is a function that maps the values of random events or related random variables to non - negative real numbers to represent the "risk" or "loss" of the random event. In applications, the loss function is usually associated with the learning criterion and the optimization problem, that is, the model is solved and evaluated by minimizing the loss function.

[0079] Paillier Algorithm: Also known as the Paillier encryption algorithm, it is a probabilistic public - key encryption algorithm invented by Pascal Paillier in 1999. This algorithm is based on the difficult problem of composite residue classes and is a homomorphic encryption algorithm that satisfies addition.

[0080] Euclidean distance: Also known as Euclidean metric, it is a commonly used distance definition, referring to the actual distance between two points in an m-dimensional space, or the natural length of a vector (i.e., the distance from this point to the origin). In two-dimensional and three-dimensional spaces, the Euclidean distance is the actual distance between two points.

[0081] For the above purposes, in the first aspect of the embodiments of the present invention, an embodiment of a multi-center federated learning method for enhancing privacy protection is proposed. Figure 1 Shown is a schematic diagram of an embodiment of the multi-center federated learning method for enhancing privacy protection provided by the present invention. As Figure 1 shown, the multi-center federated learning method for enhancing privacy protection in the embodiments of the present invention includes the following steps:

[0082] 001. The client receives the public keys {n, g} in the Paillier algorithm key {n, g, λ} generated by the server;

[0083] 002. The client receives the global parameter model established by the server, classifies the clients according to the local training results, and encrypts and sends the classification information and model parameters to the server, and the server performs calculations and decoding to obtain the initial values of the multi-center federated learning algorithm model parameters;

[0084] 003. Group the clients. The clients receive the model parameters, train the corresponding models according to their respective categories, and adjust the classification. Then, encrypt the classification information and gradient parameters or model parameters, and perform preliminary aggregation on the ciphertext within the group using the method of gradient averaging or model averaging. After that, send the result to the server, and the server decrypts the preliminary aggregation result and performs further aggregation to obtain the new round of model parameters.

[0085] In this embodiment, the server generates the Paillier algorithm key {n, g, λ}, and sends the public keys {n, g} to the clients. The server establishes the global parameter model and sends it to the clients. Model training is performed on each client, and the clients are classified according to the training results. The clients are grouped, the classification of the models trained by the clients in each group is adjusted, and the client classification information and gradient parameters or model parameters are encrypted according to the Paillier encryption algorithm with additive homomorphism, preventing other users from eavesdropping on the client privacy information. Direct preliminary operations on the client privacy information are performed on the ciphertext within each group. After the operations are completed, the operation results are sent to the server, and the server performs aggregation. The user's privacy information is not directly sent to the server. Therefore, even if there is a malicious server, this algorithm can still provide good protection for the user's privacy information.

[0086] In some embodiments of the present invention, the client receives the global parameter model established by the server, classifies the clients according to the local training results, and encrypts and sends the classification information and model parameters to the server, and the server performs calculations and decoding to obtain the initial values of the multi - center federated learning algorithm model parameters, including:

[0087] Send the global parameter model established by the server to the client, and the model parameters of the global parameter model are

[0088] Perform model training on each client to obtain the preliminary model parameters of each client

[0089] Randomly select k clients from all clients, and set the indices of the k clients as i1, …, i k , classify the clients according to the secure Euclidean distance between the model parameters of other clients and the model parameters of the k clients;

[0090] Select a client from other clients, and use the secure Euclidean distance calculation method to calculate the distance between the selected client and the model parameters of the i1, …, i k clients. If the distance between the selected client and the model parameters of the i t th client is the smallest, classify the selected client into the t - th class, and send the category information and the distance from the model parameters of the i t th client to the i t th client;

[0091] Complete the classification of the remaining clients among other clients in the same way;

[0092] Respectively take t = 1, …, k, and perform the following operations:

[0093] The i t th client counts the number M of clients belonging to the t - th class t , send it to the clients belonging to the t - th class, and record the maximum distance dis t between the i t th client and other clients within the t - th class, where the index set of the clients in the t - th class is L t ;

[0094] Each client in the t - th class divides the model parameters by the number M of clients in the corresponding class t , and sends the encrypted result to the i t th client;

[0095] The i tA client multiplies all the encrypted results within the t-th class to obtain

[0096] Send and dis t to the server for decryption to obtain

[0097] Obtain the maximum value dist of dis t in this round, where t = 1,..., k;

[0098] Repeat the above steps and select the result of the round with the smallest dist as the initial value of the multi-center federated learning algorithm.

[0099] In this embodiment, the symbol representation is introduced as follows: lcm(a, b) represents the least common multiple of a and b, and gcd(a, b) represents the greatest common divisor of a and b. represents the set {0, 1,..., n - 1}. represents the set of elements relatively prime to n in the set. For the set S n = {u < n 2 | u = 1 mod n}, define the function L on S as n

[0100]

[0101] represents the real number space, represents the n-dimensional Euclidean space. |x| represents the absolute value of x.

[0102] If there are a total of n classes and a certain data belongs to the i-th class. The method of using a vector of length n with only the i-th component being 1 and other components being 0 to represent the class to which this data belongs is called one-hot encoding.

[0103] Suppose there are k different data distributions There are M clients, and the data z of the i-th client i，1 ,..., z i，n are independently and identically distributed according to a certain where the data point z i，l is composed of the feature x i，l and the response y i，l , that is

[0104] defines the loss function with respect to the data z, where represents the parameter space. Let Z represent the set of data of the i-th client, that is, Z = {z i，1 ,..., zi，n}, where \(|Z|\) represents the number of data in set \(Z\). The loss function of the \(i\)-th client with respect to \(Z\) is defined as

[0105]

[0106] The average value of the client loss functions (weighted average can also be used) is adopted to define the overall loss function. Suppose the data of several clients follow the distribution The index set of these clients is denoted as \(M\) j}, and the number of indices is denoted as \(|M|\) j |, then the overall loss function is

[0107]

[0108] where \([k]\) represents the set \(\{1, \ldots, k\}\). The goal is to cluster the clients (that is, estimate which distribution each client follows) and find the parameter \(\theta\) that minimizes the overall loss function \(F\) j (\(\theta\)).

[0109] For the solution of the parameter \(\theta\) when the overall loss function \(F\) j (\(\theta\)) takes the minimum value, the commonly used method is the stochastic gradient descent method. First, the initial value of the parameter \(\theta\) needs to be selected. Then, for each client belonging to the \(j\)-th class, a part of the data is selected to calculate the gradient of \(F\) i (\(\theta\)) with respect to \(\theta\), denoted as The gradient can be solved by numerical methods or the error backpropagation method. Then, a gradient averaging or model averaging aggregation method is selected to update the model parameters. If the gradient averaging aggregation method is adopted, directly send \(g\) i to the server. After the server aggregates the gradients, the aggregated gradients are used to update the parameter \(\theta\) using the gradient descent method. If the model averaging aggregation method is adopted, the model parameters need to be iteratively updated locally and then sent to the server. The server aggregates the model parameters and updates the model parameters to the aggregated model parameters. Finally, repeat the above process multiple times until the parameter \(\theta\) converges or the number of iterations reaches the requirement.

[0110] The specific process of local iterative update of the model parameters is as follows:

[0111] 1. Calculate the gradient of \(F\) i (\(\theta\)) with respect to \(\theta\) by numerical methods or the error backpropagation method

[0112] 2. Update the local model parameter \(\theta=\theta - \gamma g\) i (indicating that the value of \(\theta - \gamma g\) i is used as the result of the new round of \(\theta\)), where \(\gamma\) is the step size.

[0113] Repeat the above process until the number of iterations reaches the set value τ, and output the final result of θ, denoted as Denote this process as

[0114]

[0115] The main algorithm process of the multi - center federated learning scheme, which includes clustering clients and solving for the parameter θ when each F j (θ) reaches its minimum value j .

[0116] Input: The number of clusters k, the step size γ, the initial value of each class parameter The number of parallel iterations T, the number of local gradient iteration steps τ.

[0117] for t = 0, 1, …, T - 1 (that is, perform T rounds of loops)

[0118] Server: Send to each client, j ∈ [k].

[0119] Randomly select m clients participating in this training, and perform the following calculations on these m clients in parallel:

[0120] for i = 1, 2, …, m (that is, the following operations are performed on the i - th client)

[0121] Calculate the class to which the client belongs:

[0122] Define the one - hot encoded vector When then s ij = 1

[0123] Option 1: (Gradient averaging)

[0124] Calculate the (random) gradient and send g i , s i to the central server

[0125] Option 2: (Model averaging)

[0126] Update the local model parameters and send to the central server

[0127] end (for i)

[0128] Server: Obtain the class to which each client belongs according to s i Suppose the set of indices of clients belonging to the j - th class is mj , the number of this set is |m j |.

[0129] Option 1: (Gradient averaging)

[0130]

[0131] Option 2: (Model averaging)

[0132]

[0133] end (for t)

[0134] Output

[0135] There are mainly two steps in this algorithm,

[0136] 1. The client brings the data into each model to calculate the loss function respectively, selects the model with the smallest loss function, and classifies the clients, that is, classifies the clients into the th class.

[0137] 2. In each class, use gradient averaging or model averaging to update the model parameters of each class, that is, from Calculated to get

[0138] Then repeat the above two steps until the number of loops reaches T. The model parameters are the final result

[0139] On this basis, encrypt the gradient information or model parameter information and the class to which the client belongs and send it to the server. The algorithm steps after adding the encryption algorithm are:

[0140] I. The server generates the Paillier algorithm keys {n, g, λ}, and sends the public key {n, g} to the client.

[0141] II. The server establishes a global parameter model and sets k different initial values for the parameters. The initial values of the parameters are θ′ j , j ∈ [k].

[0142] The number of clusters k is known. The idea for selecting the initial values is:

[0143] 1. The server establishes a global parameter model, and the model parameters are denoted as and sends it to each client.

[0144] 2. Model training is performed on each client to obtain the preliminary model parameters of each client

[0145] 3. Randomly select k clients (assuming their indices are i1, …, i k ), and classify these clients according to the Euclidean distances between the model parameters of other clients and those of these k clients.

[0146] For a certain client, use the confidential Euclidean distance calculation method to calculate its distances from the model parameters of the i1, …, i k clients respectively. If its distance from the model parameters of the i t -th client is the smallest, then classify it into the t-th category. After obtaining the category, this client will send the category information it belongs to and its distance from the model parameters of the i t -th client to the i t -th client.

[0147] 4. The i t -th client counts the number M t of clients belonging to the t-th category, and then sends it to the clients belonging to the t-th category (here t ∈ [k]). And record the maximum distance dis t from itself to other points within the class.

[0148] Here, assume the set of indices of clients belonging to the t-th category is L t .

[0149] 5. Each client divides its model parameters by the number M t of clients in the corresponding category, and then encrypts it (the result is denoted as ) and sends it to the i t -th client.

[0150] 6. The i t -th client multiplies all the encrypted results within the class (including itself), that is , and then sends it to the server (here t ∈ [k]). And at the same time, send dis t to the server.

[0151] 7. The server decrypts the result sent by the i s -th client to obtain the result and calculates the maximum dis t , that is, dis = max 1≤t≤k dis t .

[0152] According to the homomorphic property of encryption, the content of decrypting the result sent by the i t -th client is actually that is, the average value of the model parameters of clients belonging to the t-th category.

[0153] 8. Repeat steps 3 - 7 and select the result of the round with the minimum dis. As the initial value of the multi - center federated learning algorithm.

[0154] In the above algorithm process, the client uses a secure Euclidean distance calculation method when calculating the distance between model parameters, and the results sent to other participants are all encrypted, which can effectively prevent eavesdroppers from obtaining users' privacy information. In addition, the data received by the server is the content after operations on the ciphertext. Even for a malicious server, the initial value selection method can still protect users' privacy information.

[0155] In some embodiments of the present invention, the encryption process of the Paillier algorithm includes:

[0156] Select two prime numbers p and q, set n = pq, λ = lcm(p - 1, q - 1), and satisfy gcd(λ, n)=1, where lcm(p - 1, q - 1) represents the least common multiple of p - 1 and q - 1, and gcd(λ, n) represents the greatest common divisor of λ and n;

[0157] Select g such that it satisfies gcd(L(g λ mod n 2 ), n)=1, (n, g) is used as the public key, and λ is used as the private key;

[0158] For any Select a random number For encryption, obtain the ciphertext: c = g m r n mod n 2 , where represents the set {0, 1,..., n - 1}, represents the set of elements relatively prime to n in the set;

[0159] For any For decryption, obtain the plaintext: where, for the set S n ={u < n 2 |u = 1 mod n}, define the function L on S n as

[0160]

[0161] E(m) represents the encryption of the plaintext m, and D(c) represents the decryption of the ciphertext c;

[0162] According to the homomorphism principle, if then D(E(m1)...E(m k)) = m1 + … + m k 。

[0163] In this embodiment, (1) Key generation: Select two large prime numbers p and q, let n = pq, and λ = lcm(p - 1, q - 1), where gcd(λ, n) = 1 is satisfied. Select g such that

[0164] gcd(L(g λ mod n 2 ), n) = 1,

[0165] Take (n, g) as the public key and λ as the private key.

[0166] (2) Encryption: For any Select a random number The ciphertext is

[0167] c = g m r n mod n 2 .

[0168] (3) Decryption: For any The plaintext is

[0169]

[0170] Use E(m) to represent the encryption of the plaintext m, and use D(c) to represent the decryption of the ciphertext c. The above algorithm is homomorphic, that is, for any There is

[0171] D(E(m1)E(m2)) = m1 + m2 mod n,

[0172] So, if There is also D(E(m1)E(m2)) = m1 + m2. It is easy to obtain the result of generalizing this property to the addition of multiple data, that is, if There is D(E(m1)…E(m k )) = m1 + … + m k .

[0173] In some embodiments of the present invention, the method further includes extending the application scope of the encrypted Paillier algorithm from positive integers to real numbers. Extending the application scope of the encrypted Paillier algorithm from positive integers to real numbers includes:

[0174] Multiply the data by a certain multiple S uniformly before encryption to convert the data into integers;

[0175] Select a sufficiently large n such that the absolute value of the sum of the data after conversion into integers does not exceed n / 2;

[0176] Add n to the data less than 0, convert all the data into natural numbers, and encrypt using the above Paillier algorithm;

[0177] After decrypting with the Paillier algorithm, if the data is greater than n / 2, subtract n from it and then divide by S. If it is less than n / 2, directly divide by S to obtain the final decryption result. Here, E0(m) represents the encryption of the plaintext m extended to the real number range, and D0(c) represents the decryption of the ciphertext c extended to the real number range.

[0178] In this embodiment, since the data precision in the computer is limited, multiply all the data by a certain multiple, assumed to be S, before encryption to convert the data into integers. Then select a sufficiently large n such that the absolute value of the sum of the data after conversion into integers does not exceed n / 2. Next, add n to the data less than 0 to convert all the data into natural numbers, and directly encrypt using the above Paillier algorithm. After decrypting with the Paillier algorithm, if the data is greater than n / 2, subtract n from it and then divide by S to obtain the final decryption result. According to the homomorphism of the Paillier algorithm, the algorithm extended to the real number range still satisfies additive homomorphism. Use E0(m) and D0(c) to represent the encryption of the plaintext m and the decryption of the ciphertext c by the Paillier algorithm extended to the real number range respectively.

[0179] In some embodiments of the present invention, the method for calculating the secure Euclidean distance includes:

[0180] Input the secure positive integer vector X=(x1,...,x n ) into the first client and input the secure positive integer vector Y=(y1,...,y n ) into the second client to output f1(X,Y)=f2(X,Y)=|x1 - y1|+...+|x n -y n |;

[0181] Send the public key {N, g} in the Paillier key {N, g, λ} generated by the second client to the first client;

[0182] Send the E(x1),...,E(x n ) encrypted by the second client for the data x1,...,x n to the first client;

[0183] Calculate E(x1)E(y1) -1 ,...,E(x n )E(y n ) -1 by the first client and send it to the server after permuting the order, where E(y i )-1 refers to the multiplicative inverse of E(y i ) in the multiplicative group ;

[0184] Decrypt each item sent by the client through the server. If the obtained results are z1,..., z n , if z i > N / 2, then z' i = N - z i , otherwise z' i = z i , and then calculate the arithmetic square root of the sum of the squares of all z' i to obtain the Euclidean distance between X and Y.

[0185] In this embodiment, input: First client: Input the secure positive integer vector X = (x1,..., x n ), Second client: Input the secure positive integer vector Y = (y1,..., y n ).

[0186] Output: f1(X, Y) = f2(X, Y) = |x1 - y1| +... + |x n - y n |.

[0187] 1. The first client generates the Paillier key {N, g, λ}, and sends the public key {N, g} to the second client. Here, it is required that N satisfies x i , y i < N / 4.

[0188] 2. The first client encrypts the data x1,..., x n to obtain E(x1),..., E(x n ), and sends them to the second client.

[0189] 3. The second client calculates E(x1)E(y1) -1 ,..., E(x n )E(y n ), and sends them to the first client after permuting the order. Here, E(y -1 ) i refers to the multiplicative inverse of E(y -1 ) in the multiplicative group i .

[0190] 4. The first client decrypts each item sent by the second client. Assume the obtained results are z1,..., z n . If z i > N / 2, then z' i = N - z i ​, otherwise z' i = z i , and then find the sum of squares of all z' i , and then find the arithmetic square root of the sum. The obtained result is the Euclidean distance between X and Y, and then send the final result to the second client.

[0191] In some embodiments of the present invention, the method includes extending the application range of the secure Euclidean distance from positive integer vectors to real number vectors. Specifically, it includes:

[0192] Before applying the calculation method, multiply the data by a certain multiple S to convert the data into integers;

[0193] Select a sufficiently large n such that the absolute value of the data after being converted into integers does not exceed n / 8;

[0194] Add n / 8 to all the data to convert it into natural numbers in the range from 0 to n / 4;

[0195] After using the above algorithm to calculate the Euclidean distance for the converted data, divide by S. The obtained result is the Euclidean distance between the original two real number vectors.

[0196] In this embodiment, the input of the above algorithm is extended from positive integer vectors to real number vectors. That is, before applying the algorithm, multiply the data by a certain multiple, assumed to be S, to convert the data into integers, and then select a sufficiently large n such that the absolute value of the data after being converted into integers does not exceed n / 8. Then add n / 8 to all the data to convert all the data into natural numbers in the range from 0 to n / 4. Then use the above algorithm to calculate the Euclidean distance for the converted data and divide by S. The obtained result is exactly the Euclidean distance between the original two real number vectors.

[0197] In some embodiments of the present invention, the clients are grouped. The clients receive the model parameters, train the corresponding models according to their respective categories, and adjust the classification. Then, encrypt the classification information and gradient parameters or model parameters, and use the method of gradient averaging or model averaging within the group to perform preliminary aggregation on the ciphertext. After that, send the result to the server. The server decrypts the preliminary aggregation result and performs further aggregation to obtain a new round of model parameters, including:

[0198] The client receives the initial value θ' of the model parameters sent from the server j , j ∈ 1,..., k;

[0199] Randomly select m clients participating in this training, and randomly divide these m clients into P groups, and randomly select a client in each group as the central client. The set of indices of the clients belonging to the t-th group is set as L t, set the index of the central client in the t-th group to

[0200] Calculate the category to which the client belongs: where F i (θ′ j ) represents the result of substituting the data of the i-th client into the loss function corresponding to the j-th model parameter, where [k] = {1,..., k}, and define the one-hot encoding vector When then s ij = 1, and when ij s = 0, and encrypt it to get

[0201] If gradient averaging is selected, then: Calculate the gradient Multiply each component of g i by s i and encrypt it to get According to the t-th group to which it belongs, this client sends s′ i , g′ i to the -th client; At the -th client (t = 1,..., P), calculate the product of the encrypted results of the intra-group gradient parameters and the product of the encrypted results of the category parameters: and Send the result to the server;

[0202] If it is model averaging, then: At the i-th client (i = 1,..., m), update the local model parameter Use to multiply each component of s i and encrypt to get According to the group to which it belongs (assuming it is the t-th group), this client sends to the -th client. At the -th client (t = 1,..., P), calculate the product of the encrypted results of the intra-group model parameters and the product of the encrypted results of the category parameters, that is and Then send the result to the server;

[0203] The number of clients in the j-th category where j ∈ [k], and the server calculates the following alternative model parameters:

[0204] For gradient averaging:

[0205]

[0206] For model averaging:

[0207]

[0208] Repeat the above steps T times and output θ′ j , j ∈ [k].

[0209] Update the local model parameters at the i-th client (i = 1, …, m) including:

[0210] Calculate F using a numerical solution method or backpropagation of error i (θ′ j ) of the gradient of θ′ j

[0211] Update the local model parameter θ′ j = θ′ j - γg i , and use the value of θ′ j - γg i as the result of the new round of θ′ j , where γ is the step size;

[0212] Repeat the above process until the number of iterations reaches the set value τ, and output the final result of θ′ j , denoted as

[0213] In this embodiment, the algorithm inputs: the number of clusters k, the step size γ, the initial value θ′ of each class parameter j , j ∈ [k], the number of parallel iterations T, and the number of local gradient iteration steps τ.

[0214] 1. The server sends the initial value θ′ j , j ∈ [k] to each client.

[0215] 2. Randomly select m clients participating in this training, randomly divide these m clients into P groups, and randomly select a client in each group as the central client.

[0216] Let the set of indices of the clients belonging to the t-th group be L t , and the index of the central client of the t-th group be

[0217] 3. Calculate the category to which the client belongs: Define a one-hot encoded vector When s ij = 1, and encrypt it to get​

[0218] 4. Option 1: (Gradient Averaging)

[0219] Calculate the (random) gradient Multiply each component of s by g i and encrypt it to obtain i According to the group it belongs to (assumed to be the t-th group), this client will send s′ and g′ i to the i -th client .

[0220] Option 2: (Model Averaging)

[0221] Update the local model parameters at the i-th client (i = 1,..., m) Multiply each component of s by and encrypt it to obtain i According to the group it belongs to (assumed to be the t-th group), this client will send to the -th client .

[0222] 5. Option 1: (Gradient Averaging)

[0223] At the -th client (t = 1,..., P), calculate the product of the encrypted results of the intra-group gradient parameters and the product of the encrypted results of the class parameters, that is and Then send the result to the server

[0224] Option 2: (Model Averaging)

[0225] At the -th client (t = 1,..., P), calculate the product of the encrypted results of the intra-group model parameters and the product of the encrypted results of the class parameters, that is and Then send the result to the server

[0226] 6. The server first calculates the number of clients belonging to the j-th class where j ∈ [k]. Then update the model parameters using the following formula

[0227] Option 1: (Gradient Averaging)

[0228]

[0229] Option 2: (Model Averaging)

[0230]

[0231] Then repeat the process from 1 to 6 until the above process is repeated T times, and then output θ'. j , j ∈ [k].

[0232] In the above algorithm, the client always sends the model or gradient parameters and class information to the th client (t = 1,..., P) after encryption. Therefore, this algorithm can effectively protect the privacy information of the client. And the th client performs calculations on the ciphertext of the parameters and then sends them to the server, so that the server cannot obtain the privacy information of the client. Therefore, even for a malicious server, this algorithm can still prevent the leakage of user privacy information.

[0233] In some embodiments of the present invention, the multi-center federated learning method for enhancing privacy protection further includes: a distributed privacy-protected k-means clustering method, where each client has a piece of confidential data The distributed privacy-protected k-means clustering method includes:

[0234] I. Randomly select k client indices i1,..., i k , and classify these clients according to the Euclidean distance between the data of other clients and the data of the k clients;

[0235] II. The t th client counts the number M t of clients belonging to the t-th class, and then sends it to the clients belonging to the t-th class, where t ∈ [k]. Let the set of client indices of the t-th class be L t ;

[0236] III. Each client of the t-th class divides θ i by the number M t of clients in the corresponding class, and then sends the encrypted result E0(θ i / M t ) to the t th client;

[0237] IV. The t th client multiplies all the encrypted results within the t-th class to obtain

[0238] V. Send to the server for decryption to obtain the result

[0239] VI. Calculate the distance between each client data θ i and θ' in the serveri The confidential Euclidean distance is used to classify the clients again. One client is randomly selected from each category, and the index is updated to i1, …, i k ;

[0240] Repeat steps II - VI until the θ′ obtained twice i , i ∈ [k] is consistent, and then output the clustering center point θ′ i , i ∈ [k].

[0241] In this embodiment, a distributed privacy - protected k - means clustering method, that is, each client has a piece of data that needs to participate in clustering The clients perform k - means clustering without disclosing their data, specifically as follows:

[0242] 1. Randomly select k clients (assuming their indices are i1, …, i k ), and classify these clients according to the Euclidean distance between the data of other clients and the data of these k clients (calculated using the calculation method of the confidential Euclidean distance).

[0243] 2. The i t -th client counts the number M of clients belonging to the t - th category t , and then sends it to the clients belonging to the t - th category (here t ∈ [k]).

[0244] Here, assume that the set of indices of clients belonging to the t - th category is L t .

[0245] 3. Each client divides θ i by the number M of clients in the corresponding category t , and then encrypts it (the result is denoted as E0(θ i / M t )) and sends it to the i t -th client.

[0246] 4. The i t -th client multiplies all the encrypted results within the class (including itself), that is and then sends it to the server (here t ∈ [k]).

[0247] 5. The server decrypts the result sent by the i s -th client to obtain the result

[0248] 6. Calculate the Euclidean distance between each client's data θ i and θ′ in the server iEuclidean distance (calculated using a secure Euclidean distance calculation method), reclassify these clients according to the distance. Then randomly select one client from each category and update i1, …, i with their indices k .

[0249] 7. Repeat the process of steps 2 - 6 until the θ′ obtained twice in step 5 i , i ∈ [k] are consistent and then stop. Output the clustering center point θ′ i , i ∈ [k].

[0250] For the above purposes, the second aspect of the embodiments of the present invention proposes a computer device. Figure 2 Shown is a schematic diagram of an embodiment of the computer device provided by the present invention. As Figure 2 shown, the computer device of the embodiments of the present invention includes the following devices: at least one processor 021; and a memory 022, the memory 022 stores computer instructions 023 that can run on the processor, and the steps of the method implemented when the instructions are executed by the processor include: the client receives the public key {n, g} in the Paillier algorithm key {n, g, λ} generated by the server; the client receives the global parameter model established by the server, classifies the clients according to the local training results, and encrypts and sends the classification information and model parameters to the server, and the server performs calculations and decoding to obtain the initial values of the multi - center federated learning algorithm model parameters; and groups the clients, the clients receive the model parameters, train the corresponding models according to the categories they belong to, and adjust the classification, and then encrypt the classification information and gradient parameters or model parameters, and perform preliminary aggregation on the ciphertext using the method of gradient averaging or model averaging within the group, and then send the result to the server, and the server decrypts the preliminary aggregation result and performs further aggregation to obtain the new round of model parameters.

[0251] The present invention adds an encryption and grouping process during the multi - center federated learning. First, the clients are grouped, and the Paillier algorithm that satisfies additive homomorphism is used to encrypt the model parameters, gradient parameters, and class information. Then, preliminary calculations are performed on the ciphertext within each group, and finally the results are aggregated to the server, where the server performs the overall aggregation. Since the calculations between clients are carried out on the ciphertext, the data between different users is mutually confidential. This method also prevents the server from directly obtaining the model parameters of users. Even if the server is malicious, this method can still protect the privacy information of clients. And the present invention modifies a calculation method of a confidential Manhattan distance into a calculation method of a confidential Euclidean distance, and uses it to give a way to select the initial value of the multi - center federated learning algorithm. This way also protects the privacy of users. Moreover, according to the way of selecting the initial value, a distributed privacy - protected k - means clustering method is provided.

[0252] The present invention also provides a computer - readable storage medium. Figure 3 Shown is a schematic diagram of an embodiment of the computer - readable storage medium provided by the present invention. As Figure 3 shown, the computer - readable storage medium 031 stores a computer program 032 that, when executed by a processor, executes the above - mentioned method.

[0253] Finally, it should be noted that those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above - mentioned embodiments can be completed by instructing relevant hardware through a computer program. The program of the server centralized testing method can be stored in a computer - readable storage medium. When the program is executed, it can include the processes of the embodiments of the above - mentioned methods. Among them, the storage medium of the program can be a magnetic disk, an optical disk, a read - only memory (ROM), or a random access memory (RAM), etc. The embodiments of the above - mentioned computer program can achieve the same or similar effects as the corresponding foregoing method embodiments.

[0254] In addition, the method disclosed according to the embodiments of the present invention can also be implemented as a computer program executed by a processor, and the computer program can be stored in a computer - readable storage medium. When the computer program is executed by the processor, the above - mentioned functions defined in the method disclosed in the embodiments of the present invention are executed.

[0255] In addition, the above - mentioned method steps and system units can also be implemented by using a controller and a computer - readable storage medium for storing a computer program that enables the controller to implement the above - mentioned step or unit functions.

[0256] Those skilled in the art will also understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. The functionality that can be implemented in various ways for each particular application by those skilled in the art, but such implementation decisions should not be construed as causing a departure from the scope of the disclosure of the embodiments of the present invention.

[0257] In one or more exemplary designs, the functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored on or transmitted via a computer-readable medium as one or more instructions or code. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. The storage media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and that can be accessed by a general purpose or special purpose computer or a general purpose or special purpose processor. Additionally, any connection is properly termed a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0258] The above are exemplary embodiments of the disclosure of the present invention, but it should be noted that various changes and modifications can be made without departing from the scope of the disclosure of the embodiments of the present invention as defined by the claims. The functions, steps, and / or actions of the method claims according to the disclosed embodiments herein need not be performed in any particular order. Moreover, although the elements of the embodiments of the present invention disclosed herein may be described or claimed in a singular form, they can also be understood as plural unless explicitly limited to the singular.

[0259] It should be understood that, as used herein, unless the context clearly supports the exception, the singular form "a" is intended to also include the plural form. It should also be understood that the "and / or" used herein refers to any and all possible combinations including one or more of the associated listed items.

[0260] The serial numbers of the disclosed embodiments of the present invention above are only for description and do not represent the superiority or inferiority of the embodiments.

[0261] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing the relevant hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disc, etc.

[0262] Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope (including the claims) of the disclosure of the embodiments of the present invention is limited to these examples; under the idea of the embodiments of the present invention, the technical features between the above embodiments or different embodiments can also be combined, and there are many other variations in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention shall be included in the protection scope of the embodiments of the present invention.

Claims

1. A multi - center federated learning method for enhancing privacy protection, characterized in that, including: The client receives the public key in the Paillier algorithm key { n , g , λ} generated by the server, { n , g}; The client receives the global parameter model established by the server, classifies the clients according to the local training results, encrypts the classification information and model parameters and sends them to the server, and the server performs calculations and decoding to obtain the initial values of the multi - center federated learning algorithm model parameters; and Group the clients. The clients receive the model parameters, train the corresponding models according to their categories, adjust the classification, then encrypt the classification information and gradient parameters or model parameters, and perform preliminary aggregation on the ciphertext using the method of gradient averaging or model averaging within the group, and then send the results to the server. The server decrypts the preliminary aggregation results and performs further aggregation to obtain a new round of model parameters; The client receives the global parameter model established by the server, classifies the clients according to the local training results, encrypts the classification information and model parameters and sends them to the server, and the server performs calculations and decoding to obtain the initial values of the multi - center federated learning algorithm model parameters including: Send the global parameter model established by the server to the client, and the model parameters of the global parameter model are ; Perform model training on each client to obtain the initial model parameters for each client ; Randomly select k clients from all clients, and set the indices of the k clients as i 1,…, i k , and classify the clients according to the secure Euclidean distance between the model parameters of other clients and the model parameters of the k clients; Select a client among other clients and use the confidential Euclidean distance calculation method to calculate the distance between the selected client and the i 1,…, i k client model parameters. If the distance between the selected client and the i t client model parameters of the i t th client is the smallest, classify the selected client into the 𝑡th category, and send the category information and the distance to the i t th client; Classify the remaining clients among other clients in the same way; Take \(t = 1,\ldots\) respectively, k and perform the following operations: The i t number of clients in the t-th category is counted for each client, M t and it is sent to the clients belonging to the t-th category, and the i t maximum distance between each client and other clients within the t-th category is recorded, dis t where the index set of the clients in the t-th category is L t ; Each client of the t-th class uses the model parameters divided by the number of clients M in the corresponding class t , and sends the encrypted result to the i-th t client; The i-th t client multiplies all the encryption results within the t-th class to obtain ; Send and dis t to the server for decryption to obtain ; Obtain this round dis t , t = 1, …, k the maximum value in dist ; Repeat the above steps and select dist the result of the smallest round as the initial value of the multi - center federated learning algorithm; Group the clients. The clients receive the model parameters, train the corresponding models according to their categories, adjust the classification, then encrypt the classification information and gradient parameters or model parameters, and perform preliminary aggregation on the ciphertext using the method of gradient averaging or model averaging within the group, and then send the results to the server. The server decrypts the preliminary aggregation results and performs further aggregation to obtain a new round of model parameters including: The client receives the initial values of the model parameters sent from the server ; Randomly select m clients participating in this training, and randomly divide these m clients into P groups. Then randomly select one client in each group as the central client. The set of indices of the clients belonging to the t -th group is denoted as L t , and the index of the central client of the t -th group is denoted as ; Calculate the category to which the client belongs: , where represents the result of substituting the data of the i-th client into the loss function corresponding to the j-th model parameter, where , define the one-hot encoding vector , when then , when then , and encrypt it to obtain ; If gradient averaging is selected, then: calculate the gradient , multiply each component of by , encrypt it to obtain . According to the group it belongs to, the t th group, this client will send to the th client; At the th client ( t = 1, …, P), calculate the product of the encrypted results of the within-group gradient parameters and the product of the encrypted results of the class parameters: and , and send the result to the server; If it is model averaging, then: At the th client ( i = 1, …, m ), update the local model parameters , multiply each component of by s i , and encrypt to get . According to the group it belongs to (assuming it is the t th group), this client will send to the th client. At the th client ( t = 1, …, P), calculate the product of the encrypted results of the in-group model parameters and the product of the encrypted results of the class parameters, that is, and , and then send the result to the server; The number of clients of class , where , the following alternative model parameters are calculated by the server: For gradient averaging: For model averaging: Repeat the above steps T times and output .

2. The multi - center federated learning method for enhancing privacy protection according to claim 1, wherein, The encryption process of the Paillier algorithm includes: Select two prime numbers p and q, let n = pq, λ = lcm(p - 1, q - 1), and satisfy gcd(λ, n)=1, where lcm(p - 1, q - 1) represents the least common multiple of p - 1 and q - 1, and gcd(λ, n) represents the greatest common divisor of λ and n; Select \(g\) to satisfy , ([[]] n , g ) as the public key, λ as the private key; For any , select a random number , and perform encryption to obtain the ciphertext: , where represents the set {0, 1, …, n - 1}, represents the set of elements in n that are relatively prime to For any is decrypted to obtain the plaintext: , where for the set , it is defined that S n The function L on is ， E ( m ) represents the encryption of the plaintext m . D ( c ) represents the decryption of the ciphertext c . According to the homomorphism principle, if , then .

3. The multi - center federated learning method for enhancing privacy protection according to claim 2, wherein, Further include extending the application range of the encrypted Paillier algorithm from positive integers to real numbers. Extending the application range of the encrypted Paillier algorithm from positive integers to real numbers includes: Multiply the data by a certain multiple S uniformly before encryption to convert the data into integers; Select a sufficiently large n so that the absolute value of the sum of the data after conversion into integers does not exceed n / 2; Add n to the data less than 0, convert all the data into natural numbers, and encrypt using the above - mentioned Paillier algorithm; After decrypting using the Paillier algorithm, if the data is greater than n / 2, subtract n from it and then divide by S. If it is less than n / 2, directly divide by S to obtain the final decryption result. Among them, E 0 ( m ) represents the encryption of the plaintext extended to the real number range m . D 0 ( c ) represents the decryption of the ciphertext extended to the real number range c .

4. The multi - center federated learning method for enhancing privacy protection according to claim 1, wherein The calculation method of the secure Euclidean distance includes: Input the confidential positive integer vector into the first client and input the confidential positive integer vector into the second client to output ; Send the public key in the Paillier key { n , g , λ} generated by the second client to the first client{ n , g}; Send data encrypted by the second client to the first client x 1,…, x n to obtain E( x 1),…, E( x n ); Calculate through the first client and send it to the server after permuting its order, where refers to the multiplicative inverse in the multiplicative group ; Decrypt each item sent by the client through the server. If the obtained result is , if , then , otherwise , and then calculate the arithmetic square root of the sum of the squares of all to obtain the Euclidean distance between X and Y.

5. The multi - center federated learning method for enhancing privacy protection according to claim 4, wherein, Further include extending the application range of the secure Euclidean distance from positive integer vectors to real - number vectors. Extending the application range of the secure Euclidean distance from positive integer vectors to real - number vectors includes: Multiply the data by a certain multiple S uniformly before applying the calculation method to convert the data into integers; Select a sufficiently large such that the absolute value of the data after conversion to an integer does not exceed n / 8; Add n / 8 to all data to convert it into natural numbers within the range of 0 to n / 4; After calculating the Euclidean distance of the converted data using the above algorithm, divide it by S, and the result obtained is the Euclidean distance between the original two real - number vectors.

6. The multi - center federated learning method for enhancing privacy protection according to claim 1, wherein, At the th client ( i = 1, …, m ), update the local model parameters including: Calculate using numerical solution methods or error backpropagation methods For gradient of , Update the local model parameters , and use the value of as the result of the new round of , where γ is the step size; Repeat the above process until the number of iterations reaches the set value τ , and output the final result, denoted as .

7. The multi - center federated learning method for enhancing privacy protection according to claim 1, characterized in that, Further included is a distributed privacy-preserving k-means clustering method, where each client has a piece of confidential data , , and the distributed privacy-preserving k-means clustering method includes: I. Randomly select k client indices as i1, ⋯, i k , and classify these clients according to the Euclidean distance between the data of other clients and the data of the k clients; II. The i t number of clients in the t client statistics of the M t second category is counted, and then it is sent to the clients belonging to the t first category, where t ∈ k . Let the set of client indices of the t first category be L t ; III. Each client of the t-th type divides θ i by the number of clients M in the corresponding category t , and then sends the encrypted result to the i-th t client; IV. The i-th t client multiplies all the encryption results within the t-th category to obtain ; V. Send to the server for decryption to obtain the result ; VI. Calculate the data θ of each client i with that in the server calculate the secure Euclidean distance to classify the clients again, randomly select one client from each category, and update the index to ; Repeat steps II-VI until the output the clustering center when they are consistent .

8. A computer device, characterized in that, including: At least one processor; and A memory that stores computer instructions executable on the processor, and when the instructions are executed by the processor, the steps of the method according to any one of claims 1-7 are implemented.

Images