Privacy-protected federated learning method, terminal and storage medium

Abstract

The application relates to the technical field of computers, in particular to a privacy protection federated learning method, a terminal and a storage medium. The method comprises the following steps: S10, a global parameter model is established, and the global parameter model is sent to each client; S20, the client receives the global parameter model, trains using client data, obtains client model parameters and a corresponding loss function; S30, a privacy protection scheme of a DBSCAN algorithm is used to cluster the client model parameters, and a plurality of categories are obtained; S40, each category is averaged and aggregated based on the model parameters, and averaged and aggregated model parameters are obtained; and a global model is output according to the averaged and aggregated model parameters. The application uses a privacy protection method of a DBSCAN algorithm to secretly cluster the clients; and uses additive secret sharing to realize the safe aggregation of the model parameters in each category.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a privacy-preserving federated learning method, terminal, and storage medium. Background Technology

[0002] With the rapid development of network communication technology, personal information is easily collected, transmitted, and used by various smart terminal devices. If platform supervision is inadequate, personal privacy information can easily be leaked. At the enterprise level, the leakage of trade secrets can lead to a loss of competitive advantage, resulting in significant human and financial losses. Traditional machine learning requires collecting information on a central server, making client-side privacy data vulnerable to leakage. Federated learning, as a distributed machine learning model, does not require data to leave the local client, thus protecting data privacy. In federated learning, because data comes from various terminal devices, data heterogeneity naturally arises, meaning the data is non-identically distributed (non-IID). Utilizing data heterogeneity is particularly crucial in applications such as recommendation terminals and personalized advertising, benefiting both users and enterprises. In earlier federated learning methods, all user data was fed into the same global model for training, and after training, all users used the same model for prediction. This approach treats non-IID data as identically distributed, ignoring the differences between user data and thus failing to generate personalized predictions.

[0003] To address this technical issue, a privacy-preserving federated learning method, terminal, and storage medium are proposed. Summary of the Invention

[0004] To address the technical problems existing in the prior art, this invention provides a privacy-preserving federated learning method, terminal, and storage medium. After providing a fully automated implementation of group RAID functionality, multiple test cases can be placed in the same task on the automated testing terminal and executed automatically in sequence, with the group RAID operation for each test case being completed automatically.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, one embodiment of the present invention provides a privacy-preserving federated learning method, which includes the following steps:

[0007] S10. Establish a global parameter model and send it to each client;

[0008] S20. The client receives the global parameter model and trains it using client data to obtain client model parameters and the corresponding loss function.

[0009] S30. Cluster the client model parameters using the privacy protection scheme of the DBSCAN algorithm to obtain multiple categories;

[0010] S40. Perform average aggregation on each category based on the model parameters to obtain the average aggregated model parameters; output the global model based on the average aggregated model parameters.

[0011] As a further aspect of the present invention, servers A and B do not collude with each other.

[0012] As a further aspect of the present invention, the privacy protection scheme using the DBSCAN algorithm clusters the client model parameters to obtain multiple categories, including the following steps:

[0013] Establish keys between the client and servers A and B respectively using the DH protocol;

[0014] The client uses addition secret sharing to divide the data points into two parts, encrypts them with the corresponding keys, and sends them to the corresponding servers A and B.

[0015] Servers A and B decrypt the secret shared value of the received data points and obtain the distance between any two data points and the size of M.

[0016] Server A obtains the set of points in the ε-neighborhood of each point by using the relationship between the distance between two data points and ε. Then, Server A can directly use the DBSCAN algorithm to calculate and obtain the clustering results, which are multiple categories.

[0017] As a further aspect of the present invention, the clustering result of the model parameters is the clustering result of the client.

[0018] As a further aspect of the present invention, the keys between the client and servers A and B are established respectively through the DH protocol.

[0019] As a further aspect of the present invention, the step of establishing keys between the client and servers A and B respectively further includes:

[0020] Let the key between the l-th client and server A be . The key between server B and server B is

[0021] As a further aspect of the present invention, servers A and B respectively decrypt the secret shared value of the received data points to obtain the distance between any two data points and the size of M, and the method further includes:

[0022] The secret shared value between any two data points obtained by server A is... The secret shared value obtained by server B from any two data points is... Obtain the distance between any two data points and the value of M.

[0023] As a further aspect of the present invention, the secret shared value obtained by server A from any two data points is... The secret shared value obtained by server B from any two data points is... To obtain the distance between any two data points and the magnitude of M, the following steps are involved:

[0024] Server A calculates Server B calculates

[0025] Through multiplication secret sharing, servers A and B respectively obtain the data. and and satisfy

[0026] Server A calculates the data Server B calculates the data

[0027] Server B will sign(V) B The data is sent to server A, which then makes a preliminary judgment. And the magnitude of ε;

[0028] The method OT_Compare(|V) is used between the two servers. A |,|V B |) Comparison |V A |and|V B The size of | is then obtained And the magnitude of ε.

[0029] As a further aspect of the present invention, server B will sign(V) B The data is sent to server A, which then makes a preliminary judgment. The magnitude of ε includes the following cases:

[0030] I, V A and V B If all are greater than or equal to 0, we can obtain

[0031] II, VA and V B If all are less than or equal to 0, we can obtain

[0032] III, V A Greater than or equal to 0 and V B Less than 0, V A Less than or equal to 0 and V B If the value is greater than 0, then the method OT_Compare(|V) is used between the two servers. A |,|V B |) Comparison |V A |and|V B The size of | is then obtained And the magnitude of ε.

[0033] As a further aspect of the present invention, server B will sign(V) B The data is sent to server A, which then makes a preliminary judgment. The magnitude of ε includes the following cases:

[0034] I, V A and V B If all are greater than or equal to 0, we can obtain

[0035] II, V A and V B If all are less than or equal to 0, we can obtain

[0036] III, V A Greater than or equal to 0 and V B Less than 0, V A Less than or equal to 0 and V B If the value is greater than 0, then the method OT_Compare(|V) is used between the two servers. A |,|V B |) Comparison |V A |and|V B The size of | is then obtained And the magnitude of ε.

[0037] As a further aspect of the present invention, the method OT_Compare(|V) is used between the two servers. A |,|V B |) Comparison |V A |and|V B The size of | is then obtained The magnitude of ε includes the following cases:

[0038] I. If sign(V) A )·sign(|VA |-|V B |)≥0 can be obtained

[0039] II. If sign(V) A )·sign(|V A |-|V B |)≤0 can be obtained

[0040] As a further aspect of the present invention, step S40 includes the following steps:

[0041] S401. If it is the first iteration, skip this step; otherwise, the clients within the category divide the local model parameters into two parts through addition secret sharing, encrypt each part with the corresponding key, and send them to servers A and B respectively.

[0042] S402, Servers A and B respectively calculate the average of the secret shared values ​​of the local model parameters sent by the client within each category;

[0043] S403. Server B sends the average value of the model secret shared value for each category to Server A, which calculates the average value of the model parameters and updates the model parameters of each category to this average value.

[0044] S404. The client sends the local loss function to server A, which calculates the average value of the client's loss function within the category. If the average loss function converges, or the number of iterations reaches the requirement, the iteration stops, and server A outputs the global model.

[0045] S405. If the iteration does not stop, server A will send the updated model parameters within the category to the clients within the category.

[0046] S406. After receiving the updated model parameters, the client uses them as initial values ​​to train the new model parameters using local data, and then proceeds to step S401.

[0047] As a further aspect of the present invention, servers A and B respectively calculate the average of the secret shared values ​​of the local model parameters sent by the client within each category. Specifically, for the i-th category, server A calculates the average of the secret shared values ​​of the model parameters. Server B calculates the average of the secretly shared values ​​of the model parameters.

[0048] As a further aspect of the present invention, server A calculates the average model parameters within the category.

[0049] Secondly, in another embodiment provided by the present invention, a terminal is provided, including a memory and a processor, the memory storing a computer program, and the processor loading and executing the computer program to implement steps of a privacy-preserving federated learning method.

[0050] Thirdly, in another embodiment of the present invention, a storage medium is provided storing a computer program that, when loaded and executed by a processor, implements the steps of the privacy-preserving federated learning method.

[0051] The technical solution provided by this invention has the following beneficial effects:

[0052] The privacy-preserving federated learning method, terminal, and storage medium provided by this invention perform confidential clustering of clients using a DBCSCAN algorithm; secure aggregation of model parameters within each category is achieved by using additive secret sharing; and a key exchange algorithm is used to encrypt the secret shared values ​​transmitted between the client and the server with the exchanged key to prevent eavesdropping attacks.

[0053] These or other aspects of the invention will become more apparent from the following description of embodiments. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart of a privacy-preserving federated learning method according to an embodiment of the present invention.

[0056] Figure 2 This is a flowchart illustrating step S30 in a privacy-preserving federated learning method according to an embodiment of the present invention.

[0057] Figure 3 This is a flowchart illustrating step S50 in a privacy-preserving federated learning method according to an embodiment of the present invention.

[0058] Figure 4 This is a terminal structure diagram according to an embodiment of the present invention.

[0059] In the diagram: Processor-501, Communication Interface-502, Memory-503, Communication Bus-504. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0062] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0063] Federated Learning: First proposed by Google in 2016, federated learning was originally intended to address the issue of Android phone users updating models locally. Essentially, federated learning is a distributed machine learning technique that aims to achieve collaborative modeling while ensuring data privacy, security, and legal compliance.

[0064] DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm: It is a density-based clustering algorithm that defines a cluster as the largest set of density-connected points. It can divide regions with sufficiently high density into clusters and can discover clusters of arbitrary shapes in noisy spatial databases.

[0065] Secret Sharing: The idea behind secret sharing is to break down a secret in an appropriate way, with each share managed by a different participant. A single participant cannot recover the secret information; only by several participants working together can the secret message be recovered.

[0066] Oblivious Transfer (OT) is a cryptographic protocol in which a sender sends a message to a receiver from a list of pending messages without subsequently knowing which message the receiver received. It is sometimes also translated as unintentional transfer.

[0067] Symbolic representation: Let {0,1,…,n-1} be the set. amodc is read as a mod c, which means the remainder when a is divided by c. a=bmodc means that a and b have the same remainder when divided by c. Represents the space of real numbers. This represents n-dimensional Euclidean space. We generally use lowercase letters (e.g., x) to represent scalars, using... Vectors are represented in the form of . This indicates a bitwise XOR operation between x and y. `sign(x)` represents the sign function, and its value is...

[0068] Specifically, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0069] Please see Figure 1 , Figure 1 This is a flowchart of a privacy-preserving federated learning method provided in an embodiment of the present invention, such as... Figure 1 As shown, the privacy-preserving federated learning method includes steps S10 to S30. The privacy-preserving federated learning method is applied to a terminal, which includes at least one client and servers A and B. Servers A and B do not collude with each other.

[0070] S10. Establish a global parameter model and send it to each client;

[0071] S20. The client receives the global parameter model and trains it using client data to obtain client model parameters and the corresponding loss function.

[0072] Let the parameters of the local model trained by the l-th client be... The local loss function is F l .

[0073] By obtaining the local loss function as F l It can eliminate some malicious clients.

[0074] S30. Cluster the client model parameters using the DBSCAN algorithm's privacy protection scheme to obtain multiple categories.

[0075] The clustering result of the model parameters is the clustering result of the client. Let the key established between the l-th client and server A in the privacy protection scheme of the DBSCAN algorithm be... The key between server B and server B is The parameters sent by the l-th client to server A are... encryption As a result, the parameters sent to server B are... encryption The result is that The resulting category division is C = {C1, C2, ..., C...} k}

[0076] Additionally, regarding the privacy protection scheme of the DBSCAN algorithm, the Euclidean distance is used as the metric for the distance between data points.

[0077] Equipment requirements: Two servers, A and B, that do not communicate with each other. S clients (the number of clients can be one or more).

[0078] Data requirements: Client-side data in That is, all data points are n-dimensional vectors. Neighborhood parameters (ε, M).

[0079] In this embodiment of the invention, S30, the client model parameters are clustered using a privacy protection scheme based on the DBSCAN algorithm to obtain multiple categories, including the following steps:

[0080] S301. Establish keys between the client and each server separately using the DH protocol;

[0081] Let the key between the l-th client and server A be . The key between server B and server B is

[0082] S302. The client uses addition secret sharing to divide the data points into two parts, encrypts them with the corresponding keys, and sends them to the corresponding servers A and B.

[0083] For example, data points Divided by the secret of addition Then use the corresponding key pairs respectively. After encryption, the data is sent to servers A and B.

[0084] S303, servers A and B decrypt the secret shared value of the received data points respectively, and use the following steps to obtain the distance between any two data points and the size of M.

[0085] Among them, we use data points and Let's take an example. Assume server A receives data points. and The secret shared value Data points obtained by server B and The secret shared value The specific steps are as follows:

[0086] S3031, Server A Calculation Server B calculates

[0087] S3032. Through multiplication secret sharing, servers A and B respectively obtain data. They satisfy

[0088] S3033, Server A calculates the data. Server B calculates the data

[0089] It can be seen that V A +V B The result is As long as V can be determined A +V B The relationship with 0 can be used to determine And the magnitude of ε.

[0090] S3034, Server B will sign(V) B The data is sent to server A, which then makes a preliminary judgment. The magnitude of ε. Consider the following cases.

[0091] I, V A and V B If all are greater than or equal to 0, we can obtain

[0092] II, V A and V B If all are less than or equal to 0, we can obtain

[0093] III, V A Greater than or equal to 0 and V B Less than 0, V A Less than or equal to 0 and V B If the value is greater than 0, proceed to step S3035.

[0094] S3035, Use the method OT_Compare(|V) between two servers. A |,|V B |) Comparison |V A |and|V B The size of | is then obtained The magnitude of ε. Consider the following cases.

[0095] I. If sign(V) A )·sign(|V A |-|V B |)≥0 can be obtained

[0096] II. If sign(V) A )·sign(|V A |-|V B |)≤0 can be obtained

[0097] S304. Server A obtains the set of points within the ε-neighborhood of each point by analyzing the relationship between the distance between two data points and ε. Then, Server A can directly proceed with the DBSCAN algorithm to calculate the clustering results. The clustering results consist of multiple categories.

[0098] S40. Perform average aggregation on each category based on the model parameters to obtain the average aggregated model parameters; output the global model based on the average aggregated model parameters.

[0099] Step S40 involves averaging and aggregating the model parameters for each category to obtain the averaged and aggregated model parameters; and then outputting the global model based on the averaged and aggregated model parameters. Specifically, this includes:

[0100] a: Averaging the values ​​of each category based on the model parameters;

[0101] b: Servers A and B obtain the average aggregated model parameters;

[0102] c: Perform local training on the averaged aggregated model parameters;

[0103] d: Re-aggregate the model parameters within the categories after local training;

[0104] e: Repeat the above ad steps until the required number of iterations is reached, or the average loss function of the categories converges, and output the global model.

[0105] Specifically, within each category, two servers respectively average the secret-shared values ​​of the client's model parameters, then merge the two parts to obtain the average aggregated result for the model parameters of each category, and send this result to the client within that category. After receiving the aggregated model parameters, the client performs local model training and sends the secret-shared values ​​of the trained model parameters to each server A and B respectively, based on the additive secret sharing. Servers A and B then aggregate the model parameters within each category again until the required number of iterations is reached, or the average loss function of the category converges.

[0106] This invention utilizes the privacy protection method of the DBSCAN algorithm to achieve secure clustering of user model parameters in federated learning, thereby grouping clients and performing federated learning within each group. Since a privacy protection scheme using the DBSCAN algorithm employs two mutually independent servers, an additive secret-sharing technique is used during model parameter aggregation in federated learning to protect the privacy of user model parameters.

[0107] S40 includes the following steps:

[0108] Specifically, taking the i-th class as an example, that is, the number of clients that satisfy the index, let's call the number of clients of the i-th class.

[0109] S401. If it is the first iteration, skip this step; otherwise, the clients within the category divide the local model parameters into two parts through additive secret sharing, encrypt each part with the corresponding key, and send them to servers A and B respectively.

[0110] Since this step has already been achieved during the DBSCAN clustering phase, it can be skipped in the first loop.

[0111] S402, servers A and B respectively calculate the average of the secret shared values ​​of the local model parameters sent by the client within each category.

[0112] For the i-th category, server A calculates the average of the secret shared values ​​of the model parameters. Server B calculates the average of the secretly shared values ​​of the model parameters.

[0113] S403. Server B sends the average value of the model secret shared value for each category to Server A, which calculates the average value of the model parameters and updates the model parameters of each category to this average value.

[0114] Server A calculates the average model parameters within the category.

[0115] S404. The client sends its local loss function to server A, which calculates the average value of the client's loss function within each category. If the average loss function converges, or the required number of iterations is reached, the iteration stops, and server A outputs the global model.

[0116] S405. If the iteration does not stop, server A will send the updated model parameters within the category to the clients within the category.

[0117] S406. After receiving the updated model parameters, the client uses them as initial values ​​to train the new model parameters using local data, and then proceeds to step S401.

[0118] Specifically, the DBSCAN algorithm describes the compactness of a sample set based on the number of samples in the neighborhood. The parameters related to the DBSCAN algorithm are described below.

[0119] Assume the data point set is The following are related definitions:

[0120] ε-neighborhood. For Its ε-neighborhood refers to the set of data points D that are adjacent to each other. The set of samples whose distance is not greater than ε, i.e. The number of elements in this set is denoted as .

[0121] The core point. For any data point If its ε-neighborhood It contains at least M samples, that is but That's the core point.

[0122] Density reaches directly. If lie in Within its ε-neighborhood, and If it is the core point, then it is called Depend on Density directly affects the density. Conversely, it's not always the case, unless... This is also the core point. Therefore, density directly does not possess symmetry.

[0123] Density is achievable. For and If a sample sequence exists satisfy and Depend on If the density reaches directly, it is called... Depend on Density is achievable. It is easy to see that density achievable is transitive. Like density directness, density achievable also lacks symmetry.

[0124] Density connected. For and If a core object exists make and All by If the density can be reached, then it is called... and Density connectivity. This indicates that density connectivity exhibits symmetry.

[0125] In embodiments of the present invention, the DBSCAN algorithm groups density-connected data points into a class, or cluster. Its main steps are as follows:

[0126] Input: Sample set Neighborhood parameters (ε, M).

[0127] Output: Class partitioning C = {C1, C2, ..., C} k The value of k is unknown before the program runs.

[0128] The DBSCAN algorithm, which groups density-connected data points into one class, includes the following specific steps:

[0129] 1. Find all the core points.

[0130] Among them, if a certain point (for example) The number of points in the ε-neighborhood of ) is greater than M (i.e. This point is the core point. Let the set of all core points be Ω.

[0131] 2. Mark all points as unvisited.

[0132] 3. Extract a core point from Ω. (after removal) Mark this point as visited, set the current category number k, and create a queue for the current core point. Create the current category point set

[0133] 4. From the core queue Ω k Take an element from (after removal) ), get all in Unvisited set of points within the ε-neighborhood Unvisited core point set Will These points within the ε-neighborhood are all marked as visited, and the current class point set is updated to... The core point set of these points Add to the core point queue (after adding) ).

[0134] 5. Repeat step 4 until step 4 is completed, then the core queue Ω is finished. k It is empty. At this time, the current category point set C k Form a class and remove the core points marked as visited from Ω.

[0135] 6. Repeat steps 3-5 until the elements in Ω are empty, resulting in a category division of C = {C1, C2, ..., C...} k}

[0136] This invention employs a privacy-preserving method using the DBCSCAN algorithm to cluster clients while protecting user privacy. DBCSCAN clustering can eliminate some abnormal clients. Within each category, we use additive secret sharing to securely aggregate the client's model parameters. Federated learning for each client category significantly improves model accuracy. Furthermore, data is encrypted during the transmission of all secret parameters to prevent eavesdropping on private information.

[0137] OT (Operational Technology) is one of the most fundamental protocols for secure multi-party computation. It can be used to construct obfuscated circuits, zero-knowledge proof protocols, and other solutions. We will directly introduce the 1-out-of-N OT protocol. It is used to solve the following problems:

[0138] Alice has N values ​​v0, ..., v N-1 Bob wants to know one of the v σ ,σ∈{0,…,N-1}, Bob can obtain v by executing the OT protocol. σ The value of v cannot be obtained. i The value of i is not equal to the value of σ. Alice does not know which value Bob obtained, meaning Alice does not know the value of σ.

[0139] 1-out-of-N OT can be implemented as follows:

[0140] 1. Preparation Phase. The protocol is applicable to groups of order q with large prime numbers. The above operation (that is, the results of the operations in this protocol are all modulo q), selection group A primitive root g. A random oracle function H (e.g., SHA-1) is chosen. Parameters q, g, and H are shared by Alice and Bob.

[0141] 2. Initialization Phase: Alice selects N-1 random numbers C1, C2, ..., C N-1 Then select a random number r and calculate g. r Then C1, C2, ..., CN-1 ,g r Send to Bob. Alice pre-calculates (C1). r (C2) r ,…,(C N-1 ) r (Due to the difficulty of discrete logarithms, Bob cannot obtain C1, C2, ..., C...) N-1 (the discrete logarithm and the value of r).

[0142] 3. Online calculation stage:

[0143] a. Bob selects a random number k and sets...

[0144]

[0145] Then PK0 is sent to Alice. (Alice cannot retrieve the value of k).

[0146] b. Alice calculation (PK0) r Then calculate (PK) i ) r =(C i / PK0) r , 1≤i≤N-1. Then choose a random string R (R should be long enough to ensure that two different data have different hash values) for each M. i Encryption is performed for numbers 0 ≤ i ≤ N-1. Then the encrypted result and R are sent to Bob.

[0147] c. Bob can calculate (PK) σ ) r =(C σ / PK0) r =(g k ) r =(g r ) k Then use H((PK) σ ) r Decryption of R,σ) yields M i .

[0148] This protocol can then be used to safely compare the size of two numbers.

[0149] Suppose Alice has data x, and Bob has data y, and x, y ∈ {0, ..., N-1}. We can compare the size of x and y using the following steps.

[0150] 1. Alice constructs N plaintext messages.

[0151] 2. Bob obtains m through 1-out-of-N OT. y The value can be used to obtain the result.

[0152]

[0153] Bob then sends the size results of both results to Alice.

[0154] If x and y are ordinary real numbers, x can be represented in N-ary form, for example, x = x p-1 …x0.x -1 …x -q That is Integers have p digits, and decimals have q digits. y can also be represented in N-ary form as y = y p-1 …y0.y -1 …y -q Then, starting from the most significant bit, compare the two sizes.

[0155] 1. If x i =y i If -q≤i≤p-1, then x=y.

[0156] 2. If there exists k such that when i > k, x i =y i When i = k, x i >y i If x > y, then x > y.

[0157] 3. If there exists k such that when i > k, x i =y i When i = k, x i <y i Then x <y。

[0158] We use OT_Compare(x,y) to represent the process of comparing the size of the two.

[0159] Secret sharing is an important technique in secure multi-party computation. Because it is relatively simple to use, it is widely used in the field of privacy protection.

[0160] In the two-party additive secret sharing, data x is randomly divided into the sum of two data (x0, x1), that is, x = x0 + x1. Then x0 and x1 are stored in the participants P0 and P1 respectively. Data x can only be recovered by combining the data from participants P0 and P1.

[0161] Let's assume participant P0 has data x, and another participant P1 has data y. Following the process below, we can achieve secret sharing in addition.

[0162] 1. P0 generates a random number x0 and calculates x1 = x - x0, then sends x1 to P1.

[0163] 2. P1 generates a random number y0 and calculates y1 = y - y0, then sends y0 to P0.

[0164] 3. P0 calculates z0 = x0 + y0, and P1 calculates z1 = x1 + y1.

[0165] To calculate the sum of x and y, we only need to aggregate z0 and z1 to the requester, resulting in z0 + z1. It can be seen that this process does not leak the data x and y, effectively protecting data privacy.

[0166] One-out-of-N OT can be used to achieve secret sharing of two-way multiplication.

[0167] Let participant P0 possess data x, and participant P1 possess data y. Express x in N-ary form: x = x p-1 …x0.x -1 …x -q That is Integers have p digits, and decimals have q digits. y can also be represented in N-ary form as y = y p-1 …y0.y -1 …y -q When i = -q, ..., p-1, the following methods are used for calculation.

[0168] 1. Bob generation (m i,0 ,…,m i,N-1 ), where m i,0 It is a random number, m i,j =N i jy-m i,0 .

[0169] 2. Alice uses 1-out-of-N OT to obtain...

[0170] 3. Alice Calculation Bob Calculation It's easy to see that there is a z A +z B =x·y, meaning to get the value of x·y, we only need to sum up z. A ,z B The values ​​of x and y are sufficient; there is no need to disclose them.

[0171] It should be understood that although the above description follows a certain order, these steps are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, some steps in this embodiment may include multiple steps or multiple stages, which are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of the steps or stages in other steps.

[0172] In one embodiment, see Figure 4 As shown, an embodiment of the present invention also provides a terminal, including a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other through the communication bus 504.

[0173] Memory 503 is used to store computer programs;

[0174] The processor 501, when executing a computer program stored in the memory 503, performs the privacy-preserving federated learning method, and when executing instructions, implements the steps in the above method embodiments.

[0175] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0176] The communication interface is used for communication between the aforementioned terminal and other devices.

[0177] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0178] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0179] The terminal includes user equipment and network equipment. The user equipment includes, but is not limited to, computers, smartphones, and PDAs. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing, which is a type of distributed computing consisting of a super virtual computer composed of a group of loosely coupled computers. The terminal can operate independently to implement this invention, or it can connect to a network and interact with other terminals on the network to implement this invention. The network in which the terminal is located includes, but is not limited to, the Internet, wide area network (WAN), metropolitan area network (MAN), local area network (LAN), and VPN network.

[0180] The terminal includes user equipment and network equipment. The user equipment includes, but is not limited to, computers, smartphones, and PDAs. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing, which is a type of distributed computing consisting of a super virtual computer composed of a group of loosely coupled computers. The terminal can operate independently to implement this invention, or it can connect to a network and interact with other terminals on the network to implement this invention. The network in which the terminal is located includes, but is not limited to, the Internet, wide area network (WAN), metropolitan area network (MAN), local area network (LAN), and VPN network.

[0181] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0182] In one embodiment of the present invention, a storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the above method embodiments.

[0183] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Furthermore, any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory.

[0184] It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the associatedly listed items. The embodiment numbers disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0185] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A privacy-preserving federated learning method, characterized in that, The method includes: S10. Establish a global parameter model and send it to each client; S20. The client receives the global parameter model and trains it using client data to obtain client model parameters and the corresponding loss function. S30. Cluster the client model parameters using the privacy protection scheme of the DBSCAN algorithm to obtain multiple categories; S40. Perform average aggregation on each category based on the model parameters to obtain the averaged aggregated model parameters; output the global model based on the averaged aggregated model parameters; The privacy-preserving scheme using the DBSCAN algorithm clusters the client model parameters to obtain multiple categories, including the following steps: Establish separate client and server The key between them; The client uses an additive secret sharing mechanism to divide the data points into two parts, encrypts each part with a corresponding key, and then sends them to the corresponding server. ; The server Decrypt the secret shared value of each received data point and obtain the distance between any two data points. Size; The server The distance between two data points and The size relationship is used to obtain the value of each point. The set of points within the neighborhood, and then the server. The DBSCAN algorithm is used to calculate the clustering results. Among them, by M and ε Constructing neighborhood parameters ( ).

2. The privacy-preserving federated learning method as described in claim 1, characterized in that, The server Decrypt the secret shared value of each received data point to obtain the distance between any two data points. The size also includes: server The secret shared value obtained from any two data points is ,server Obtain the secret shared value of any two data points ; to obtain the distance between any two data points and Size.

3. The privacy-preserving federated learning method as described in claim 2, characterized in that, The server The secret shared value obtained from any two data points is ,server Obtain the secret shared value of any two data points ; Get the distance between any two data points and The size is determined by the following steps: server calculate ,server calculate ; Through multiplication secret sharing, the server Data were obtained separately , ,and satisfy ; The server Calculated data ,server B Calculated data ; The server Will Send to the server By the server Preliminary assessment and Size; On the server Methods used between Compare and The size is obtained and Size.

4. The privacy-preserving federated learning method as described in claim 3, characterized in that, The server Will Send to server By server Preliminary assessment and The size of includes the following cases: I, and If all are greater than or equal to 0, then we get ; II. and If all are less than or equal to 0, then we get ; III. Greater than or equal to 0 and Less than 0 Less than or equal to 0 and If the value is greater than 0, then the method is used between the two servers. Compare and The size is obtained and Size.

5. The privacy-preserving federated learning method as described in claim 3, characterized in that, Methods used between two servers Compare and The size, and thus obtain and The size, which includes: I. If ,get , II. If ,get .

6. The privacy-preserving federated learning method as described in claim 1, characterized in that, S40 includes the following steps: S401. If this is the first iteration, skip this step; otherwise, clients within the category divide their local model parameters into two parts through additive secret sharing, encrypt each part using the corresponding key, and send them to the server separately. ; S402, Server Average the secret-shared values ​​of the local model parameters sent by the client within each category; S403, Server Send the average value of the model secret sharing for each category to the server. By server Calculate the average value of the model parameters and update the model parameters of each category to this average value; S404, The client sends the local loss function to the server. By server Calculate the average value of the client loss function within each category. If the average loss function converges, or the number of iterations reaches the required number, stop iterating and then server A outputs the global model. S405, If iteration has not stopped, then the server... Send the updated model parameters within the category to clients within the category; S406. After receiving the updated model parameters, the client uses them as initial values ​​to train the new model parameters using local data, and then proceeds to step S401.

7. The privacy-preserving federated learning method as described in claim 6, characterized in that, The server The secret shared values ​​of the local model parameters sent by the client are averaged within each category, where, for the , Each category, server Calculate the average of the secret shared values ​​of the model parameters. ,server Calculate the average of the secret shared values ​​of the model parameters. .

8. A terminal comprising a memory and a processor, the memory storing a computer program, the processor loading and executing the computer program to implement the steps of the privacy-preserving federated learning method as described in any one of claims 1-7.

9. A storage medium storing a computer program that, when loaded and executed by a processor, implements the steps of the privacy-preserving federated learning method as described in any one of claims 1-7.

Images