Random group interactive privacy federation aggregation method, system and electronic device

Abstract

The present invention discloses a random group interactive privacy federation aggregation method, system and electronic device, and the method is as follows: all mobile sensing nodes reach an agreement, the server sends initial parameters, introduces volunteer nodes, and initializes the protocol; participants broadcast public keys, mobile sensing nodes randomly select volunteer nodes to form groups, and the volunteer nodes in the group reach an agreement with the mobile sensing nodes to generate shared keys; model training is performed based on local real data to obtain local model gradients, and the mobile sensing nodes generate random masks through all shared keys obtained by interacting with volunteer nodes in the group; the mobile sensing nodes upload mask gradients to the server, and the server calculates the sum of mask gradients; the volunteer nodes upload mask sums to the server, and the server collects random numbers and aggregates them; a global model gradient is obtained; the server averages the global model gradients and feeds them back to the mobile sensing nodes as the initial parameters for the next round of training until the model converges on all mobile sensing nodes.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and specifically designs a random grouping interactive privacy federation aggregation method, system and electronic equipment for mobile sensing nodes. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, traditional centralized training methods usually rely on aggregating large amounts of data to servers for unified processing. Although this method has obvious advantages in computing efficiency, it poses a significant risk of data privacy leakage in application scenarios involving sensitive data (such as medical data). To solve this problem, federated learning, as an emerging distributed machine learning method, has gradually been applied to the field of privacy data protection. The core advantage of federated learning is that it allows multiple users to perform data processing and model training locally without uploading sensitive data to the cloud or central server. This method can not only effectively protect user privacy, but also significantly reduce the risk of data leakage.

[0003] Mobile sensing nodes are particularly widely used in the framework of federated learning. Mobile sensing nodes usually refer to mobile devices with sensing, computing and communication capabilities, such as smartphones, smart watches, health monitoring devices, etc. They can collect, process and analyze data locally without uploading users' sensitive information to external servers. For example, in medical health monitoring, devices such as smart watches or mobile phones can collect personal health data such as heart rate, number of steps, sleep quality, etc. in real time and perform preliminary processing locally. Then, through the federated learning framework, these devices can share model updates and learned knowledge with other devices without exposing any personal sensitive data. This approach ensures that users' privacy is effectively protected while conducting large-scale distributed learning, reducing the risk of data leakage and abuse.

[0004] However, in the process of federated learning joint training, the model parameters (such as gradient information) uploaded by the mobile sensing nodes may contain sensitive information, so there is still a potential risk of privacy leakage, especially in the communication process between the mobile sensing nodes and the server, attackers may infer the content of the original data by analyzing the uploaded gradient data. At present, some protection protocols have been proposed to enhance the security of federated learning, but the existing technologies have high communication complexity, high local computing cost, and high communication cost consumption, ignoring the need for lightweight and secure model frameworks. Summary of the invention

[0005] In order to solve the problem of model update information exposure caused by naked gradient transmission of mobile sensing nodes, and to solve the problems of high computational complexity and high communication cost caused by complex interactions of mobile sensing nodes, the present invention discloses a random grouped interactive privacy federated aggregation method, system and electronic device.

[0006] The present invention adopts the following technical scheme:

[0007] The random group interactive privacy federated aggregation method is as follows:

[0008] (1) Initialization

[0009] After all mobile sensing nodes reach an agreement, the server first sends the initial model parameters, and then introduces volunteer nodes to perform protocol initialization operations;

[0010] (2) Random grouping

[0011] All participants broadcast the public key, the mobile sensing node randomly selects volunteer nodes to form a group, and each volunteer node in the group reaches an agreement with the mobile sensing node to generate a shared key;

[0012] (3) Protection gradient

[0013] The local model gradient is obtained by model training based on local real data. The mobile sensing node generates a random mask by interacting with all shared keys obtained by the volunteer nodes in the group, and uses the random mask to protect the local model gradient, which is the masked gradient.

[0014] (4) Aggregation gradient

[0015] All mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients; all volunteer nodes upload random number sums to the server, and the server collects and aggregates random number sums from each volunteer node; the true global model gradient is obtained through calculation;

[0016] (5) Model iteration

[0017] The server averages the global model gradient and feeds it back to all mobile sensing nodes as the initial parameters for the next round of training until the model converges on all mobile sensing nodes.

[0018] Preferably, in step (1), the protocol initialization is specifically as follows:

[0019] Private key generation: The mobile sensing node and the volunteer node use the predefined prime number p and generator g; each party independently selects a private key, and the private key of the mobile sensing node is denoted as c sk , the private key of the volunteer node is recorded as v sk , both are randomly chosen and less than p;

[0020] Public key generation: Based on their respective private keys, each party calculates their public key:

[0021] The public key calculated by the mobile sensing node:

[0022] Among them, c pk is the public key of the mobile sensing node c;

[0023] The public key calculated by the volunteer node:

[0024] Where mod represents the modulus operator, which is defined as the remainder of division; v pk is the public key of volunteer node v.

[0025] Preferably, step (2) is as follows: all participants broadcast their public keys, and the mobile sensing node randomly selects volunteer nodes to form a group, where 1≤|num|≤|V|, and each volunteer node can be selected by different mobile sensing nodes to form a group; each volunteer node v∈num in the group v ∈V reaches an agreement with the mobile sensing node c and generates a shared key k according to the Diffie-Hellman key exchange mechanism c,v :

[0026]

[0027] Preferably, step (3) is as follows: after the mobile sensing nodes are grouped, model training is performed based on local real data to obtain local model gradients; the mobile sensing nodes generate random mask PRG (k c,v ), using random mask to protect local model gradient, that is, mask gradient; the encryption formula of local gradient of each mobile sensing node is as follows, x is the real model gradient or model parameter of each mobile sensing node:

[0028]

[0029] For each volunteer node selected into a group, it can interact with multiple mobile sensing nodes in a group; let the set of mobile sensing nodes that select the volunteer node be denoted as num c , the volunteer node uses the following formula to calculate the corresponding random number and R v :

[0030]

[0031] Preferably, step (4) is as follows: all mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients; all volunteer nodes upload random number sums to the server, and the server collects random number sums R from each volunteer node. v And aggregate them; the true global model gradient is obtained by calculation:

[0032]

[0033] Among them, C is the set of all mobile sensing nodes, and V is the set of all volunteer nodes.

[0034] The present invention also discloses a random group interactive privacy federation aggregation system for executing the above method, which includes a mobile sensing node, a volunteer node and a server;

[0035] Each mobile sensing node has real data for model training, and independently and randomly selects any number of volunteer nodes and groups them. Within the group, the mobile nodes interact with the volunteer nodes and generate masks.

[0036] The volunteer nodes reach an agreement with the mobile sensing nodes to generate a shared key. Each volunteer node calculates the sum of all random numbers it processes and sends the aggregated sum value to the server.

[0037] The server merges the mask gradients transmitted by all mobile sensing nodes and the mask value of the random number provided by the volunteer node.

[0038] The present invention also discloses an electronic device, comprising:

[0039] processor;

[0040] The memory is used to store the program. When the program is called and executed by the processor, the processor executes the above method.

[0041] In view of the above problems existing in the prior art, the present invention proposes a random group interactive efficient privacy federated aggregation technology solution based on federated learning to protect the gradient information privacy of mobile sensing nodes. Specifically, the mobile sensing node randomly selects any volunteer node and forms a group, uses the Diffie-Hellman protocol to interactively generate a shared key, and uses this key as the seed of the pseudo-random number generator (PRG), generates a mask and adds the sum value to the model gradient, and then uploads it to the server. The server aggregates the mask values ​​generated by all volunteer nodes and removes the mask to obtain the true global model gradient. Compared with the prior art, the present invention effectively protects the model gradient privacy of the mobile sensing node and prevents the privacy leakage problem in the data upload process, thereby improving the security of the mobile terminal in the practical application of federated learning. In addition, in the present invention, the protocol is based on a lightweight cryptographic protocol, and the mobile sensing node can autonomously select any number of volunteer nodes according to its own computing power, without interacting with all introduced volunteer nodes, so the computing and communication costs of the mobile sensing node can be greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a random group interactive privacy federation aggregation system architecture diagram of a preferred embodiment of the present invention;

[0043] Figure 2 It is a flow chart of a random group interactive privacy federation aggregation method according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0044] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0045] like Figure 1 As shown, a random group interactive privacy federated aggregation system in this embodiment consists of three entities: a mobile sensing node, a volunteer node and a server.

[0046] Each mobile sensing node c∈C (C is a set of mobile sensing nodes) has a certain amount of real data for model training. According to the protocol, each node voluntarily abides by the protocol rules to promote the joint training process. Specifically, they independently and randomly select any number of volunteer nodes and form groups. Within the group, the mobile nodes interact with the volunteer nodes and generate masks to protect the parameters of their local models, thereby ensuring the privacy of the original data.

[0047] The volunteer nodes (defined as a set V) can represent authoritative institutions in various fields, for example, in the medical field, they can be hospitals or medical entities conducting research within the healthcare sector. They neither collude nor participate in any training process. Their main task is to reach a consensus with the client (the mobile sensing node that selected the volunteer node) on the shared key, which is the seed for generating random numbers used to mask the gradient. In each round of the protocol, the selected volunteer node reaches an agreement with the active mobile sensing nodes (online mobile sensing nodes, participating in federated training, and inactive refers to offline nodes) to generate a shared key. Subsequently, each volunteer node calculates the sum of all random numbers it processes and sends the aggregated sum value to the server.

[0048] The server merges the local mask gradients transmitted by all mobile sensing nodes and the mask value of the random number provided by the volunteer node.

[0049] Preliminary knowledge: The pseudo-random generator (PRG) starts with an initial seed value and generates a seemingly random numerical sequence through a deterministic algorithm (hash function, such as SHA-256 function, or other hash functions). This sequence is unpredictable to the observer and is very similar to true random numbers in statistical properties. However, if the generation process and seed value are known, the same sequence can be reproduced. Therefore, in cryptographic applications, it is very important to choose a sufficiently secure seed value to ensure that the generated pseudo-random number sequence cannot be easily predicted or reproduced.

[0050] like Figure 2 As shown, this embodiment discloses a random group interactive privacy federation aggregation method, which is as follows:

[0051] (1) Initialization

[0052] After all mobile sensing nodes reach an agreement, federated learning training begins. First, the server sends the initial model parameters, and then introduces volunteer nodes to perform protocol initialization operations:

[0053] Private key generation: The mobile sensing node and the volunteer node use a predefined large prime number p and a generator g. Each party independently selects a private key. The private key of the mobile sensing node is denoted as c sk , the private key of the volunteer node is recorded as v sk , both of which are chosen randomly and are less than p.

[0054] Public key generation: Based on their respective private keys, each party calculates their public key:

[0055] The mobile sensing node calculates its public key:

[0056] The volunteer node calculates its public key:

[0057] (2) Random grouping

[0058] All participants broadcast their public keys, and the mobile sensing node randomly selects a certain number of volunteers (1≤|num|≤|V|) to form a group to prepare for the generation of shared keys. Each volunteer node can be selected by different mobile sensing nodes to form a group multiple times. v ∈V will reach an agreement with the mobile sensing node c and generate a shared key k according to the Diffie-Hellman key exchange mechanism c,v :

[0059]

[0060] (3) Protection gradient

[0061] After the mobile sensing nodes are grouped, they train the model based on the local real data to obtain the local model gradient. The mobile sensing nodes generate random masks PRG(k c,v ), using random mask to protect local model gradient, which is masked gradient. The encryption formula of local gradient of each mobile sensing node is as follows, where x is the real model gradient or model parameter of each mobile sensing node:

[0062]

[0063] For each volunteer node selected into a group, it can interact with multiple mobile sensing nodes in a group. Assume that the set of mobile sensing nodes that selects this volunteer node is represented as num c The volunteer node calculates the sum of its random numbers R using the following formula v :

[0064]

[0065] The mobility-aware nodes will not know the volunteer nodes selected by others, and the volunteer nodes will neither collude nor participate in other computations.

[0066] (4) Aggregation gradient

[0067] All mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients. All volunteer nodes upload random numbers and R to the server, and the server collects random numbers and R from each volunteer node. v And aggregate them. The true global model gradient is obtained by calculation:

[0068]

[0069] (5) Model iteration

[0070] The server averages the global model gradient and feeds it back to all mobile sensing nodes as the initial parameters for the next round of training until the model converges on all mobile sensing nodes. The specific convergence condition refers to the threshold of the loss function: when the loss function of the model on the training data reaches a preset small value, the model is considered good enough and training can be stopped.

[0071] The following is an explanation with reference to specific application cases.

[0072] Taking the health monitoring scenario as an example, there are four existing mobile sensing nodes that perform the task of jointly training the model without disclosing the local data set. Specifically, the mobile sensing node performs local training through the protocol of the present invention and uploads the mask gradient to the server. Finally, the server calculates the global model gradient through the aggregation algorithm commonly used in federated learning. In the protocol registration phase, it is stipulated that the mobile sensing nodes use the same network and optimizer for local training.

[0073] (1) Protocol initialization: The server sends the initial model parameters of the mobile sensing node and introduces 8 volunteer nodes. The mobile sensing node and the introduced volunteer nodes are initialized and public and private keys are generated.

[0074] (2) Random grouping: The mobile sensing node randomly selects any number of volunteer nodes to interact through the Diffie-Hellman mechanism to generate a shared key (1≤|num|≤|V|).

[0075] Sensing node 1 selects volunteer nodes 1, 3, 6, and 8 to form a group and generate a shared key k 1,1 ,k 1,3 ,k 1,6 ,k 1,8 ;

[0076] Perception node 2 selects volunteer nodes 2, 3, and 5 to form a group and generate a shared key k 2,2 ,k 2,3 ,k 2,5 ;

[0077] Sensing node 3 selects volunteer nodes 3, 4, 6, 7, and 8 to form a group and generate a shared key k 3,3 ,k 3,4 ,k 3,6 ,k 3,7 ,k 3,8 ;

[0078] Perception node 4 selects volunteer nodes 2 and 8 to form a group and generates a shared key k 4,2 ,k 4,8 .

[0079] (3) Protecting gradients: The mobile sensing node transmits the processed raw data to the local training network (the local training network preset by the protocol can be a fully connected network, a convolutional network or other deep learning network. In this example, a fully connected network - Multi-Layer Perceptron, MLP) is used. The model parameters are updated using the specified optimization algorithm (the local update optimizer of the mobile sensing node preset by the protocol can be stochastic gradient descent, adaptive optimization, etc. In this example, Adaptive Moment Estimation, Adam optimizer is used) to calculate the gradient of the local model. Next, the mobile sensing node generates all random numbers through the pseudo-random generator PRG and adds the sum as a mask to the model gradient to ensure the privacy and unpredictability of the gradient. Then, the masked gradient will be uploaded to the server for subsequent processing.

[0080] Sensing node 1 mask gradient:

[0081] Sensing node 2 mask gradient:

[0082] Sensing node 3 mask gradient:

[0083] Sensing node 4 mask gradient:

[0084] The volunteer node calculates the sum of all its random numbers and sends it to the server:

[0085] Volunteer node 1: R1 = PRG (k 1,1 );

[0086] Volunteer node 2: R2 = ∑PRG(k 2,2 ,,k 2,4 );

[0087] Volunteer node 3: R3 = ∑PRG(k 3,1 ,k 3,2 ,k 3,3 );

[0088] Volunteer node 4: R4 = PRG (k 4,3 );

[0089] Volunteer node 5: R5 = PRG (k 5,2 );

[0090] Volunteer node 6: R6 = ∑PRG(k 6,1 ,,k 6,3 );

[0091] Volunteer node 7: R7 = PRG (k 7,3 );

[0092] Volunteer node 8: R8 = ∑PRG(k 8,1 ,k 8,3 ,k 8,4 );

[0093] (4) Aggregation gradient: After receiving the mask gradients uploaded by all mobile sensing nodes, the server first aggregates the mask sum. By accumulating the mask gradients uploaded by all mobile sensing nodes, the server obtains an encrypted global model gradient. Then, the mask sum (random number and R) of all volunteer nodes is collected. v ), and obtain the true global model gradient by eliminating the mask operation.

[0094] Aggregation Mask:

[0095]

[0096] Calculate the global model gradient X:

[0097]

[0098] (5) Model iteration: The server averages the global model gradient X and feeds this value back to all mobile sensing nodes as the initial parameter for the next round of training until the model converges on all mobile sensing nodes.

[0099] The above description is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, according to the ideas provided by the present invention, there will be changes in the specific implementation methods, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. A random group interactive privacy federated aggregation method, characterized by: Follow these steps: S1, initialization: The initialization process refers to that after all mobile sensing nodes reach an agreement, the server first sends the initial model parameters, and then introduces the volunteer node to perform the protocol initialization operation; S2, random grouping: The random grouping process refers to all participants broadcasting public keys, the mobile sensing node randomly selecting volunteer nodes to form a group, and each volunteer node in the group reaches an agreement with the mobile sensing node to generate a shared key; S3, protection gradient: The gradient protection process refers to obtaining a local model gradient by training the model based on local real data. The mobile sensing node generates a random mask by interacting with all shared keys obtained by the volunteer nodes in the group, and uses the random mask to protect the local model gradient, which is the masked gradient. Step S3 is as follows: After the mobile sensing nodes are grouped, model training is performed based on local real data to obtain local model gradients; each mobile sensing node generates a random mask PRG (k c,v ); The encryption formula of the local gradient of each mobile sensing node is as follows: in, refers to the protected local gradient of the mobile sensing node c, x c Refers to the original local gradient of the mobile sensing node c; For each volunteer node selected into a group, it can interact with multiple mobile sensing nodes in a group; let the set of mobile sensing nodes that select the volunteer node be represented by num c , the volunteer node uses the following formula to calculate the corresponding random number and R v : S4, aggregation gradient: The aggregate gradient process refers to that all mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients; all volunteer nodes upload random number sums to the server, and the server collects and aggregates random number sums from each volunteer node; the real global model gradient is obtained by calculation; Step S4 is as follows: All mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients, where the mask gradient refers to the protected local gradient of the mobile sensing node c; all volunteer nodes upload the sum of mask gradients to the server, and the server collects random numbers and R from each volunteer node. v And aggregate; the true global model gradient is obtained by calculation: Among them, C is the set of all mobile sensing nodes, V is the set of all volunteer nodes, and X is the global model gradient value aggregated by the server; S5, model iteration: The model iteration process refers to the server averaging the global model gradient and feeding back the global model gradient to all mobile sensing nodes as the initial parameters for the next round of training until the model converges on all mobile sensing nodes.

2. The random group interactive privacy federation aggregation method as claimed in claim 1, characterized in that the step In S1, the protocol initialization is as follows: Private key generation: The mobile sensing node and the volunteer node use the predefined prime number p and generator g; each party independently selects a private key, and the private key of the mobile sensing node is denoted as c sk , the private key of the volunteer node is recorded as v sk , mobile sensing nodes and volunteer nodes are randomly selected and less than p; Public key generation: Based on their respective private keys, each party calculates the corresponding public key: Public key calculated by the mobile sensing node: c pk =g csk mod p; Among them, c pk is the public key of the mobile sensing node c; Public key calculated by the volunteer node: v pk =g vsk mod p; Where mod represents the modulus operator, which is defined as the remainder of division; v pk is the public key of volunteer node v.

3. The random group interactive privacy federation aggregation method as claimed in claim 2, characterized in that: Step (2) is as follows: All participants broadcast their public keys, and the mobile sensing node randomly selects volunteer nodes to form a group, where 1≤|num|≤|V|, num refers to the set of volunteer nodes selected by the sensing node, and V refers to the set of all volunteer nodes. Each volunteer node can be selected by different mobile sensing nodes to form a group; each volunteer node v∈num in the group v ∈V, and reach an agreement with the mobile sensing node c to generate a shared key k according to the Diffie-Hellman key exchange mechanism c,v : kc,v=(cpk)v sk mod p=(vpk)c sk mod p; num v Refers to the set of volunteer nodes selected by a mobile sensing node.

4. A random group interactive privacy federated aggregation system, used to execute the method according to any one of claims 1 to 3, characterized in that: Includes mobile sensing nodes, volunteer nodes and servers; Each mobile sensing node has real data for model training, and independently randomly selects any number of volunteer nodes and groups them. Within the group, the mobile sensing node interacts with all volunteer nodes and generates masks. The volunteer nodes reach an agreement with the mobile sensing nodes to generate a shared key. Each volunteer node calculates the sum of all random numbers it processes and sends the aggregated sum value to the server. The server merges the mask gradients transmitted by all mobile sensing nodes and the mask value of the random number provided by the volunteer node.

5. An electronic device, characterized in that: include: processor; The memory is used to store a program, and when the program is called and executed by the processor, the processor executes the method as claimed in any one of claims 1 to 3.

Images