Random grouping interactive privacy federal aggregation method and system and electronic equipment

Abstract

The invention discloses a random grouping interactive privacy federal aggregation method and system and electronic equipment. The method comprises the following steps: all mobile sensing nodes reach a protocol, a server sends an initial parameter, a volunteer node is introduced, and the protocol is initialized; a participant broadcasts a public key, a mobile sensing node randomly selects volunteer nodes to form a group, the volunteer nodes in the group and the mobile sensing node reach a protocol, and a shared key is generated; performing model training according to local real data to obtain a local model gradient, and generating a random mask by a mobile sensing node through all shared keys obtained by interaction with volunteer nodes in a group; the mobile sensing node uploads the mask gradient to the server, and the server calculates the sum of the mask gradient; the volunteer node uploads the mask sum to a server, and the server collects random numbers and aggregates the random numbers; obtaining a global model gradient; and the server averages the global model gradient and feeds back the global model gradient to the mobile sensing node as an initial parameter of the next round of training until the model converges on all the mobile sensing nodes.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and specifically designs a random grouping interactive privacy federation aggregation method, system and electronic equipment for mobile sensing nodes. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, traditional centralized training methods usually rely on aggregating large amounts of data to servers for unified processing. Although this method has obvious advantages in computing efficiency, it poses a significant risk of data privacy leakage in application scenarios involving sensitive data (such as medical data). To solve this problem, federated learning, as an emerging distributed machine learning method, has gradually been applied to the field of privacy data protection. The core advantage of federated learning is that it allows multiple users to perform data processing and model training locally without uploading sensitive data to the cloud or central server. This method can not only effectively protect user privacy, but also significantly reduce the risk of data leakage.

[0003] Mobile sensing nodes are particularly widely used in the framework of federated learning. Mobile sensing nodes usually refer to mobile devices with sensing, computing and communication capabilities, such as smartphones, smart watches, health monitoring devices, etc. They can collect, process and analyze data locally without uploading users' sensitive information to external servers. For example, in medical health monitoring, devices such as smart watches or mobile phones can collect personal health data such as heart rate, number of steps, sleep quality, etc. in real time and perform preliminary processing locally. Then, through the federated learning framework, these devices can share model updates and learned knowledge with other devices without exposing any personal sensitive data. This approach ensures that users' privacy is effectively protected while conducting large-scale distributed learning, reducing the risk of data leakage and abuse.

[0004] However, in the process of federated learning joint training, the model parameters (such as gradient information) uploaded by the mobile sensing nodes may contain sensitive information, so there is still a potential risk of privacy leakage, especially in the communication process between the mobile sensing nodes and the server, attackers may infer the content of the original data by analyzing the uploaded gradient data. At present, some protection protocols have been proposed to enhance the security of federated learning, but the existing technologies have high communication complexity, high local computing cost, and high communication cost consumption, ignoring the need for lightweight and secure model frameworks. Summary of the invention

[0005] In order to solve the problem of model update information exposure caused by naked gradient transmission of mobile sensing nodes, and to solve the problems of high computational complexity and high communication cost caused by complex interactions of mobile sensing nodes, the present invention discloses a random grouped interactive privacy federated aggregation method, system and electronic device.

[0006] The present invention adopts the following technical scheme: The random group interactive privacy federated aggregation method is as follows: (1) Initialization After all mobile sensing nodes reach an agreement, the server first sends the initial model parameters, and then introduces volunteer nodes to perform protocol initialization operations; (2) Random grouping All participants broadcast the public key, the mobile sensing node randomly selects volunteer nodes to form a group, and each volunteer node in the group reaches an agreement with the mobile sensing node to generate a shared key; (3) Mask gradient The local model gradient is obtained by model training based on local real data. The mobile sensing node generates a random mask by interacting with all shared keys obtained by the volunteer nodes in the group. (4) Aggregation gradient All mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients; all volunteer nodes upload mask sums to the server, and the server collects random numbers from each volunteer node and aggregates them; the true global model gradient is obtained through calculation; (5) Model iteration The server averages the global model gradient and feeds it back to all mobile sensing nodes as the initial parameters for the next round of training until the model converges on all mobile sensing nodes.

[0007] Preferably, in step (1), the protocol initialization is specifically as follows: Private key generation: Mobile sensing nodes and volunteer nodes use predefined prime numbers and generators ; Each party independently chooses a private key, and the private key of the mobile sensing node is recorded as , the private key of the volunteer node is recorded as , mobile sensing nodes and volunteer nodes are randomly selected and less than ; Public key generation: Based on their respective private keys, each party calculates the corresponding public key: The public key calculated by the mobile sensing node: ; in, is the public key of the mobile sensing node c; The public key calculated by the volunteer node: ; in, represents the modulo operator, which is defined as taking the remainder of a division; is the public key of volunteer node v.

[0008] Step (2) is as follows: All participants broadcast their public keys, and the mobile sensing node randomly selects volunteer nodes to form a group, where , num refers to the set of volunteer nodes selected by the sensing node, V refers to the set of all volunteer nodes, each volunteer node can be selected into a group by different mobile sensing nodes; each volunteer node in the group , and with the mobile sensing node Reach an agreement and generate a shared key based on the Diffie-Hellman key exchange mechanism : ; Refers to the set of volunteer nodes selected by a mobile sensing node.

[0009] Preferably, step (3) is specifically as follows: after the mobile sensing nodes are grouped, model training is performed based on local real data to obtain local model gradients; each mobile sensing node generates a random mask by interacting with all volunteer nodes in the group to obtain all shared keys. ; The encryption formula of the local gradient of each mobile sensing node is as follows: ; in, Mobile Sensing Node Protected local gradients, Mobile Sensing Node Original local gradient; For each volunteer node selected into a group, it can interact with multiple mobile sensing nodes in a group; let the set of mobile sensing nodes that select the volunteer node be represented as , the volunteer node uses the following formula to calculate the corresponding random number and : .

[0010] Preferably, step (4) is as follows: all mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients, where the mask gradient refers to the local gradient protected by the protocol; all volunteer nodes upload the sum of mask gradients to the server, and the server collects random numbers from each volunteer node. And aggregate them; the true global model gradient is obtained by calculation: ; Among them, C is the set of all mobile sensing nodes, V is the set of all volunteer nodes, and X is the global model gradient value aggregated by the server.

[0011] The present invention also discloses a random group interactive privacy federation aggregation system for executing the above method, which includes a mobile sensing node, a volunteer node and a server; Each mobile sensing node has real data for model training, and independently and randomly selects any number of volunteer nodes and groups them. Within the group, the mobile nodes interact with the volunteer nodes and generate masks. The volunteer nodes reach an agreement with the mobile sensing nodes to generate a shared key. Each volunteer node calculates the sum of all random numbers it processes and sends the aggregated sum value to the server. The server merges the mask gradients transmitted by all mobile sensing nodes and the mask value of the random number provided by the volunteer node.

[0012] The present invention also discloses an electronic device, comprising: processor; The memory is used to store a program. When the program is called and executed by the processor, the processor executes the above method or system.

[0013] In view of the above problems existing in the prior art, the present invention proposes a random group interactive efficient privacy federated aggregation technology solution based on federated learning to protect the gradient information privacy of mobile sensing nodes. Specifically, the mobile sensing node randomly selects any volunteer node and forms a group, uses the Diffie-Hellman protocol to interactively generate a shared key, and uses this key as the seed of the pseudo-random number generator (PRG), generates a mask and adds the sum value to the model gradient, and then uploads it to the server. The server aggregates the mask values ​​generated by all volunteer nodes and removes the mask to obtain the true global model gradient. Compared with the prior art, the present invention effectively protects the model gradient privacy of the mobile sensing node and prevents the privacy leakage problem in the data upload process, thereby improving the security of the mobile terminal in the practical application of federated learning. In addition, in the present invention, the protocol is based on a lightweight cryptographic protocol, and the mobile sensing node can autonomously select any number of volunteer nodes according to its own computing power, without interacting with all the introduced volunteer nodes, so the computing and communication costs of the mobile sensing node can be greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a random group interactive privacy federation aggregation system architecture diagram of a preferred embodiment of the present invention; Figure 2 It is a flow chart of a random group interactive privacy federation aggregation method according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0015] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0016] like Figure 1 As shown, a random group interactive privacy federated aggregation system in this embodiment consists of three entities: a mobile sensing node, a volunteer node and a server.

[0017] Each mobile sensing node ( is a collection of mobile sensing nodes) with a certain amount of real data for model training. According to the protocol, each node voluntarily abides by the protocol rules to promote the joint training process. Specifically, they independently and randomly select any number of volunteer nodes and form groups. Within the group, the mobile nodes interact with the volunteer nodes and generate masks to protect the parameters of their local models, thereby ensuring the privacy of the original data.

[0018] Volunteer nodes (defined as a collection of ) can represent the authorities in each field, for example, in the medical field, it can be hospitals or medical entities conducting research within the healthcare sector, and they neither collude nor participate in any training process. Their main task is to reach a consensus with the client (the mobile sensing node that selected this volunteer node) on the shared key, which is the seed for generating random numbers used to mask the gradient. In each round of the protocol, the selected volunteer node reaches an agreement with the active mobile sensing nodes (online mobile sensing nodes, participating in federated training, and inactive refers to offline nodes) to generate a shared key. Subsequently, each volunteer node calculates the sum of all random numbers it processes and sends the aggregated sum value to the server.

[0019] The server merges the local mask gradients transmitted by all mobile sensing nodes and the mask value of the random number provided by the volunteer node.

[0020] Preliminary knowledge: The pseudo-random generator (PRG) starts with an initial seed value and generates a seemingly random numerical sequence through a deterministic algorithm (hash function, such as SHA-256 function, or other hash functions). This sequence is unpredictable to the observer and is very similar to true random numbers in statistical properties. However, if the generation process and seed value are known, the same sequence can be reproduced. Therefore, in cryptographic applications, it is very important to choose a sufficiently secure seed value to ensure that the generated pseudo-random number sequence cannot be easily predicted or reproduced.

[0021] like Figure 2 As shown, this embodiment discloses a random group interactive privacy federation aggregation method, which is as follows: (1) Initialization After all mobile sensing nodes reach an agreement, federated learning training begins. First, the server sends the initial model parameters, and then introduces volunteer nodes to perform protocol initialization operations: Private key generation: Mobile sensing nodes and volunteer nodes use predefined prime numbers and generators ; Each party independently chooses a private key, and the private key of the mobile sensing node is recorded as , the private key of the volunteer node is recorded as , mobile sensing nodes and volunteer nodes are randomly selected and less than ; Public key generation: Based on their respective private keys, each party calculates the corresponding public key: The public key calculated by the mobile sensing node: ; in, is the public key of the mobile sensing node c; The public key calculated by the volunteer node: ; in, represents the modulo operator, which is defined as taking the remainder of a division; is the public key of volunteer node v.

[0022] (2) Random grouping All participants broadcast their public keys, and the mobile sensing node randomly selects a certain number of volunteers ( ) to prepare for the generation of shared keys. Each volunteer node can be selected into a group by different mobile sensing nodes multiple times. Mobile Sensing Node Reach an agreement and generate a shared key based on the Diffie-Hellman key exchange mechanism : .

[0023] (3) Mask gradient After the mobile sensing nodes are grouped, they train the model based on the local real data to obtain the local model gradient. The mobile sensing nodes generate random masks by interacting with all the shared keys obtained by the volunteers in the group. The encryption formula of the local gradient of each mobile sensing node is as follows, The true model gradient or model parameter for each mobile sensing node: ; For each volunteer node selected into a group, it can interact with multiple mobile sensing nodes in a group. Assume that the mobile sensing node set that selects the volunteer node is represented as The volunteer node calculates the sum of its random numbers using the following formula : ; The mobility-aware nodes will not know the volunteer nodes selected by others, and the volunteer nodes will neither collude nor participate in other computations.

[0024] (4) Aggregation gradient All mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients. All volunteer nodes upload mask sums to the server, and the server collects random numbers from each volunteer node. And aggregate them. The true global model gradient is obtained by calculation: .

[0025] (5) Model iteration The server averages the global model gradient and feeds it back to all mobile sensing nodes as the initial parameters for the next round of training until the model converges on all mobile sensing nodes. The specific convergence condition refers to the threshold of the loss function: when the loss function of the model on the training data reaches a preset small value, the model is considered good enough and training can be stopped. The following is an explanation with reference to specific application cases.

[0026] Taking the health monitoring scenario as an example, there are four existing mobile sensing nodes that perform the task of jointly training the model without disclosing the local data set. Specifically, the mobile sensing node performs local training through the protocol of the present invention and uploads the mask gradient to the server. Finally, the server calculates the global model gradient through the aggregation algorithm commonly used in federated learning. In the protocol registration phase, it is stipulated that the mobile sensing nodes use the same network and optimizer for local training.

[0027] (1) Protocol initialization: The server sends the initial model parameters of the mobile sensing node and introduces 8 volunteer nodes. The mobile sensing node and the introduced volunteer nodes are initialized and public and private keys are generated.

[0028] (2) Random grouping: The mobile sensing node randomly selects any number of volunteer nodes to interact and generate a shared key through the Diffie-Hellman mechanism. ).

[0029] Sensing node 1 selects volunteer nodes 1, 3, 6, and 8 to form a group and generate a shared key ; Sensing node 2 selects volunteer nodes 2, 3, and 5 to form a group and generate a shared key ; Sensing node 3 selects volunteer nodes 3, 4, 6, 7, and 8 to form a group and generate a shared key ; Sensing node 4 selects volunteer nodes 2 and 8 to form a group and generates a shared key ; (3) Masked gradient: The mobile sensing node passes the processed raw data to the local training network (the local training network preset by the protocol can be a fully connected network, a convolutional network or other deep learning network. In this example, a fully connected network - Multi-Layer Perceptron, MLP) is used. The model parameters are updated using the specified optimization algorithm (the local update optimizer of the mobile sensing node preset by the protocol can be stochastic gradient descent, adaptive optimization, etc. In this example, Adaptive Moment Estimation, Adam optimizer is used) to calculate the gradient of the local model. Next, the mobile sensing node generates all random numbers through the pseudo-random generator PRG and adds the sum value as a mask to the model gradient to ensure the privacy and unpredictability of the gradient. Then, the masked gradient will be uploaded to the server for subsequent processing.

[0030] Sensing node 1 mask gradient: ; Sensing node 2 mask gradient: ; Sensing node 3 mask gradient: ; Sensing node 4 mask gradient: ; The volunteer node calculates the sum of all its random numbers and sends it to the server: Volunteer Node 1: ; Volunteer Node 2: ; Volunteer Node 3: ; Volunteer Node 4: ; Volunteer Node 5: ; Volunteer Node 6: ; Volunteer Node 7: ; Volunteer Node 8: ; (4) Aggregation of gradients: After receiving the masked gradients uploaded by all mobile sensing nodes, the server first aggregates the masked sum. By accumulating the masked gradients uploaded by all mobile sensing nodes, the server obtains an encrypted global model gradient. Then, the masked sums of all volunteer nodes are collected, and the true global model gradient is obtained by eliminating the masking operation.

[0031] Aggregation Mask: ; Compute global model gradients : .

[0032] (5) Model iteration: The server averages the global model gradient and feeds this value back to all mobile sensing nodes as the initial parameter for the next round of training until the model converges on all mobile sensing nodes.

[0033] The above description is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, according to the ideas provided by the present invention, there will be changes in the specific implementation methods, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. Randomized group interactive privacy federated aggregation method, characterized by: Follow these steps: S1, initialization: The initialization process refers to that after all mobile sensing nodes reach an agreement, the server first sends the initial model parameters, and then introduces the volunteer node to perform the protocol initialization operation; S2, random grouping: The random grouping process refers to all participants broadcasting public keys, the mobile sensing node randomly selecting volunteer nodes to form a group, and each volunteer node in the group reaches an agreement with the mobile sensing node to generate a shared key; S3, mask gradient: The mask gradient process refers to obtaining a local model gradient by training the model based on local real data, and the mobile sensing node generates a random mask by all the shared keys obtained through interaction with the volunteer nodes in the group; S4, aggregation gradient: The gradient aggregation process refers to all mobile sensing nodes uploading mask gradients to the server, and the server aggregates and calculates the sum of mask gradients; all volunteer nodes upload the mask sum to the server, and the server collects and aggregates random numbers from each volunteer node; The true global model gradient is obtained by calculation; S5, model iteration: The model iteration process refers to the server averaging the global model gradient and feeding back the global model gradient to all mobile sensing nodes as the initial parameters for the next round of training until the model converges on all mobile sensing nodes.

2. The random group interactive privacy federation aggregation method as claimed in claim 1, characterized in that the step In S1, the protocol initialization is as follows: Private key generation: Mobile sensing nodes and volunteer nodes use predefined prime numbers and generators ; Each party independently chooses a private key, and the private key of the mobile sensing node is recorded as , the private key of the volunteer node is recorded as , mobile sensing nodes and volunteer nodes are randomly selected and less than ; Public key generation: Based on their respective private keys, each party calculates the corresponding public key: The public key calculated by the mobile sensing node: ; in, is the public key of the mobile sensing node c; The public key calculated by the volunteer node: ; in, represents the modulo operator, which is defined as taking the remainder of a division; is the public key of volunteer node v.

3. The random group interactive privacy federation aggregation method as claimed in claim 2, characterized in that: Step (2) is as follows: All participants broadcast their public keys, and the mobile sensing node randomly selects volunteer nodes to form a group, where , num refers to the set of volunteer nodes selected by the sensing node, V refers to the set of all volunteer nodes, each volunteer node can be selected into a group by different mobile sensing nodes; each volunteer node in the group , and with the mobile sensing node Reach an agreement and generate a shared key based on the Diffie-Hellman key exchange mechanism : ； Refers to the set of volunteer nodes selected by a mobile sensing node.

4. The random group interactive privacy federation aggregation method as claimed in claim 3, characterized in that: Step S3 is as follows: After the mobile sensing nodes are grouped, the model is trained based on the local real data to obtain the local model gradient; each mobile sensing node generates a random mask by interacting with all the shared keys obtained by the volunteer nodes in the group. ; The encryption formula of the local gradient of each mobile sensing node is as follows: ； in, Mobile Sensing Node Protected local gradients, Mobile Sensing Node Original local gradient; For each volunteer node selected into a group, it can interact with multiple mobile sensing nodes in a group; let the mobile sensing node set that selects the volunteer node be represented as , the volunteer node uses the following formula to calculate the corresponding random number and : 。 5. The random group interactive privacy federation aggregation method as claimed in claim 4, characterized in that: Step S4 is as follows: all mobile sensing nodes upload mask gradients to the server, and the server aggregates and calculates the sum of mask gradients, where the mask gradient refers to the local gradient protected by the protocol; all volunteer nodes upload the sum of mask gradients to the server, and the server collects random numbers from each volunteer node. And aggregate; the true global model gradient is obtained by calculation: ； Among them, C is the set of all mobile sensing nodes, V is the set of all volunteer nodes, and X is the global model gradient value aggregated by the server.

6. A random group interactive privacy federated aggregation system, used to execute the method according to any one of claims 1 to 5, characterized in that: Includes mobile sensing nodes, volunteer nodes and servers; Each mobile sensing node has real data for model training, and independently randomly selects any number of volunteer nodes and groups them. Within the group, the mobile sensing node interacts with all volunteer nodes and generates masks. The volunteer nodes reach an agreement with the mobile sensing nodes to generate a shared key. Each volunteer node calculates the sum of all random numbers it processes and sends the aggregated sum value to the server. The server merges the mask gradients transmitted by all mobile sensing nodes and the mask value of the random number provided by the volunteer node.

7. An electronic device, characterized in that: include: processor; A memory for storing a program, wherein when the program is called and executed by a processor, the processor executes the method according to any one of claims 1 to 5 or the system according to claim 6.

Images