Data security aggregation method and device, equipment and storage medium

Abstract

The invention discloses a data security aggregation method, apparatus and device, and a storage medium. The method comprises the steps of obtaining a noised local data set sent by user equipment; wherein the local data set is obtained by adding noise into an original local data set of a user by the user equipment; according to the local data set after noise addition and a local model issued by the server, local model parameters of the local model are updated, and the updated local model parameters are uploaded to the server, so that the server aggregates all the updated local model parameters to obtain corresponding global model parameters; wherein the local model is a CNN (Convolutional Neural Network) model with He initialization, and the weight of the He initialization is obtained by carrying out random sampling from preset Gaussian distribution. According to the invention, the privacy security of uploading the data to the third party can be improved, and the security of the data is ensured.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to a data security aggregation method, device, equipment and storage medium. Background Art

[0002] At present, with the rapid development of artificial intelligence, machine learning, as a core technology for processing massive data, has received widespread attention for its privacy. In the training and testing process of traditional machine learning, the computing power of user devices is relatively weak, so many studies have added edge nodes to train user data. However, since most solutions simply upload user data directly to edge nodes without any privacy protection for the data, semi-honest edge nodes learn user data while ensuring the security of the protocol. The data may include sensitive information of users. If this information is attacked for privacy, personal safety and property safety will be threatened. This indirectly leads to individual users being unwilling to disclose their own user information. At the same time, data is stored in isolation on servers within the enterprise, and data is not shared between enterprises, resulting in poor data quality, hindering machine learning from training valuable models, and causing data islands. Summary of the invention

[0003] The present invention provides a data security aggregation method, device, equipment and storage medium to solve the technical problem.

[0004] In order to solve the above technical problems, an embodiment of the present invention provides a data security aggregation method, including:

[0005] Acquire a noisy local data set sent by a user device; wherein the local data set is obtained by the user device determining a user's privacy budget according to the user's privacy security level, then generating corresponding noise according to the privacy budget, and adding the noise to the user's original local data set;

[0006] According to the noisy local data set and the local model sent by the server, the local model parameters of the local model are updated, and the updated local model parameters are uploaded to the server, so that the server aggregates all the updated local model parameters to obtain corresponding global model parameters;

[0007] Among them, the local model is a CNN model with He initialization, the He-initialized weights of the CNN model are randomly sampled from a preset Gaussian distribution, and the variance of the Gaussian distribution is calculated based on the number of input neurons and the number of output neurons in each layer of the CNN model.

[0008] As a preferred solution, the updating of the local model parameters of the local model according to the noisy local data set and the local model sent by the server includes:

[0009] Divide the noisy local dataset into several batches;

[0010] According to the local data in each batch of local data sets, the preset loss function and the local model sent by the server, the loss function value of the local model under the current local model parameters is calculated, and the gradient of the loss function with respect to the current local model parameters is calculated according to the loss function value;

[0011] According to the gradient and the preset learning rate, the local model parameters of the local model are updated. As a preferred solution, the local data set after adding noise is:

[0012] Noise_data=data+LaplaceNoise(Δf,ε i );

[0013] ε i =β×(1-S i );

[0014] Among them, Noise_data is the local data set after adding noise, data is the initial local data set, ε i is the privacy budget, Δf is the sensitivity, β is the preset adjustment coefficient, S i The privacy security level of the user.

[0015] As a preferred solution, the weight of He initialization is:

[0016]

[0017] Among them, W l is the weight matrix of the lth layer, The mean is 0 and the standard deviation is The normal distribution of .

[0018] As a preferred solution, uploading the updated local model parameters to the server includes:

[0019] A double mask is added to the updated local model parameters, and then the local model parameters with the double mask added are uploaded to the server.

[0020] As a preferred solution, the above-mentioned aggregating all updated local model parameters to obtain corresponding global model parameters includes:

[0021] According to the preset secret sharing algorithm, the double masks in all updated local model parameters are removed, and then the local model parameters after the double masks are removed are aggregated to obtain the corresponding global model parameters.

[0022] Based on the above embodiment, another embodiment of the present invention provides a data security aggregation device, including: a local data set acquisition module and a model parameter update module;

[0023] The local data set acquisition module is used to acquire the noisy local data set sent by the user device; wherein the local data set is obtained by the user device determining the user's privacy budget according to the user's privacy security level, then generating corresponding noise according to the privacy budget, and adding the noise to the user's original local data set;

[0024] The model parameter updating module is used to update the local model parameters of the local model according to the noisy local data set and the local model sent by the server, and upload the updated local model parameters to the server, so that the server aggregates all the updated local model parameters to obtain the corresponding global model parameters;

[0025] Among them, the local model is a CNN model with He initialization, the He-initialized weights of the CNN model are randomly sampled from a preset Gaussian distribution, and the variance of the Gaussian distribution is calculated based on the number of input neurons and the number of output neurons in each layer of the CNN model.

[0026] As a preferred solution, the updating of the local model parameters of the local model according to the noisy local data set and the local model sent by the server includes:

[0027] Divide the noisy local dataset into several batches;

[0028] According to the local data in each batch of local data sets, the preset loss function and the local model sent by the server, the loss function value of the local model under the current local model parameters is calculated, and the gradient of the loss function with respect to the current local model parameters is calculated according to the loss function value;

[0029] The local model parameters of the local model are updated according to the gradient and a preset learning rate.

[0030] Based on the above embodiments, another embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the data security aggregation method described in the above embodiments of the invention is implemented.

[0031] Based on the above embodiments, another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the data security aggregation method described in the above invention embodiment.

[0032] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0033] The present invention provides a data security aggregation method, which first obtains a noisy local data set sent by a user device; then, based on the noisy local data set and a local model sent by a server, updates the local model parameters of the local model, and uploads the updated local model parameters to the server, so that the server aggregates all the updated local model parameters to obtain corresponding global model parameters.

[0034] Among them, in view of the situation that semi-honest or malicious edge nodes may be curious about the collected user data and even try to directly obtain the user's sensitive information, before uploading the local data set, the user device first determines the user's privacy budget according to the user's privacy security level, and then generates corresponding noise according to the determined privacy budget, and adds noise to the local data set to solve the problem of semi-honest or malicious edge nodes obtaining user sensitive information, improve the privacy security of data uploaded to a third party, and ensure the security of the data.

[0035] However, when training the local model at the edge node, the noise is added to the local data set, which will cause the local model to converge slowly and require more iterations to reach the local optimum. In order to improve the convergence speed of the local model, the present invention also adopts the He initialization method and uses a CNN model with He initialization. The He-initialized weights of the CNN model are randomly sampled from a preset Gaussian distribution, and the variance of the weights is adjusted according to the number of input and output neurons in each layer to ensure the stability of the activation value in the forward propagation, reduce the performance loss caused by differential privacy, and improve the convergence speed of the local model of each edge node. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of a data security aggregation method provided by an embodiment of the present invention;

[0037] Figure 2 It is a model diagram of the data aggregation process of the present invention;

[0038] Figure 3 is a flow chart of a specific embodiment of the present invention;

[0039] Figure 4It is a structural schematic diagram of a data security aggregation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0042] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0043] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0045] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0046] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0047] Embodiment 1

[0048] Please refer to Figure 1 , is a flow chart of a data security aggregation method provided by an embodiment of the present invention, comprising the following specific steps:

[0049] S1. Obtain a noisy local data set sent by a user device; wherein the local data set is obtained by the user device determining a privacy budget of the user according to the privacy security level of the user, then generating corresponding noise according to the privacy budget, and adding the noise to the original local data set of the user;

[0050] Preferably, the local model parameters of the local model are updated according to the noisy local data set and the local model sent by the server, including: dividing the noisy local data set into several batches; calculating the loss function value of the local model under the current local model parameters according to the local data in each batch of the local data set, a preset loss function and the local model sent by the server, and calculating the gradient of the loss function with respect to the current local model parameters according to the loss function value; and updating the local model parameters of the local model according to the gradient and a preset learning rate.

[0051] Preferably, the local data set after adding noise is:

[0052] Noise_data=data+LaplaceNoise(Δf,ε i );

[0053] ε i =β×(1-S i );

[0054] Among them, Noise_data is the local data set after adding noise, data is the initial local data set, ε i is the privacy budget, Δf is the sensitivity, β is the preset adjustment coefficient, S i The privacy security level of the user.

[0055] Preferably, the weight of He initialization is:

[0056]

[0057] Among them, W l is the weight matrix of the lth layer, The mean is 0 and the standard deviation is The normal distribution of .

[0058] For details, please refer to Figure 2 , which is a model diagram of the data aggregation process of the present invention. First, before the user uploads the local data set, the present invention allocates a privacy budget to the user according to the user's privacy security level and adds the generated noise to the data; secondly, in order to improve the local model convergence speed of each edge node, the He initialization method is adopted to reduce the performance loss caused by differential privacy; finally, in order to ensure the security aggregation of the model, the parameter security is protected by adding a mask to the parameter. At the same time, in order to prevent malicious edge nodes from tampering with the received shared shares, verifiable secret sharing is adopted so that the edge nodes can transmit correctly to ensure the validity of the aggregated value. The general steps are as follows:

[0059] (1) Before training the initial model, the server collects data sets from some loyal users, trains the initial model based on these data, obtains the initial model parameters, and sends them to each edge node.

[0060] (2) Each edge node allocates a privacy budget to nearby user devices. The user device then generates noise based on ε and sensitivity f and adds it to its dataset. Finally, each user device uploads the noisy dataset to the edge node. After uploading, the user device can be offline without occupying limited computing resources for training tasks. The sensitivity of the user device is maintained locally by the device without notifying other devices and edge nodes.

[0061] (3) When training the model at the edge nodes, the local model converges slowly due to the noise added to the data, and more iterations are required to reach the local optimum. In order to improve the convergence speed of the model, this paper adopts the He initialization method to adjust the variance of the weights according to the number of input and output neurons in each layer, ensuring the stability of the activation value in the forward propagation and reducing the performance loss caused by differential privacy.

[0062] (4) When edge nodes transmit model parameters, malicious attackers can calculate model gradients through changes in model parameters. To protect data privacy, we add a verifiable secret sharing algorithm to verify the key and achieve secure aggregation results through double masking.

[0063] Please refer to Figure 3, is a flowchart of a specific implementation of the present invention, and the specific implementation of the above steps is described below:

[0064] 1. Initialization

[0065] A trusted third party (TTP) generates a public-private key pair for each edge node It is used for identity and message authentication. At the same time, TTP generates a secure prime number k, which has the following characteristics: k-1 must have a large prime factor, and the number of bits of k must be greater than 1024; there is a number l, which is a prime number that satisfies k=2l+1;

[0066] Specify a finite field F for the secret sharing algorithm;

[0067] Establish communication channels to achieve communication interconnection at all levels;

[0068] In a verifiable secret sharing protocol, we set a value n and a threshold t, and choose The q-order subgroup of g is a multiplicative group. p-order elements of , where p and q are large prime numbers, and q divides p-1. These system parameters are public;

[0069] All edge nodes use the secure prime number k to generate parameters pp=KA·param(k) for the DH key exchange protocol.

[0070] 2. Noising the Local Dataset

[0071] In order to solve the problem that semi-honest or malicious edge nodes may be curious about the collected user data and even try to directly obtain the user's sensitive information, the present invention designs a personalized local differential privacy protection mechanism, which aims to enhance the privacy security of user data by applying different noises to the user data. This mechanism ensures that even if the data is collected, the user's sensitive information will not be easily exposed to the edge nodes, thereby ensuring the security of user data.

[0072] First, each user defines a privacy security level S i , where i represents the user, and this privacy security level is based on the user's preference P i , Data sensitivity D i To determine, the definition formula is as follows:

[0073] S i =α·P i +(1-α)·D i (1)

[0074] Here, α is a weight parameter used to balance the impact of user privacy preference and data sensitivity. If data sensitivity and user privacy preference are considered equally important, α = 0.5.

[0075] Second, for the local differential privacy mechanism, we need to assign a privacy budget ε to each user i The privacy budget determines the amount of noise added ε i The smaller the value, the more noise is added, and the higher the privacy protection of the data is, but it may reduce the practicality of the data, so it is particularly important to select an appropriate privacy budget. The allocation of the privacy budget is based on the user's privacy security level and is obtained according to the logarithmic mapping function. The formula for the privacy budget is as follows:

[0076] ε i =β×(1-S i ) (2)

[0077] Among them, β is an adjustment coefficient that protects data security as much as possible when the data is available.

[0078] Finally, the user device is allocated according to the privacy budget ε i And sensitivity Δf generates noise and adds it to its data set. The noise adding formula is as follows:

[0079] Noise_data=data+LaplaceNoise(Δf,ε i ) (3)

[0080] After uploading the noisy data to the edge node, the user device can be offline without occupying limited computing resources for training tasks. The sensitivity of the user device is maintained locally on the device without notifying other devices and edge nodes.

[0081] 3. Local Model Training

[0082] Adding differential privacy to data brings more performance loss. In order to speed up the convergence of the model, He initialization is used in CNN neural network. He initialization is a weight initialization method in deep learning, especially suitable for networks using ReLU (Rectified Linear Unit) activation function and its variants. The key to He initialization is that in the initialization stage, the scale of the weight is adjusted according to the number of nodes in the previous layer to ensure that the signal in the deep layer of the network is not too large or too small. Specifically, the weight is set from 0 as the mean to 0. is randomly drawn from a normal distribution with standard deviation, where n l is the number of nodes in the previous layer.

[0083] First, the server sends a CNN model with He initialization to the edge node and initializes the model parameters. The initialization formula of He is:

[0084]

[0085] Among them, W l represents the weight matrix of the lth layer, The mean is 0 and the standard deviation is According to this formula, when the number of nodes in the previous layer increases, the variance of the weight matrix will decrease, which means that the initial value of the weight will be closer to 0, which will help improve the convergence speed of the model and avoid performance loss caused by noise addition.

[0086] Secondly, the edge node selects data from the noisy data set in batches, calculates the loss function value under the current model parameters for each batch of data, and then calculates the gradient of the loss function with respect to the model parameters based on the loss function value. Finally, the model parameters are updated through the SGD method based on the gradient and learning rate. That is, adjust the parameters in the direction of reducing the loss. After completing one round of iteration, repeat this process until all rounds of iteration are completed.

[0087] S2. According to the noisy local data set and the local model sent by the server, the local model parameters of the local model are updated, and the updated local model parameters are uploaded to the server, so that the server aggregates all the updated local model parameters to obtain corresponding global model parameters;

[0088] Among them, the local model is a CNN model with He initialization, the He-initialized weights of the CNN model are randomly sampled from a preset Gaussian distribution, and the variance of the Gaussian distribution is calculated based on the number of input neurons and the number of output neurons in each layer of the CNN model.

[0089] Preferably, uploading the updated local model parameters to the server includes: adding a double mask to the updated local model parameters, and then uploading the local model parameters with the double mask added to the server.

[0090] Preferably, the aggregating all updated local model parameters to obtain corresponding global model parameters includes: removing double masks from all updated local model parameters according to a preset secret sharing algorithm, and then aggregating the local model parameters after removing the double masks to obtain corresponding global model parameters.

[0091] 4. Security Aggregation

[0092] When the edge node model training is completed, the local model parameter vector is obtained. Afterwards, the parameters will be masked and sent to the server for aggregation. However, there are malicious edge nodes that receive and The shared shares are tampered, resulting in the failure of the server to recover the key, which in turn leads to the failure of the model parameter aggregation value. Therefore, a secure aggregation scheme based on Pedersen commitment is proposed. The specific secure aggregation process is as follows:

[0093] Round 0:

[0094] Edge node u i :For each edge node u i Generate two pairs of public and private key pairs based on pp: and Then use the digital signature algorithm to sign the two public keys to generate The two generated public keys together with the signature result are Sent to the server through the authentication channel.

[0095] Server: The server checks whether the number of messages collected in all edge nodes is less than t. If it is less than t, the algorithm is terminated, and the set of edge nodes that receive the message is recorded as U1. The server broadcasts its own ID and the IDs of other edge nodes, two public keys and signature results to all edge nodes in U1, that is,

[0096] Round 1:

[0097] Edge node u i : Received from the server In order to verify |U1|≥t and prevent the server from forging edge nodes to obtain edge node data, first perform message verification on all edge nodes in U1:

[0098]

[0099] Since the public key and private key are distributed by a trusted third party, if the server forges the edge node, then this equation will not hold and the edge node will terminate the algorithm.

[0100] Randomly pick one It is used to generate seeds for pseudo-random generators, and the secret share is calculated by a verifiable secret sharing algorithm as shown in formula (6). Similarly, the private key The secret sharing share calculation process is as follows:

[0101]

[0102] At the same time, in order to make these shares verifiable, for each share and Independently selected random numbers and Calculate the corresponding commitment:

[0103]

[0104]

[0105] For other edge nodes in U1, authentication encryption technology is used for each edge node. The key used is the result of key negotiation between two edge nodes.

[0106]

[0107] All as well as and The promise is transmitted to the server, and Contains u i ,u j address data.

[0108] Server side: The server checks whether the number of collected messages in all edge nodes is less than t. If it is less than t, the algorithm is terminated and the set of edge nodes that receive the message is recorded as U2, where And the received The messages are broadcast to the corresponding clients respectively, but the promises are not sent to the clients.

[0109] Round 2:

[0110] Edge node u i : Collect the corresponding information from the server and other edge nodes Get the set U2 and compare it with t. If it is less than t, terminate the algorithm.

[0111] For each edge node u i , calculate the shared mask between it and other clients in U2, and obtain the key through key negotiation, Then use this key as a seed and send it to PRG to generate m vectors of the same size as the privacy vector as the mask vector. The formula is as follows:

[0112]

[0113] When u i >u j , When u i <u j , and and are opposite numbers, and

[0114] Compute the personal mask vector for each edge node:

[0115]

[0116] The privacy vector is then added to the personal mask vector and the shared mask vector to hide personal data:

[0117]

[0118] If any operation in the above process, such as key negotiation or PRG operation, fails, the operation will be terminated directly. Otherwise, the generated Send to the server.

[0119] Server side: The server checks whether the number of collected messages in all edge nodes is less than t. If it is less than t, the algorithm is terminated and the set of edge nodes that receive the message is recorded as U3, where And send the U3 list to each edge node.

[0120] Round 3 (Consistency Check):

[0121] Edge node u i : Collect the received client set U3 and check whether the size of U3 is greater than t. If it is less than t, terminate the algorithm directly.

[0122] If it is greater than or equal to t, then U3 is signed with a digital signature as follows to obtain a

[0123]

[0124] The purpose of this signature is to prevent the server from maliciously deceiving most clients in the previous step to obtain information. If the server deliberately creates the illusion that a client is offline in the previous step, the U3 received by the client that wants to obtain information and other clients will be different. This will cause problems with the signature and allow the client to discover the server's fraud.

[0125] Server side: Collect all Confirm whether the number of received messages is greater than t, otherwise terminate. If it is greater than t, then this part of users is recorded as U4, and For each user in the set, send the corresponding signature result

[0126] Round 4:

[0127] Edge node: receives the signature result from the server confirm And |U4|≥t, then confirm For all uj ∈U4, this step is mainly to prevent the server from deliberately deceiving the client to obtain client information. If any of the above links goes wrong, the algorithm will be terminated directly.

[0128] In U2, except u i Among the other clients, for each client u j , you can use the key to decrypt the authenticated encrypted We get these four results, the first two of which first verify whether they satisfy u i =u i ′∧u j =u′ j If it does not meet the requirements, it means that the information is sent incorrectly or lost, and the algorithm is terminated directly. Otherwise, different key shares will be sent according to the needs of the server. or

[0129] Server: Collect at least t pieces of information from the client, this group of clients is recorded as U5, otherwise the algorithm is terminated. j ∈U2\U3, that is, the offline user collects the key sharing shares related to the offline edge node from the online edge node And verify whether the commitment share obtained by formula (8) is correct. If the verified share is incorrect, it will be regarded as a disconnected edge node and continue to collect relevant shares from the online edge node and verify. If the verification is successful, the key will be restored. And send it to PRG to calculate all the A normal edge node will send And verify whether the share is correct according to formula (9). If the verification is successful, restore the seed b ui And send it to PRG to calculate

[0130] By eliminating the private mask and public mask Afterwards, the aggregated result is obtained:

[0131]

[0132] The new global model parameters are obtained by averaging the sum of the aggregated model parameters:

[0133]

[0134] Finally, the server returns the global model parameters to each edge node for the next training.

[0135] It can be seen that the present invention provides a method for secure data aggregation, which improves the security of data upload based on local differential privacy technology, and allows secure parameter aggregation based on verifiable secret sharing and double mask technology, and can be used for secure aggregation tasks in the training process of sensitive data.

[0136] In addition, the present invention can also be used as a training data solution on an untrusted third party, adding local differential privacy to the user's local data, greatly improving the privacy security of data uploaded to a third party. And for training model parameters, it is only required to upload the parameters to a third-party server and securely aggregate them. By introducing verifiable secret sharing, the effectiveness of the model parameter aggregation value is greatly improved.

[0137] Embodiment 2

[0138] Please refer to Figure 4 , is a structural diagram of a data security aggregation device provided by an embodiment of the present invention, the device comprising: a local data set acquisition module and a model parameter update module;

[0139] The local data set acquisition module is used to acquire the noisy local data set sent by the user device; wherein the local data set is obtained by the user device determining the user's privacy budget according to the user's privacy security level, then generating corresponding noise according to the privacy budget, and adding the noise to the user's original local data set;

[0140] The model parameter updating module is used to update the local model parameters of the local model according to the noisy local data set and the local model sent by the server, and upload the updated local model parameters to the server, so that the server aggregates all the updated local model parameters to obtain the corresponding global model parameters;

[0141] Among them, the local model is a CNN model with He initialization, the He-initialized weights of the CNN model are randomly sampled from a preset Gaussian distribution, and the variance of the Gaussian distribution is calculated based on the number of input neurons and the number of output neurons in each layer of the CNN model.

[0142] Preferably, the local model parameters of the local model are updated according to the noisy local data set and the local model sent by the server, including: dividing the noisy local data set into several batches; calculating the loss function value of the local model under the current local model parameters according to the local data in each batch of the local data set, a preset loss function and the local model sent by the server, and calculating the gradient of the loss function with respect to the current local model parameters according to the loss function value; and updating the local model parameters of the local model according to the gradient and a preset learning rate.

[0143] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0144] Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0145] Embodiment 3

[0146] Accordingly, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the data security aggregation method described in the above-mentioned embodiment of the invention when executing the computer program.

[0147] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The device may include, but is not limited to, a processor and a memory.

[0148] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the device, and various interfaces and lines are used to connect various parts of the entire device.

[0149] Embodiment 4

[0150] Accordingly, an embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the data security aggregation method described in the above-mentioned embodiment of the invention.

[0151] The memory can be used to store the computer program, and the processor realizes various functions of the device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0152] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each method embodiment described above can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0153] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A data security aggregation method, characterized in that: include: Acquire a noisy local data set sent by a user device; wherein the local data set is obtained by the user device determining a user's privacy budget according to the user's privacy security level, then generating corresponding noise according to the privacy budget, and adding the noise to the user's original local data set; According to the noisy local data set and the local model sent by the server, the local model parameters of the local model are updated, and the updated local model parameters are uploaded to the server, so that the server aggregates all the updated local model parameters to obtain corresponding global model parameters; Among them, the local model is a CNN model with He initialization, the He-initialized weights of the CNN model are randomly sampled from a preset Gaussian distribution, and the variance of the Gaussian distribution is calculated based on the number of input neurons and the number of output neurons in each layer of the CNN model.

2. The data security aggregation method according to claim 1, characterized in that: The updating of the local model parameters of the local model according to the noisy local data set and the local model sent by the server includes: Divide the noisy local dataset into several batches; According to the local data in each batch of local data sets, the preset loss function and the local model sent by the server, the loss function value of the local model under the current local model parameters is calculated, and the gradient of the loss function with respect to the current local model parameters is calculated according to the loss function value; The local model parameters of the local model are updated according to the gradient and a preset learning rate.

3. The data security aggregation method according to claim 1, characterized in that: The local data set after adding noise is: Noise_data=data+LaplaceNoise(Δf,ε i ); e i =β×(1-S i ); Wherein, Noise_data=data is the original Noise data of ap after adding ta noise + L (Δ_data fd, a set εtia), =data+ is the initial Noise data of La place (Δfdata, ε i Set), ε i is the privacy budget, Δf is the sensitivity, β is the preset adjustment coefficient, S i The privacy security level of the user.

4. The data security aggregation method according to claim 3, characterized in that: The weight of He initialization is: Among them, W l is the weight matrix of the lth layer, The mean is 0 and the standard deviation is The normal distribution of .

5. The data security aggregation method according to claim 1, characterized in that: The step of uploading the updated local model parameters to the server includes: A double mask is added to the updated local model parameters, and then the local model parameters with the double mask added are uploaded to the server.

6. The data security aggregation method according to claim 1, characterized in that: The method aggregates all updated local model parameters to obtain corresponding global model parameters, including: According to the preset secret sharing algorithm, the double masks in all updated local model parameters are removed, and then the local model parameters after the double masks are removed are aggregated to obtain the corresponding global model parameters.

7. A data security aggregation device, characterized in that: include: Local data set acquisition module and model parameter update module; The local data set acquisition module is used to acquire the noisy local data set sent by the user device; wherein the local data set is obtained by the user device determining the user's privacy budget according to the user's privacy security level, then generating corresponding noise according to the privacy budget, and adding the noise to the user's original local data set; The model parameter updating module is used to update the local model parameters of the local model according to the noisy local data set and the local model sent by the server, and upload the updated local model parameters to the server, so that the server aggregates all the updated local model parameters to obtain the corresponding global model parameters; Among them, the local model is a CNN model with He initialization, the He-initialized weights of the CNN model are randomly sampled from a preset Gaussian distribution, and the variance of the Gaussian distribution is calculated based on the number of input neurons and the number of output neurons in each layer of the CNN model.

8. The data security aggregation device according to claim 7, characterized in that: The updating of the local model parameters of the local model according to the noisy local data set and the local model sent by the server includes: Divide the noisy local dataset into several batches; According to the local data in each batch of local data sets, the preset loss function and the local model sent by the server, the loss function value of the local model under the current local model parameters is calculated, and the gradient of the loss function with respect to the current local model parameters is calculated according to the loss function value; The local model parameters of the local model are updated according to the gradient and a preset learning rate.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the data security aggregation method as described in any one of claims 1 to 6 when executing the computer program.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the data security aggregation method as described in any one of claims 1 to 6.

Images