A malicious security federated learning method based on data enhancement and L2 enhancement defense

Abstract

The present invention relates to the technical field of privacy computing applications, and proposes a malicious security federated learning method based on data enhancement and L2 enhancement defense, comprising the following steps: local clients each generate average sample data, and encrypt and send them to a server; each local client trains its local model in combination with the encrypted average sample data and local data received from the server, obtains a gradient vector of the trained local model, and encrypts and sends the gradient vector to the server; the server retains the gradient vector whose component length does not exceed a preset component length threshold, calculates the L2 norm corresponding to the gradient value of the retained gradient vector, and obtains the gradient L2 norm; the gradient values ​​whose gradient L2 norm exceeds a preset upper limit are filtered, and the global model is updated using the final retained gradient value; the method can improve the performance of the global model, protect privacy, and reduce the risk of model destruction by resisting gradient increase.

Description

Technical Field

[0001] The present invention relates to the technical field of privacy computing applications, and more specifically, to a malicious security federated learning method based on data enhancement and L2 enhancement defense. Background Art

[0002] The rapid development of the Internet of Things has generated a large amount of data that can be used. Federated learning, as a privacy protection technology, allows multiple participants to jointly train a global model without uploading local data, thus achieving the safe circulation and use of data. Since federated learning does not require uploading local data during use, it is conducive to protecting personal privacy data and solves the problem that data cannot be transmitted to the server due to privacy issues, legal restrictions or large data volumes.

[0003] However, in actual use, different participants have different preferences, so different clients generate different data. During training, the client obtains a local model with large local differences, and finally obtains a global model with poor training results. In addition, when the gradient of federated learning is updated, the gradient information submitted by the client may also leak information about its local data set. Therefore, in order to make the data usable in privacy-sensitive applications, federated learning should protect the privacy information of each gradient, thereby protecting the privacy information of the data set. In addition, in the updated gradient submitted by the client, in the absence of any defense measures, a single deformed gradient (for example: with a very high norm) may also cause a huge deviation in the global model. Malicious clients can improve their gradients (expand to a large norm) to destroy the global model, and many model poisoning attacks rely on this strategy.

[0004] In summary, in the process of using federated learning, the existing technology has the problem that the global model performance is poor due to the large local differences of the local model, and there is a risk of gradient information leakage and malicious clients using gradients to increase and destroy the global model. Summary of the invention

[0005] In order to overcome the defects of the above-mentioned prior art in that the global model performance is poor, and the risks of information leakage and attack are relatively high, the present invention provides a malicious security federated learning method based on data enhancement and L2 enhancement defense, which has strong global model performance, can protect privacy, and reduce the risk of model destruction by resisting gradient increase.

[0006] To solve the above technical problems, the technical solution of the present invention is implemented based on several local clients and servers, and the steps are as follows:

[0007] The local clients each generate average sample data, and encrypt the average sample data and send it to the server;

[0008] The server sends the encrypted average sample data of several local clients to each local client;

[0009] Each local client trains the global model by combining the received encrypted average sample data and its own local data to obtain a gradient vector of the trained global model, and sends the gradient vector to the server using encryption technology;

[0010] The server checks the component lengths of the received gradient vectors, retains the gradient vectors whose component lengths do not exceed a preset component length threshold, and calculates the L2 norm corresponding to the gradient value of the retained gradient vector to obtain the gradient L2 norm;

[0011] The server regards the gradient value whose gradient L2 norm exceeds a preset upper limit as a malicious gradient, filters the malicious gradient to obtain the final retained gradient value, and uses the final retained gradient value to update the global model.

[0012] The present invention also proposes a malicious security federated learning system based on data enhancement and L2 enhancement defense for implementing the above-mentioned malicious security federated learning method based on data enhancement and L2 enhancement defense, the system comprising:

[0013] An average sample data generating module, used for generating average sample data using the local clients respectively, and encrypting the average sample data and sending it to the server;

[0014] An encrypted information distribution module, used to send the encrypted average sample data of several local clients to each local client using the server;

[0015] A gradient vector generation module is used to train the global model using each local client in combination with the received encrypted average sample data and its own local data to obtain a gradient vector of the trained global model, and send the gradient vector to the server using encryption technology;

[0016] a gradient vector filtering module, configured to use the server to check the component lengths of the received gradient vectors, retain the gradient vectors whose component lengths do not exceed a preset component length threshold, and calculate the L2 norm corresponding to the gradient value of the retained gradient vectors to obtain the gradient L2 norm;

[0017] The global model updating module is used to use the server to regard the gradient value whose gradient L2 norm exceeds the preset upper limit as a malicious gradient, filter the malicious gradient, obtain the final retained gradient value, and use the final retained gradient value to update the global model.

[0018] The present invention also proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the above-mentioned malicious security federated learning method based on data enhancement and L2 enhancement defense.

[0019] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0020] Each local client of the present invention trains its local model in combination with the received encrypted average sample data and its own local data, which can effectively avoid the problem of large local differences in local models corresponding to different local clients and effectively improve the performance of the global model; and use encryption technology to send the gradient vector to the server to reduce the risk of gradient information leakage; in addition, by only retaining the gradient vector whose component length does not exceed the preset component length threshold, and further filtering the gradient value whose gradient L2 norm exceeds the preset upper limit, the risk of malicious clients using gradient increase to destroy the global model is effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a first flow chart of the malicious security federated learning method based on data enhancement and L2 enhancement defense proposed in Example 1;

[0022] Figure 2 A second flow chart of the malicious security federated learning method based on data enhancement and L2 enhancement defense proposed in Example 2;

[0023] Figure 3 This is an overall framework diagram of a malicious security federated learning system based on data enhancement and L2 enhancement defense proposed in Example 3. DETAILED DESCRIPTION

[0024] The accompanying drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0025] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0026] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0028] Example 1

[0029] This embodiment proposes a malicious security federated learning method based on data enhancement and L2 enhancement defense. Figure 1This is a first flow chart of the malicious security federated learning method based on data enhancement and L2 enhancement defense in this embodiment;

[0030] A malicious security federated learning method based on data enhancement and L2 enhancement defense proposed in this embodiment includes the following steps:

[0031] S1: The local clients each generate average sample data, and encrypt the average sample data and send it to the server;

[0032] S2: the server sends the encrypted average sample data of several local clients to each local client;

[0033] S3: Each local client trains its local model based on the received encrypted average sample data and its own local data to obtain a gradient vector of the trained local model, and sends the gradient vector to the server using encryption technology;

[0034] S4: the server checks the component lengths of the received gradient vectors, retains the gradient vectors whose component lengths do not exceed a preset component length threshold, and calculates the L2 norm corresponding to the gradient value of the retained gradient vector to obtain the gradient L2 norm;

[0035] S5: Using the server to regard the gradient value whose gradient L2 norm exceeds a preset upper limit as a malicious gradient, filtering the malicious gradient to obtain the final retained gradient value, and using the final retained gradient value to update the global model.

[0036] In the specific implementation process, each local client of the present application trains its local model in combination with the received encrypted average sample data and its own local data, which can effectively avoid the problem of large local differences in local models corresponding to different local clients and effectively improve the performance of the global model; and use encryption technology to send the gradient vector to the server to reduce the risk of gradient information leakage; in addition, by retaining only the gradient vectors whose component length does not exceed the preset component length threshold, and further filtering the gradient value whose gradient L2 norm exceeds the preset upper limit, the risk of malicious clients using gradient increase to destroy the global model is effectively reduced.

[0037] In this embodiment, data enhancement technology applied to federated learning is used (encrypted average sample data and local data are used as training data) to solve the local model deviation problem caused by different data properties and further improve the performance of the global model of federated learning. That is, each client can use data enhancement technology to solve the problem of different sample data by using the average sample data provided by other clients, and train a model with smaller local model differences to improve the global performance. When the degree of heterogeneity of the data owned by the client is reduced, the performance of the model can be greatly improved.

[0038] In an optional embodiment, the server includes server S0 and server S1;

[0039] When the local clients generate average sample data respectively and encrypt the average sample data and send them to the server, the local clients segment the average sample data into two parts to obtain the encrypted average sample data in two parts, one part is sent to the server S0, and the other part is sent to the server S1;

[0040] Among them, let the data set owned by the i-th local client be D i , dataset D i The amount of data is d i , and the average number of sample data required to be generated by the i-th local client is s i , then the data set D i The average score is s i group, let the data set corresponding to the i-th group of the i-th local client be Sd i , and the data set Sd i The amount of data is J i , then the expressions for the generated average sample data include:

[0041]

[0042] In the formula, Represents the average sample data The average characteristics of express The corresponding average label, x represents the data set Sd i The features in , y represents the label corresponding to x, (x, y) represents the data set Sd i A piece of data in .

[0043] In an optional embodiment, the local client divides the average sample data into two parts to obtain the average sample data encrypted in two parts. The average sample data is encrypted in two parts, one part is The other part is And i∈s i , r i1 and r i2 Respectively represent the preset first shard encryption value and the second shard encryption value.

[0044] In an optional embodiment, each local client trains its local model by combining the received encrypted average sample data and its own local data, and the step of obtaining the gradient vector of the trained local model includes:

[0045] The local client decrypts the received encrypted average sample data to obtain the original average sample data; wherein the average sample data includes a plurality of average sample data sets, all average sample data of a local client constitute an average sample data set, and different local clients correspond to different average sample data sets;

[0046] The local client randomly extracts a batch of data from its own local data to form a data set and selecting an average sample data set from the average sample data as an input of its local model to train the local model;

[0047] During the training process, the loss function is iteratively solved based on the input of the local model When the number of iterations reaches the preset value or the loss function When the function value reaches the minimum, the iteration is stopped and the gradient vector of the trained local model is obtained.

[0048] In an optional embodiment, the loss function The expressions include:

[0049]

[0050]

[0051]

[0052] In the formula, (x s ,y s ) represents the local data of the local client itself, x s It means (x s ,y s ), y s Represents x s The corresponding label; Represents the characteristics of the average sample data in the selected average sample data set, express The corresponding label; l represents the original loss function of the local model; α represents the weight.

[0053] In an optional embodiment, when the gradient vector is sent to the server using encryption technology, the encryption technology used includes Boolean secret sharing technology; wherein the Boolean secret sharing technology converts the gradient vector into a Boolean gradient share;

[0054] Before the server checks the component length of the received gradient vector, it is necessary to preset a component length threshold, and the expression of the component length threshold includes: w -1, where w represents the preset bit number threshold;

[0055] When the server checks the component length of the received gradient vector, if the length of any component value in the Boolean gradient share corresponding to the gradient vector of the local client is greater than w, the component length of the gradient vector is considered to exceed a preset component length threshold, and the local client sending the gradient vector is considered to be a malicious client, and the gradient vector corresponding to the malicious client is filtered;

[0056] Otherwise, the gradient vector is retained if the component length of the gradient vector does not exceed a preset component length threshold.

[0057] In an optional embodiment, the server includes server S0 and server S1;

[0058] Before calculating the L2 norm corresponding to the gradient value of the retained gradient vector and obtaining the gradient L2 norm, the server S0 and the server S1 interact using oblivious transmission, and during the interaction, the Boolean gradient share is converted into an arithmetic gradient share using secure bit multiplication;

[0059] When calculating the L2 norm corresponding to the gradient value of the retained gradient vector, the server uses the Beaver multiplication triplet to calculate the L2 norm of the gradient vector corresponding to the arithmetic gradient share to obtain the gradient L2 norm; and the server uses a secure adder to compare the gradient L2 norm with a preset upper limit, and regards the gradient value whose gradient L2 norm exceeds the preset upper limit as a malicious gradient, filters the malicious gradient, and obtains the final retained gradient value.

[0060] In this optional embodiment, by filtering the component length of the gradient vector exceeding the preset component length threshold, and the gradient value whose gradient L2 norm exceeds the preset upper limit, it is possible to effectively resist the L2 enhanced defense of gradient increase to resist complex poisoning attacks, improve the robustness and robustness of federated learning, so as to accelerate the convergence speed of the entire federated learning training process and improve the accuracy of the global model.

[0061] In an optional embodiment, before using the final retained gradient value to update the global model, the server adds the gradient components corresponding to all the final retained gradient values, and detects whether the amount of gradient information in the added result exceeds a preset threshold. If so, the global model is updated using the final retained gradient value, otherwise, the global model is not updated.

[0062] This embodiment proposes an efficient and maliciously secure federated learning method based on data enhancement and L2 enhancement defense in a non-IID data scenario. The method allows multiple federated learning clients to obtain the average data of other participants through data enhancement, thereby generating data sets related to other clients without leaking their original data. Each client performs training locally using its own data set and the data set generated by data enhancement, and sends the updated gradient to the server after the training is completed. In the present invention, the aggregation stage of federated learning involves the collaboration of two servers S0 and S1. Specifically, after the training is completed, the local client generates two gradient bit components by bit decomposition of the locally updated gradient, and sends them to the two servers respectively. The server performs L2 decomposition on the received bit components. Π Check and filter out malicious gradient updates, then restore the arithmetic component through the secure bit multiplication technology, and then calculate the corresponding L2 norm to eliminate malicious updates that do not meet the conditions. The method of the present invention effectively solves the problems of slow updates and unsatisfactory training effects under the condition of non-independent and identically distributed data. Through the enhanced L2 defense mechanism, malicious gradient updates can be effectively eliminated, malicious and secure federated learning aggregation is achieved, and the accuracy of federated learning training results is significantly improved.

[0063] Example 2

[0064] This embodiment is based on the malicious security federated learning method based on data enhancement and L2 enhancement defense proposed in Embodiment 1. Figure 2 This is a second flow chart of the malicious security federated learning method proposed in this embodiment. Figure 2 1 means grouping the data of each client. Figure 2 The 2 means generating an average sample data set for the grouped data. Figure 2 3 means sending the generated average sample data set to the server. Figure 2 4 means that the server sends the generated sample data to each client. Figure 2 5 means using the local dataset and the generated sample dataset to train the local model. Figure 2 6 means sending the updated gradient share to the server. Figure 2 The 7 indicates that the server checks whether the bit length of each component of the gradient vector is within the agreed range. Figure 2 The 8 indicates that the server interacts to convert the Boolean shares into arithmetic shares. Figure 2 The 9 represents the L2 norm of the gradient value calculated by the server interaction. Figure 2 The value 10 indicates that the client uses an adder to check whether the L2 norm of the gradient value exceeds the threshold.

[0065] like Figure 2 As shown, this embodiment proposes the following specific implementation examples:

[0066] S1. Determine the number of groups of samples generated by each local client participating in federated learning;

[0067] S2, each client of the federated learning generates average sample data, and sends the generated average sample data shards to server S0 and server S1 respectively;

[0068] S3, server S0, and server S1 send the average sample data slices to each client;

[0069] S4, the local client uses the average sample data of other clients distributed by the server and local data to perform model training;

[0070] S5, the local client uses Boolean secret sharing technology to send the gradient vector to server S0 and server S1 respectively;

[0071] S6. Each server checks the length of the gradient vector component sent by the client and only retains the share value within the limit;

[0072] S7, the server converts the Boolean secret share into the original arithmetic secret share using secure bit multiplication;

[0073] S8. Calculate the L2 norm of the gradient value of each client;

[0074] S9. Check whether the gradient value meets the L2 defense requirements and filter out the gradient values ​​that do not meet the requirements (large norm).

[0075] S10. The server updates the global model using the gradient value that meets the requirements.

[0076] In one possible implementation, each client of the federated learning generates average sample data to be used by the client to train the model in the data augmentation technique, which may be:

[0077] Assume that the average number of samples required to be generated by client i is s i , the data set owned by client i is D i , the amount of data is d i , the corresponding data set subscript is [0, d i -1], then the data set D i The average score is s iGroups are as follows: For the dataset of group 1, the subscripts are: It can be deduced from this that for the i-th group of data sets, its subscript is The last set of data sets, namely the sth i The subscripts of the groups are: The above process is to averagely group the data set of a certain client. Then the data set Sd of the i-th group is i , the amount of data is J i , the average data generated is:

[0078]

[0079] In a possible implementation, after each client of the federated learning generates average sample data, the generated sample data shards are sent to server S0 and server S1 respectively, specifically:

[0080] The average data set that each client will generate Send to server S0, (i∈s i ) is sent to server S1.

[0081] In a possible implementation, the local client uses the average sample data of other clients distributed by the server and the local data set to train the model, specifically:

[0082] First, the client collects the average data sets obtained from servers S0 and S1 and restores them to the original data sets. Then client i uses the collected data sets and its own data sets to train the model. Specifically, a batch of data sets are randomly extracted from the local data sets. Then select a data set from the collected data and calculate the loss function as follows, where l is the original corresponding loss function:

[0083]

[0084]

[0085]

[0086] In one possible implementation, the local client uses Boolean secret sharing technology to send the gradient vector to server S0 and server S1 respectively. Specifically, for each client, we assume that the gradient data to be transmitted is x, where x is a vector with m components and the length of each component is upper bounded by w. where i represents the i-th component of the gradient vector x, i∈{1, 2, 3, ..., m}, and j is the index of the bit, j∈[w], generating the shares x respectively. (0)and x (1) ,(x (0) and x (1) Integer ring ), and x (b) Send to server b, b∈{0,1}.

[0087] In one possible implementation, each server checks the length of the gradient vector component sent by the client and only retains the share value within the limit. Specifically, the server checks each component of the Boolean secret share sent by each client. When the length of the component value is greater than w, the client is identified as a malicious client. The client has sent a gradient component that does not meet the size requirement, and the current gradient is filtered out.

[0088] In a possible implementation, the server converts the Boolean secret share into the original arithmetic secret share, calculates the L2 norm of the gradient, and filters out the gradient values ​​that do not meet the requirements, specifically:

[0089] First, the two servers use oblivious transfer to implement secure bit multiplication to convert the Boolean secret shares into arithmetic secret shares. Specifically:

[0090] In oblivious transfer, the server as the sender has an input Δ∈{0,1} λ , Where Δ is the verification parameter of COT (a variant of oblivious transfer), λ is the length of the arithmetic share; the input of the server as the receiver is Assume H is a hash function under the random hash machine assumption, and The two servers interact as follows. Suppose server S0 is the sender of the oblivious transmission, server S1 is the receiver of the oblivious transmission, and the Boolean secret share held by server S0 is x (0) , the Boolean secret share held by server S1 is x (1) , for each component of the gradient vector, there is the following process:

[0091] Server S0 first calculates v0←H(c||q) and v1←H(c||q+Δ), and sets y (0) ←-v0(mod 2 j ), where c is a global counter. Server S0 will u←v0+v1+x (0) (mod 2 j ) is sent to server S1.

[0092] Server S1 calculates v←H(c||t), then the corresponding arithmetic secret share is:

[0093] The above process implements secure bit multiplication. Then the two servers interact and process each component of the gradient vector to convert the Boolean secret share into an arithmetic secret share. Specifically:

[0094] Server B is set up Then, each bit of each component of the gradient vector is processed, and the above-mentioned secure bit multiplication is called to obtain y (b) ,calculate (Here i corresponds to the i-th bit of each component). After traversing w bits, one component of the gradient vector can be restored. The same operation is performed on all components to restore the arithmetic secret share of the entire gradient vector.

[0095] After processing and calculation of the above interactive protocol, the server can obtain the corresponding arithmetic secret share for subsequent L2 norm calculation and checking.

[0096] In one possible implementation, the L2 norm of the gradient value is calculated as follows:

[0097] For all components in all local client gradient vectors, corresponding Beaver multiplication triplets are generated. The server opens the multiplication triplets and has e←z i -a i , server S0 calculation Server S1 calculation

[0098] After the above operations, each server can obtain the components of the L2 norm of the gradient vector, so as to facilitate the subsequent check whether the L2 norm exceeds its limit.

[0099] In one possible implementation, the server checks whether the gradient value meets the L2 defense requirements. Otherwise, the gradient value that does not meet the requirements is filtered out. Specifically,

[0100] For server S0, calculate z (0) ←z (0) -u 2 , where u 2 is the upper bound of the L2 norm. Server b inputs z into the secure adder (b) To get z (b) The sign bit of the two sign bits is taken and the secure bit multiplication is performed on the two sign bits. When the result of the sign bit multiplication is 0, it means that the L2 norm of the gradient value exceeds the upper bound. Therefore, the gradient value is regarded as a modified malicious gradient value and the gradient value of the client is rejected.

[0101] In one possible implementation, the server updates the global model using a gradient value that meets the requirements, specifically:

[0102] For the remaining updated gradient information that meets the enhanced L2 enhanced defense, server b adds all the gradient components together and records them as Where i∈[m]; when the amount of collected gradient information exceeds τ, the complete updated gradient value is interactively reconstructed and the model is globally updated; otherwise, this round of update is abandoned.

[0103] The method provided in this description enables the client to train a model with smaller local model differences in the case of non-independent and identically distributed data, so as to improve the global performance; at the same time, L2 enhanced defense is used to filter out maliciously increased gradient information to defend against poisoning attacks from the client, thereby improving the accuracy, robustness and stability of the global model of federated learning.

[0104] This embodiment also provides another specific implementation example, including the following steps:

[0105] A1. The client of federated learning groups the local data set according to the settings and generates an average data sample for each grouped data set. The client sends the generated data sample to the server, and the server sends the data sample to other clients.

[0106] A2. The local client uses the collected data samples and the local data set to train the model, and sends the updated gradient information of the trained model to the server;

[0107] A3. The server checks whether each component of the gradient information exceeds the limited bit to achieve L Π examine;

[0108] A4. Convert the gradient share from Boolean share to arithmetic share, calculate the L2 norm of the gradient vector, and check whether the L2 norm exceeds the upper bound;

[0109] A5. Use the remaining gradient vectors that meet the requirements to update the model.

[0110] In this embodiment, data enhancement technology is used to solve the problem of large local model deviation and poor global model performance due to different data. In the present invention, the average sample data generated by each client avoids information leakage caused by directly sending sample data. In the uploaded gradient information, secret sharing is used to protect the gradient information. In the Boolean share, the bit length of the Boolean share can be directly checked, and L ΠDefense, filter out gradient values ​​with abnormal component lengths. After converting Boolean shares into arithmetic shares, calculate the L2 norm of the gradient value to implement L2 defense, that is, the above is L2 enhanced defense, filter out maliciously increased gradient values, and enhance the stability and robustness of the global model.

[0111] Furthermore, the client of federated learning groups the local data set according to the settings and generates an average data sample for each grouped data set; the client sends the generated data sample to the server, and the server sends the data sample to other clients, specifically:

[0112] A11: The client of federated learning groups the local data set according to the settings and generates an average data sample for each grouped data set. The client sends the generated data sample to the server, and the server sends the data sample to other clients.

[0113] A12: Average data set generated by each client (i∈s i ) is sent to server S0, (i∈s i ) is sent to server S1.

[0114] Furthermore, the local client uses the collected data samples and the local data set to train the model, and sends the updated gradient information of the trained model to the server, specifically:

[0115] A21: First, the client collects the average data sets obtained from servers S0 and S1 and restores them to the original data sets. Then client i uses the collected data sets and its own data sets to train the model. Specifically, a batch of data sets are randomly extracted from the local data sets. Then select a data set from the collected data and calculate the loss function as follows, where l is the original corresponding loss function:

[0116]

[0117]

[0118]

[0119] Furthermore, the server checks whether each component of the gradient information exceeds the limited bit to achieve L Π Check; specifically:

[0120] A31: For each client, we assume that the gradient data to be transmitted is x, which is a vector with m components, and the length of each component is capped at w. where i represents the i-th component of the gradient vector x, i∈{1, 2, 3, ..., m}, and j is the index of the bit, j∈[w], generating the shares x respectively. (0) and x (1) ,(x (0) and x (1) Integer ring ), and x (b) Send to server b, b∈{0,1}.

[0121] Furthermore, the gradient shares are converted from Boolean shares to arithmetic shares, the L2 norm of the gradient vector is calculated, and it is checked whether the L2 norm exceeds the upper bound, specifically:

[0122] A41: In oblivious transfer, the server as the sender's input is Where Δ is the verification parameter of COT (a variant of oblivious transfer), λ is the length of the arithmetic share; the input of the server as the receiver is Assume H is a hash function under the random hash machine assumption, and The two servers interact as follows. Suppose server S0 is the sender of the oblivious transmission, server S1 is the receiver of the oblivious transmission, and the Boolean secret share held by server S0 is x (0) , the Boolean secret share held by server S1 is x (1) , for each component of the gradient vector, there is the following process:

[0123] Server S0 first calculates v0←H(c||q) and v1←H(c||q+Δ), and sets y (0) ←-v0(mod 2 j ), where c is a global counter. Server S0 will u←v0+v1+x (0) (mod 2 j ) is sent to server S1.

[0124] A42: Server S1 calculates v←H(c||t), then the corresponding arithmetic secret share is:

[0125] The above process implements secure bit multiplication. Then the two servers interact and process each component of the gradient vector to convert the Boolean secret share into an arithmetic secret share. Specifically:

[0126] A43: Server b is set up Then, each bit of each component of the gradient vector is processed, and the above-mentioned secure bit multiplication is called to obtain y (b) ,calculate (Here i corresponds to the i-th bit of each component). After traversing w bits, one component of the gradient vector can be restored. The same operation is performed on all components to restore the arithmetic secret share of the entire gradient vector.

[0127] After processing and calculation of the above interactive protocol, the server can obtain the corresponding arithmetic secret share for subsequent L2 norm calculation and checking.

[0128] Furthermore, the remaining gradient vectors that meet the requirements are used to update the model, specifically:

[0129] A51: For the remaining updated gradient information that meets the enhanced L2 enhanced defense, server b adds all the gradient components together, recorded as Where i∈[m]; when the amount of collected gradient information exceeds τ, the complete updated gradient value is interactively reconstructed and the model is globally updated; otherwise, this round of update is abandoned.

[0130] Example 3

[0131] This embodiment proposes a malicious security federated learning system based on data enhancement and L2 enhancement defense, which is used to implement a malicious security federated learning method based on data enhancement and L2 enhancement defense proposed in Example 1.

[0132] Figure 3 This is an overall framework diagram of the malicious security federated learning system based on data enhancement and L2 enhancement defense in this embodiment.

[0133] The malicious security federated learning system based on data enhancement and L2 enhancement defense includes:

[0134] An average sample data generating module, used for generating average sample data using the local clients respectively, and encrypting the average sample data and sending it to the server;

[0135] An encrypted information distribution module, used to send the encrypted average sample data of several local clients to each local client using the server;

[0136] A gradient vector generation module is used to train the local model of each local client by combining the received encrypted average sample data and its own local data, obtain the gradient vector of the trained local model, and send the gradient vector to the server by using encryption technology;

[0137] a gradient vector filtering module, configured to use the server to check the component lengths of the received gradient vectors, retain the gradient vectors whose component lengths do not exceed a preset component length threshold, and calculate the L2 norm corresponding to the gradient value of the retained gradient vectors to obtain the gradient L2 norm;

[0138] The global model updating module is used to use the server to regard the gradient value whose gradient L2 norm exceeds the preset upper limit as a malicious gradient, filter the malicious gradient, obtain the final retained gradient value, and use the final retained gradient value to update the global model.

[0139] This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the malicious security federated learning method based on data enhancement and L2 enhancement defense described in Example 1.

[0140] It can be understood that the malicious security federated learning system and computer device based on data enhancement and L2 enhancement defense in this embodiment improves the method of Example 1. The options in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0141] The same or similar reference numerals correspond to the same or similar components;

[0142] The terms used to describe the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0143] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A malicious security federated learning method based on data enhancement and L2 enhancement defense, the method is implemented based on several local clients and servers, characterized in that: The following steps are involved: The local clients each generate average sample data, and encrypt the average sample data and send it to the server; The server sends the encrypted average sample data of several local clients to each local client; Each local client trains its local model based on the received encrypted average sample data and its own local data to obtain a gradient vector of the trained local model, and sends the gradient vector to the server using encryption technology; The server checks the component lengths of the received gradient vectors, retains the gradient vectors whose component lengths do not exceed a preset component length threshold, and calculates the L2 norm corresponding to the gradient value of the retained gradient vector to obtain the gradient L2 norm; The server regards the gradient value whose gradient L2 norm exceeds a preset upper limit as a malicious gradient, filters the malicious gradient to obtain the final retained gradient value, and uses the final retained gradient value to update the global model.

2. The malicious security federated learning method based on data enhancement and L2 enhancement defense according to claim 1 is characterized in that: The servers include server S0 and server S1; When the local clients generate average sample data respectively and encrypt the average sample data and send them to the server, the local clients segment the average sample data into two parts to obtain the encrypted average sample data in two parts, one part is sent to the server S0, and the other part is sent to the server S1; Among them, let the data set owned by the i-th local client be D i , dataset D i The amount of data is d i , and the average number of sample data required to be generated by the i-th local client is s i , then the data set D i The average score is s i group, let the data set corresponding to the i-th group of the i-th local client be Sd i , and the data set Sd i The amount of data is J i , then the expressions for the generated average sample data include: In the formula, Represents the average sample data The average characteristics of express The corresponding average label, x represents the data set Sd i The features in , y represents the label corresponding to x, (x, y) represents the data set Sd i A piece of data in .

3. The malicious security federated learning method based on data enhancement and L2 enhancement defense according to claim 2 is characterized in that: The local client divides the average sample data into two parts to obtain the average sample data encrypted in two parts. The average sample data is encrypted in two parts, one part is The other part is And i∈s i , r i1 and r i2 Respectively represent the preset first shard encryption value and the second shard encryption value.

4. The malicious security federated learning method based on data enhancement and L2 enhancement defense according to claim 1 is characterized in that: Each local client trains its local model by combining the received encrypted average sample data and its own local data. The steps of obtaining the gradient vector of the trained local model include: The local client decrypts the received encrypted average sample data to obtain the original average sample data; wherein the average sample data includes a plurality of average sample data sets, all average sample data of a local client constitute an average sample data set, and different local clients correspond to different average sample data sets; The local client randomly extracts a batch of data from its own local data to form a data set and selecting an average sample data set from the average sample data as an input of its local model to train the local model; During the training process, the loss function is iteratively solved based on the input of the local model When the number of iterations reaches the preset value or the loss function When the function value reaches the minimum, the iteration is stopped and the gradient vector of the trained local model is obtained.

5. The malicious security federated learning method based on data enhancement and L2 enhancement defense according to claim 4 is characterized in that: The loss function The expressions include: In the formula, (x s ,y s ) represents the local data of the local client itself, x s It means (x s ,y s ), y s Represents x s The corresponding label; Represents the characteristics of the average sample data in the selected average sample data set, express The corresponding label; l represents the original loss function of the local model; α represents the weight.

6. The malicious security federated learning method based on data enhancement and L2 enhancement defense according to any one of claims 1 to 5, characterized in that: When the gradient vector is sent to the server using encryption technology, the encryption technology used includes Boolean secret sharing technology; wherein the Boolean secret sharing technology converts the gradient vector into a Boolean gradient share; Before the server checks the component length of the received gradient vector, it is necessary to preset a component length threshold, and the expression of the component length threshold includes: w -1, where w represents the preset bit number threshold; When the server checks the component length of the received gradient vector, if the length of any component value in the Boolean gradient share corresponding to the gradient vector of the local client is greater than w, the component length of the gradient vector is considered to exceed a preset component length threshold, and the local client sending the gradient vector is considered to be a malicious client, and the gradient vector corresponding to the malicious client is filtered; Otherwise, the gradient vector is retained if the component length of the gradient vector does not exceed a preset component length threshold.

7. The malicious security federated learning method based on data enhancement and L2 enhancement defense according to claim 6 is characterized in that: The servers include server S0 and server S1; Before calculating the L2 norm corresponding to the gradient value of the retained gradient vector and obtaining the gradient L2 norm, the server S0 and the server S1 interact using oblivious transmission, and during the interaction, the Boolean gradient share is converted into an arithmetic gradient share using secure bit multiplication; When calculating the L2 norm corresponding to the gradient value of the retained gradient vector, the server uses the Beaver multiplication triplet to calculate the L2 norm of the gradient vector corresponding to the arithmetic gradient share to obtain the gradient L2 norm; and the server uses a secure adder to compare the gradient L2 norm with a preset upper limit, and regards the gradient value whose gradient L2 norm exceeds the preset upper limit as a malicious gradient, filters the malicious gradient, and obtains the final retained gradient value.

8. The malicious security federated learning method based on data enhancement and L2 enhancement defense according to claim 1 is characterized in that: Before using the final retained gradient value to update the global model, the server adds up the gradient components corresponding to all the final retained gradient values, and detects whether the amount of gradient information in the added result exceeds a preset threshold. If so, the global model is updated using the final retained gradient value, otherwise, the global model is not updated.

9. A malicious security federated learning system based on data enhancement and L2 enhancement defense, used to implement the malicious security federated learning method based on data enhancement and L2 enhancement defense as described in any one of claims 1 to 8, characterized in that: include: An average sample data generating module, used for generating average sample data using the local clients respectively, and encrypting the average sample data and sending it to the server; An encrypted information distribution module, used to send the encrypted average sample data of several local clients to each local client using the server; A gradient vector generation module is used to train the local model of each local client by combining the received encrypted average sample data and its own local data, obtain the gradient vector of the trained local model, and send the gradient vector to the server by using encryption technology; a gradient vector filtering module, configured to use the server to check the component lengths of the received gradient vectors, retain the gradient vectors whose component lengths do not exceed a preset component length threshold, and calculate the L2 norm corresponding to the gradient value of the retained gradient vectors to obtain the gradient L2 norm; The global model updating module is used to use the server to regard the gradient value whose gradient L2 norm exceeds the preset upper limit as a malicious gradient, filter the malicious gradient, obtain the final retained gradient value, and use the final retained gradient value to update the global model.

10. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that: When the computer-readable instructions are executed by the processor, the processor executes the steps of the malicious security federated learning method based on data enhancement and L2 enhancement defense as described in any one of claims 1 to 8.

Images