Federal learning aggregation method based on locality sensitive hashing and related equipment

Abstract

The invention provides a federated learning aggregation method based on locality sensitive hashing and related equipment. The method comprises the following steps: receiving updating parameters of each model and averaging to obtain query point parameters; for each model update parameter, performing hash mapping by using the first function set and the second function set to obtain a first hash signature and a second hash signature, and constructing a first candidate parameter set based on each first hash signature and the query point parameter; constructing a second candidate parameter set based on each second hash signature and the query point parameter; and performing weighted average on the intersection of the first candidate set and the second candidate set to obtain global model updating parameters, and updating the global model. According to the scheme, through the local sensitive hashing method combining the Euclidean distance and the cosine similarity, subtle differences of model updating are detected more finely, precision and robustness are improved, through the Hash mapping and screening process, model updating is detected and aggregated under the condition that information is not leaked, and data privacy is effectively protected.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a federated learning aggregation method based on local sensitive hashing and related equipment. Background Art

[0002] Federated Learning, as a distributed machine learning framework that enables data to be used locally, provides a new solution to data silos and privacy protection issues, and has broad application prospects in the fields of medicine, finance, and the Internet of Things. In the process of federated learning, there is a possibility of poisoning attacks.

[0003] Poisoning attack is a security attack method against machine learning models. In this attack, malicious users will deliberately modify the training data to affect the training results or performance of the model. In order to effectively resist poisoning attacks, federated learning systems usually use a variety of security technologies to protect the security and integrity of model updates. A common method is malicious client detection based on a single feature (such as Euclidean distance or cosine similarity). This method determines whether there is an abnormal model update by calculating the feature similarity between the model update submitted by the participant and other model updates. Specifically, the Euclidean distance method evaluates the numerical difference between model updates, while the cosine similarity method evaluates the angular difference between them. If the similarity between the model update submitted by a participant and other model updates is lower than the set threshold, the parameter server will mark it as a malicious client and take corresponding measures, such as excluding its model update or performing further verification and processing.

[0004] The above methods have the following disadvantages: First, the accuracy is limited, especially in the face of complex poisoning attacks, it is often unable to capture hidden malicious model updates. Second, the robustness is poor and it is vulnerable to adversarial attacks. Attackers can bypass the detection mechanism by designing poisoning strategies. Third, the model parameters or calculation results of the participants may be involved in the model verification and anomaly detection process, which poses a risk of privacy information leakage. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a federated learning aggregation method and related equipment based on local sensitive hashing to achieve the purpose of solving the shortcomings of the prior art in terms of accuracy, robustness and privacy protection.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] A first aspect of an embodiment of the present invention discloses a federated learning aggregation method based on locality sensitive hashing, which is applied to a central server. The method includes:

[0008] Receive the model update parameters sent by each client, and calculate the mean of each model update parameter to obtain the query point parameter;

[0009] For each of the model update parameters, hash mapping is performed using a preset first function set and a preset second function set, respectively, to obtain a first hash signature and a second hash signature correspondingly; the first function set includes a plurality of Euclidean distance local sensitive hash functions; the second function set includes a plurality of cosine similarity local sensitive hash functions;

[0010] Based on each of the first hash signatures and the query point parameter, query a first candidate parameter from each of the model update parameters, and construct a first candidate parameter set consisting of each of the first candidate parameters;

[0011] Based on each of the second hash signatures and the query point parameter, query a second candidate parameter from each of the model update parameters, and construct a second candidate parameter set consisting of each of the second candidate parameters;

[0012] Performing weighted averaging on the intersection of the first candidate set and the second candidate set to obtain a global model update parameter;

[0013] The global model is updated based on the global model update parameters.

[0014] Preferably, each of the clients corresponds to a plurality of hash tables, each of the hash tables contains a plurality of hash buckets, and the hash tables are pre-constructed based on a plurality of Euclidean distance local sensitive hash functions and a plurality of cosine similarity local sensitive hash functions;

[0015] For each of the model update parameters, hash mapping is performed using a preset first function set and a preset second function set, respectively, to obtain a first hash signature and a second hash signature correspondingly, including:

[0016] For each of the model update parameters, based on a preset first function set and the model update parameter, determine a first hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameter, map the model update parameter to the first hash bucket, combine the indexes of each of the first hash buckets in the hash table, and obtain a first hash signature;

[0017] For each of the model update parameters, based on a preset second function set and the model update parameter, a second hash bucket is determined from the hash buckets contained in each of the hash tables corresponding to the model update parameter, the model update parameter is mapped to the second hash bucket, and the indexes of each of the second hash buckets in the hash table are combined to obtain a second hash signature.

[0018] Preferably, the querying of first candidate parameters from each of the model update parameters based on each of the first hash signatures and the query point parameters, and constructing a first candidate parameter set consisting of each of the first candidate parameters, includes:

[0019] For each of the first hash signatures, insert it into each of the first hash buckets of the corresponding model update parameter mapping;

[0020] For each of the hash buckets, calculating the density of the first hash signature according to the number of the first hash signatures in the hash bucket;

[0021] Based on the preset first function set and the query point parameter, determine a third hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameters, map the query point parameter to the third hash bucket, combine the indexes of each of the third hash buckets in the hash table, and obtain a third hash signature;

[0022] From each of the hash buckets, determine the hash bucket corresponding to the first hash signature consistent with the third hash signature as a target hash bucket, and if the density of the first hash signature in the target hash bucket is greater than a density threshold, and the target hash bucket is mapped with the corresponding model update parameter, determine the model update parameter corresponding to the target hash bucket as a first candidate parameter;

[0023] A first candidate set is constructed based on each of the first candidate parameters.

[0024] Preferably, the querying of second candidate parameters from each of the model update parameters based on each of the second hash signatures and the query point parameters, and constructing a second candidate parameter set consisting of each of the second candidate parameters, includes:

[0025] For each of the second hash signatures, insert it into each of the second hash buckets of the corresponding model update parameter mapping;

[0026] For each of the hash buckets, calculating the density of the second hash signatures according to the number of the first hash signatures in the hash bucket;

[0027] Based on the preset second function set and the query point parameter, determine a fourth hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameters, map the query point parameter to the fourth hash bucket, combine the indexes of each of the fourth hash buckets in the hash table, and obtain a fourth hash signature;

[0028] From each of the hash buckets, determine the hash bucket corresponding to the first hash signature consistent with the fourth hash signature as a target hash bucket, and if the density of the first hash signature in the target hash bucket is greater than a density threshold, and the target hash bucket is mapped with the corresponding model update parameter, determine the model update parameter corresponding to the target hash bucket as a second candidate parameter;

[0029] A second candidate set is constructed based on each of the second candidate parameters.

[0030] Preferably, the method further comprises:

[0031] Respectively updating the global model using the model update parameters in the first candidate set and the second candidate set, to obtain a first global model and a second global model respectively;

[0032] Based on preset weights, the first global model and the second global model are weightedly combined to obtain an updated global model.

[0033] A second aspect of an embodiment of the present invention discloses a federated learning aggregation device based on locality sensitive hashing, which is applied to a central server. The device includes:

[0034] A receiving unit, configured to receive the model update parameters sent by each client, and calculate the mean of each of the model update parameters to obtain the query point parameter;

[0035] A mapping unit, configured to perform hash mapping using a preset first function set and a preset second function set for each of the model update parameters, respectively, to obtain a first hash signature and a second hash signature correspondingly; the first function set includes a plurality of Euclidean distance local sensitive hash functions; the second function set includes a plurality of cosine similarity local sensitive hash functions;

[0036] a first query unit, configured to query first candidate parameters from each of the model update parameters based on each of the first hash signatures and the query point parameter, and construct a first candidate parameter set consisting of each of the first candidate parameters;

[0037] a second query unit, configured to query second candidate parameters from each of the model update parameters based on each of the second hash signatures and the query point parameter, and construct a second candidate parameter set consisting of each of the second candidate parameters;

[0038] A weighting unit, configured to perform weighted averaging on the intersection of the first candidate set and the second candidate set to obtain a global model update parameter;

[0039] An updating unit is used to update the global model based on the global model update parameters.

[0040] Preferably, each of the clients corresponds to a plurality of hash tables, each of the hash tables contains a plurality of hash buckets, and the hash tables are pre-constructed based on a plurality of Euclidean distance local sensitive hash functions and a plurality of cosine similarity local sensitive hash functions;

[0041] The mapping unit is specifically used for:

[0042] For each of the model update parameters, based on a preset first function set and the model update parameter, determine a first hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameter, map the model update parameter to the first hash bucket, combine the indexes of each of the first hash buckets in the hash table, and obtain a first hash signature;

[0043] For each of the model update parameters, based on a preset second function set and the model update parameter, a second hash bucket is determined from the hash buckets contained in each of the hash tables corresponding to the model update parameter, the model update parameter is mapped to the second hash bucket, and the indexes of each of the second hash buckets in the hash table are combined to obtain a second hash signature.

[0044] Preferably, the first query unit is specifically used to:

[0045] For each of the first hash signatures, insert it into each of the first hash buckets of the corresponding model update parameter mapping;

[0046] For each of the hash buckets, calculating the density of the first hash signature according to the number of the first hash signatures in the hash bucket;

[0047] Based on the preset first function set and the query point parameter, determine a third hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameters, map the query point parameter to the third hash bucket, combine the indexes of each of the third hash buckets in the hash table, and obtain a third hash signature;

[0048] From each of the hash buckets, determine the hash bucket corresponding to the first hash signature consistent with the third hash signature as a target hash bucket, and if the density of the first hash signature in the target hash bucket is greater than a density threshold, and the target hash bucket is mapped with the corresponding model update parameter, determine the model update parameter corresponding to the target hash bucket as a first candidate parameter;

[0049] A first candidate set is constructed based on each of the first candidate parameters.

[0050] A third aspect of an embodiment of the present invention discloses a storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the local sensitive hashing-based federated learning aggregation method described in any one of the first aspects of the embodiments of the present invention.

[0051] A fourth aspect of an embodiment of the present invention discloses an electronic device, including: a memory and a processor;

[0052] The memory is used to store computer programs;

[0053] The processor is used to execute the computer program, specifically to implement the local sensitive hashing-based federated learning aggregation method described in any one of the first aspects of the embodiments of the present invention.

[0054] Based on the above-mentioned embodiment of the present invention, a federated learning aggregation method based on local sensitive hashing and related devices are provided. Model update parameters sent by each client are received, and the mean of each model update parameter is calculated to obtain a query point parameter; for each model update parameter, a preset first function set and a preset second function set are used for hash mapping, and a first hash signature and a second hash signature are obtained accordingly; the first function set includes multiple Euclidean distance local sensitive hash functions; the second function set includes multiple cosine similarity local sensitive hash functions; based on each of the first hash signatures and the query point parameters, a first candidate parameter is queried from each of the model update parameters, and a first candidate parameter set composed of each of the first candidate parameters is constructed; based on each of the second hash signatures and the query point parameters, a second candidate parameter is queried from each of the model update parameters, and a second candidate parameter set composed of each of the second candidate parameters is constructed; a weighted average is performed on the intersection of the first candidate set and the second candidate set to obtain a global model update parameter; and the global model is updated based on the global model update parameter. In this scheme, by combining the local sensitive hashing method of Euclidean distance and cosine similarity, subtle differences in model updates are detected more finely, improving accuracy and robustness. Through hash mapping and filtering processes, model updates are detected and aggregated without leaking information, effectively protecting data privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0056] Figure 1A flowchart of a federated learning aggregation method based on locality sensitive hashing disclosed in an embodiment of the present invention;

[0057] Figure 2 A flowchart of a federated learning aggregation algorithm based on locality sensitive hashing disclosed in an embodiment of the present invention;

[0058] Figure 3 A structural diagram of a federated learning aggregation method based on locality sensitive hashing disclosed in an embodiment of the present invention;

[0059] Figure 4 The present invention is a structural diagram of an electronic device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0062] like Figure 1 , which is a flowchart of a federated learning aggregation method based on local sensitive hashing disclosed in an embodiment of the present invention, the method is applied to a central server and includes the following steps:

[0063] Step S101: receiving the model update parameters sent by each client, and calculating the mean of each model update parameter to obtain the query point parameter.

[0064] It should be noted that in each round of federated learning aggregation, each client downloads the current global model parameters w from the central server. i , each client is trained based on local data and current global model parameters to generate model update parameters Δw i .

[0065] It is understandable that some malicious clients will upload maliciously modified model update parameters, thereby poisoning the global model. Therefore, it is necessary to effectively filter the impact of typical poisoning attacks such as data poisoning and model poisoning from the model update parameters uploaded by each client, thereby ensuring the security and reliability of the federated learning system.

[0066] In step S101, in each round of aggregation, the central server calculates the mean Δw of all model update parameters avg As a query point parameter.

[0067] Step S102: For each model update parameter, hash mapping is performed using a preset first function set and a preset second function set, respectively, to obtain a first hash signature and a second hash signature accordingly.

[0068] The first function set includes a plurality of Euclidean distance local sensitive hash functions; and the second function set includes a plurality of cosine similarity local sensitive hash functions.

[0069] Specifically, the process of constructing the first function set and the second function set is as follows:

[0070] First, a suitable hash function family H is selected so that for any two similar data points p and q, the probability that they are mapped to the same hash bucket is high. On the contrary, for dissimilar data points, this probability is low.

[0071] H={h1,h2,…,h k}

[0072] The hash function family H includes hash functions for Euclidean distance and cosine similarity.

[0073] When designing the hash function family H, different methods can be selected to construct the hash function for different application scenarios or data types to achieve good performance and effect. The details are as follows:

[0074] 1. For Euclidean distance, use p-stable distribution: Euclidean distance is usually used to measure the distance between vectors. In the design of hash functions, if you want to maintain the consistency or approximation of Euclidean distance in hash space, you can use p-stable distribution to construct hash functions. p-stable distribution has stable properties and is suitable for processing spatial metrics such as Euclidean distance, especially in high-dimensional space.

[0075] 2. For cosine similarity, use the random hyperplane method: Cosine similarity is often used to measure the directional similarity between vectors, regardless of their size. In hash function design, if you need to maintain or approximate the cosine similarity of vectors in the hash space, you can use the random hyperplane method. The random hyperplane method refers to dividing the data space by randomly selecting a hyperplane and mapping data in different areas to different hash values, thereby maintaining the neighbor relationship of similar vectors in the hash space.

[0076] Then, design a set of hash functions:

[0077] 1. Euclidean distance hash function set, that is, the first function set:

[0078]

[0079] Among them, a i is a vector of random samples from a standard normal distribution, b i is a bucket vector uniformly distributed in [0, w) to avoid bucket solidification, w is the bucket width, Δw i It is the newly generated local model parameter of each federated learning client in each iteration.

[0080] 2. Cosine similarity hash function set, that is, the second function set:

[0081]

[0082] Among them, r j is a vector of random samples from a standard normal distribution.

[0083] Finally, L hash tables are generated for each client, each table contains k Euclidean distance hash functions in the first function set and k cosine similarity hash functions in the second function set, a total of 2k hash functions. The hash table set is represented as follows:

[0084] HashTable j ={h j1 ,h j2 ,…,h jk}for j=1,2,…,L

[0085] In one embodiment, the number of L and k is adjusted according to data distribution and query performance requirements. Increasing L can improve the recall rate, but will increase the amount of calculation; increasing k can improve the accuracy, but will increase the calculation time of a single query.

[0086] In the specific implementation process of step S102, each client corresponds to multiple hash tables, each hash table contains multiple hash buckets, and the hash table is pre-constructed based on multiple Euclidean distance local sensitive hash functions and multiple cosine similarity local sensitive hash functions.

[0087] For each model update parameter, based on a preset first function set and the model update parameter, a first hash bucket is determined from the hash buckets contained in each hash table corresponding to the model update parameter, the model update parameter is mapped to the first hash bucket, and the indexes of each first hash bucket in the hash table are combined to obtain a first hash signature.

[0088] More specifically, update the parameter Δw for each model i First, we use the Euclidean distance local sensitive hash function set (i.e., the first function set) for hash mapping and generate the same value as Δw from the standard normal distribution. i Multiple random projection vectors a of the same dimension i , and calculate Δw i Projected values ​​onto these random vectors.

[0089] To quantify the projection value, b is introduced i is a bucket vector uniformly distributed in [0, w) to avoid bucket solidification. The projection value is divided into multiple intervals according to the bucket width w, and each interval corresponds to a hash bucket. The model update parameter Δw i Mapped to the corresponding hash bucket, and finally the model update parameter Δw i The hash bucket indexes in the multiple hash tables are combined to form a first hash signature.

[0090] The first hash signature set is:

[0091]

[0092] For each model update parameter, based on a preset second function set and the model update parameter, a second hash bucket is determined from the hash buckets contained in each hash table corresponding to the model update parameter, the model update parameter is mapped to the second hash bucket, and the indexes of each second hash bucket in the hash table are combined to obtain a second hash signature.

[0093] More specifically, update the parameter Δw for each model i , use the cosine similarity local sensitive hash function set for hash mapping, and randomly generate a vector r randomly sampled from a standard normal distribution j , calculate r j ·Δw i The size of the corresponding hash bucket is determined by: j ·Δw i <0, then otherwise, Update the model parameters Δw i Mapped to the corresponding hash bucket, and finally the model update parameter Δw i The hash bucket indexes in the multiple hash tables are combined to form a second hash signature.

[0094] The second hash signature set is:

[0095]

[0096] It should be noted that when inserting or searching for a parameter in a hash table, the hash value of the parameter is first calculated through a hash function. The hash value determines where the parameter should be stored in the hash table, that is, determines the hash bucket to which the parameter belongs.

[0097] Step S103: Based on each first hash signature and the query point parameter, query the first candidate parameter from each model update parameter, and construct a first candidate parameter set consisting of each first candidate parameter.

[0098] In step S103, each first hash signature is inserted into the corresponding hash bucket, and for the query point parameter Δw avg , use the first function set to perform hash mapping to obtain a hash signature and find the model update parameter Δw in the bucket with the same first hash signature i If the density of the first hash signature is high and the model update parameter Δw is satisfied at the same time i and the query point parameter Δw avg In the same bucket, the corresponding client can be considered honest, and Δw i Reserved as the first candidate set C eud .

[0099] The specific implementation process of step S103 includes the following steps:

[0100] Step S31: insert each first hash signature into each first hash bucket of the corresponding model update parameter mapping.

[0101] In step S31, the first hash signature is composed of the index of each first hash bucket of the model update parameter mapping in the hash table, and the first hash signature is inserted into each corresponding first hash bucket.

[0102] Step S32: For each hash bucket, the density of the first hash signature is calculated according to the number of first hash signatures in the hash bucket.

[0103] In step S32, data points in low-density hash buckets with a density of the first hash signature in each hash bucket are marked as outliers. These data points have low similarity with other data points in the high-dimensional space.

[0104] It is understandable that the data points are the model update parameters of the hash bucket mapping.

[0105] Step S33: Based on the preset first function set and query point parameters, determine a third hash bucket from the hash buckets contained in each hash table corresponding to the model update parameter, map the query point parameter to the third hash bucket, combine the indexes of each third hash bucket in the hash table, and obtain a third hash signature.

[0106] In step S33, the query point parameters are used as model update parameters, and hash mapping is performed based on a preset first function set to obtain a corresponding hash signature. The specific implementation process is similar to step S102.

[0107] Step S34: From each hash bucket, determine the hash bucket corresponding to the first hash signature consistent with the third hash signature as the target hash bucket. If the density of the first hash signature in the target hash bucket is greater than the density threshold, and the target hash bucket is mapped with a corresponding model update parameter, then determine the model update parameter corresponding to the target hash bucket as the first candidate parameter.

[0108] Step S35: construct a first candidate set based on each first candidate parameter.

[0109] In one embodiment, the hash mapping and query processes may be parallelized to improve the efficiency and responsiveness of the system.

[0110] Step S104: Based on each second hash signature and the query point parameter, query the second candidate parameter from each model update parameter, and construct a second candidate parameter set consisting of each second candidate parameter.

[0111] In step S104, each second hash signature is inserted into the corresponding hash bucket, and for the query point parameter Δw avg , use the second function set to perform hash mapping to obtain a hash signature and find the model update parameter Δw in the bucket with the same second hash signature i If the density of the second hash signature is high and the model updates the parameter Δw i and the query point parameter Δw avg In the same bucket, the corresponding client can be considered honest, and Δw i Reserved as the second candidate set C cos .

[0112] The specific implementation process of step S104 includes the following steps:

[0113] Step S41: insert each second hash signature into each second hash bucket of the corresponding model update parameter mapping.

[0114] In step S41, the second hash signature is composed of the index of each second hash bucket of the model update parameter mapping in the hash table, and the second hash signature is inserted into each corresponding second hash bucket.

[0115] Step S42: For each hash bucket, the density of the second hash signature is calculated according to the number of the first hash signatures in the hash bucket.

[0116] In step S42, data points in low-density hash buckets with a density of the second hash signature in each hash bucket are marked as outliers. These data points have low similarity with other data points in the high-dimensional space.

[0117] Step S43: Based on the preset second function set and query point parameters, determine the fourth hash bucket from the hash buckets contained in each hash table corresponding to the model update parameter, map the query point parameter to the fourth hash bucket, combine the indexes of each fourth hash bucket in the hash table, and obtain a fourth hash signature.

[0118] In step S43, the query point parameters are used as model update parameters, and hash mapping is performed based on a preset second function set to obtain a corresponding hash signature. The specific implementation process is similar to step S102.

[0119] Step S44: From each hash bucket, determine the hash bucket corresponding to the first hash signature consistent with the fourth hash signature as the target hash bucket. If the density of the first hash signature in the target hash bucket is greater than the density threshold, and the target hash bucket is mapped with a corresponding model update parameter, then determine the model update parameter corresponding to the target hash bucket as the second candidate parameter.

[0120] Step S45: construct a second candidate set based on each second candidate parameter.

[0121] Step S105: performing weighted averaging on the intersection of the first candidate set and the second candidate set to obtain global model update parameters.

[0122] In step S105, a comprehensive candidate set C containing various global model update parameters is generated. Candidates as follows:

[0123] C Candidates =C eud ∩C cos

[0124] Take a weighted average of the intersection of the first candidate set and the second candidate set, and update the global model by Δw agg :

[0125]

[0126] In one embodiment, for each client, if the model update parameters sent by the client are not in the comprehensive candidate set, the client is marked as a malicious client, and in the next round of training, the model update parameters sent by the malicious client are not received.

[0127] Step S106: updating the global model based on the global model update parameters.

[0128] In step S106, the central server updates the global model w after updating the parameters of the aggregated global models:

[0129] w←w+Δw agg

[0130] It can be understood that after updating the global model, the client downloads the parameters of the updated global model w and uses it to update the local model, thereby entering the next round of federated learning.

[0131] In one embodiment, the global model is updated using the model update parameters in the first candidate set and the second candidate set, respectively, to obtain the first global model and the second global model correspondingly; based on preset weights, the first global model and the second global model are weighted combined to obtain an updated global model.

[0132] Exemplarily, the global model is updated using the model update parameters in the first candidate set and the second candidate set respectively, to obtain different global model update results w e and w c , for w e and w c Perform a weighted combination to update the global model:

[0133] w i =mw e +(1-m)w c ,m∈[0,1]

[0134] Different weight combinations are assigned according to the training results of each round to comprehensively measure similarity and adapt to different similarity measurement requirements.

[0135] like Figure 2 As shown, it is a flowchart of a federated learning aggregation algorithm based on local sensitive hashing disclosed in an embodiment of the present invention, which includes the following steps:

[0136] Step S201: The central server designs a hash function and constructs a hash table.

[0137] Step S202: Each client downloads the global model training local model parameters from the central server to obtain model update parameters.

[0138] Step S203: Calculate the query point parameters based on the update parameters of each model.

[0139] Step S204: Use the Euclidean distance locality sensitive hash function to query and obtain the first candidate set.

[0140] Step S205: Use the cosine similarity local sensitive hash function to query and obtain a second candidate set.

[0141] Step S206: Integrate the first candidate set and the second candidate set to obtain a comprehensive candidate set, and perform a global model update.

[0142] Step S207: Determine whether the model update parameters of each client are in the comprehensive candidate set; if so, execute step S209; if not, execute step S208.

[0143] Step S208: Determine that the client is a malicious client.

[0144] Step S209: Each non-malicious client performs the next round of local model parameter training based on the global model updated by the central server.

[0145] Step S210: Optimize parameters and expand strategies based on the aggregation results.

[0146] Parameter optimization includes adjusting the number of L and k according to the data distribution of the aggregation results and the query performance requirements. Increasing L can improve the recall rate, but it will increase the amount of calculation; increasing k can improve the accuracy, but it will increase the calculation time of a single query.

[0147] The strategy expansion includes parallelizing the hash mapping and query process to improve the efficiency and response speed of the system, and updating the global model using the model update parameters in the first candidate set and the second candidate set respectively, and obtaining the first global model and the second global model accordingly; based on the preset weights, the first global model and the second global model are weightedly combined to obtain the updated global model.

[0148] It should be noted that the specific implementation process of each step of the above algorithm has been described in detail in the above embodiments of the present invention, and reference can be made to each other and will not be repeated here.

[0149] Based on the above-mentioned embodiment of the present invention, a federated learning aggregation method based on local sensitive hashing is disclosed. In this scheme, local sensitive hashing technology is introduced as a key method to resist poisoning attacks in federated learning. Local sensitive hashing can map model updates to buckets in a high-dimensional space, and compare the similarities between these buckets through hash functions, so as to effectively detect and eliminate abnormal model updates and improve the accuracy and efficiency of malicious client detection. Secondly, cosine similarity and Euclidean distance are combined as the basic metrics of hash functions to avoid the limitations of a single indicator. This combination can more comprehensively consider the similarities and differences of model updates, and improve the system's recognition ability and robustness to malicious model updates.

[0150] Corresponding to the federated learning aggregation method based on local sensitive hashing disclosed in the above embodiment of the present invention, as Figure 3 As shown, it is a structural diagram of a federated learning aggregation device based on local sensitive hashing disclosed in an embodiment of the present invention. The device is applied to a central server and includes: a receiving unit 301, a mapping unit 302, a first query unit 303, a second query unit 304, a weighting unit 305 and an updating unit 306.

[0151] The receiving unit 301 is used to receive the model update parameters sent by each client, and calculate the mean of each model update parameter to obtain the query point parameter.

[0152] A mapping unit 302 is used to update parameters of each model, and perform hash mapping using a preset first function set and a preset second function set, respectively, to obtain a first hash signature and a second hash signature correspondingly; the first function set includes multiple Euclidean distance local sensitive hash functions; the second function set includes multiple cosine similarity local sensitive hash functions.

[0153] In one embodiment, each client corresponds to multiple hash tables, each hash table includes multiple hash buckets, and the hash table is pre-constructed based on multiple Euclidean distance local sensitive hash functions and multiple cosine similarity local sensitive hash functions. Accordingly, the mapping unit 302 is specifically used to:

[0154] For each model update parameter, based on a preset first function set and the model update parameter, a first hash bucket is determined from hash buckets included in each hash table corresponding to the model update parameter, the model update parameter is mapped to the first hash bucket, and the indexes of each first hash bucket in the hash table are combined to obtain a first hash signature;

[0155] For each model update parameter, based on a preset second function set and the model update parameter, a second hash bucket is determined from the hash buckets contained in each hash table corresponding to the model update parameter, the model update parameter is mapped to the second hash bucket, and the indexes of each second hash bucket in the hash table are combined to obtain a second hash signature.

[0156] The first query unit 303 is used to query the first candidate parameters from each model update parameter based on each first hash signature and the query point parameter, and construct a first candidate parameter set consisting of each first candidate parameter.

[0157] In one embodiment, the first query unit is specifically used for:

[0158] For each first hash signature, insert it into each first hash bucket of the corresponding model update parameter mapping;

[0159] For each hash bucket, calculate the density of the first hash signature according to the number of first hash signatures in the hash bucket;

[0160] Based on the preset first function set and the query point parameter, determine a third hash bucket from the hash buckets contained in each hash table corresponding to the model update parameter, map the query point parameter to the third hash bucket, combine the indexes of each third hash bucket in the hash table, and obtain a third hash signature;

[0161] From each hash bucket, determine the hash bucket corresponding to the first hash signature consistent with the third hash signature as the target hash bucket, if the density of the first hash signature in the target hash bucket is greater than the density threshold, and the target hash bucket is mapped with a corresponding model update parameter, then determine the model update parameter corresponding to the target hash bucket as the first candidate parameter;

[0162] A first candidate set is constructed based on each first candidate parameter.

[0163] The second query unit 304 is used to query second candidate parameters from each model update parameter based on each second hash signature and the query point parameter, and construct a second candidate parameter set consisting of each second candidate parameter.

[0164] In one embodiment, the second query unit 304 is specifically configured to:

[0165] For each second hash signature, insert it into each second hash bucket of the corresponding model update parameter mapping;

[0166] For each hash bucket, calculate the density of the second hash signature according to the number of the first hash signatures in the hash bucket;

[0167] Based on the preset second function set and the query point parameter, determine a fourth hash bucket from the hash buckets contained in each hash table corresponding to the model update parameter, map the query point parameter to the fourth hash bucket, combine the indexes of each fourth hash bucket in the hash table, and obtain a fourth hash signature;

[0168] From each hash bucket, determine the hash bucket corresponding to the first hash signature consistent with the fourth hash signature as the target hash bucket, if the density of the first hash signature in the target hash bucket is greater than the density threshold, and the target hash bucket is mapped with a corresponding model update parameter, then determine the model update parameter corresponding to the target hash bucket as the second candidate parameter;

[0169] A second candidate set is constructed based on each second candidate parameter.

[0170] The weighting unit 305 is used to perform weighted averaging on the intersection of the first candidate set and the second candidate set to obtain a global model update parameter.

[0171] The updating unit 306 is used to update the global model based on the global model update parameters.

[0172] In one embodiment, the federated learning aggregation device based on locality sensitive hashing further includes:

[0173] The optimization unit is used to update the global model using the model update parameters in the first candidate set and the second candidate set respectively, and obtain the first global model and the second global model accordingly; based on the preset weights, the first global model and the second global model are weighted combined to obtain the updated global model.

[0174] Based on the above-mentioned embodiment of the present invention, a federated learning aggregation device based on local sensitive hashing is disclosed. In this scheme, local sensitive hashing technology is introduced as a key method to resist poisoning attacks in federated learning. Local sensitive hashing can map model updates to buckets in a high-dimensional space, and compare the similarities between these buckets through hash functions, so as to effectively detect and eliminate abnormal model updates and improve the accuracy and efficiency of malicious client detection. Secondly, cosine similarity and Euclidean distance are combined as the basic metrics of hash functions to avoid the limitations of a single indicator. This combination can more comprehensively consider the similarities and differences of model updates, and improve the system's recognition ability and robustness to malicious model updates.

[0175] The present invention also provides an electronic device, please refer to Figure 4 , the electronic device includes a memory 401 and a processor 402.

[0176] Wherein, the memory 401 is used to store computer programs;

[0177] The processor 402 is used to execute a computer program, specifically to implement the locality-sensitive hashing-based federated learning aggregation method provided in any embodiment of the present application.

[0178] The present application also provides a computer storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the federated learning aggregation method based on local sensitive hashing provided in any embodiment of the present application.

[0179] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0180] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0181] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A federated learning aggregation method based on locality sensitive hashing, characterized in that: Applied to a central server, the method comprises: Receive the model update parameters sent by each client, and calculate the mean of each model update parameter to obtain the query point parameter; For each of the model update parameters, hash mapping is performed using a preset first function set and a preset second function set, respectively, to obtain a first hash signature and a second hash signature correspondingly; the first function set includes a plurality of Euclidean distance local sensitive hash functions; the second function set includes a plurality of cosine similarity local sensitive hash functions; Based on each of the first hash signatures and the query point parameter, query a first candidate parameter from each of the model update parameters, and construct a first candidate parameter set consisting of each of the first candidate parameters; Based on each of the second hash signatures and the query point parameter, query a second candidate parameter from each of the model update parameters, and construct a second candidate parameter set consisting of each of the second candidate parameters; Performing weighted averaging on the intersection of the first candidate set and the second candidate set to obtain a global model update parameter; The global model is updated based on the global model update parameters.

2. The method according to claim 1, characterized in that Each of the clients corresponds to a plurality of hash tables, each of the hash tables includes a plurality of hash buckets, and the hash tables are pre-constructed based on a plurality of Euclidean distance local sensitive hash functions and a plurality of cosine similarity local sensitive hash functions; For each of the model update parameters, hash mapping is performed using a preset first function set and a preset second function set, respectively, to obtain a first hash signature and a second hash signature correspondingly, including: For each of the model update parameters, based on a preset first function set and the model update parameter, determine a first hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameter, map the model update parameter to the first hash bucket, combine the indexes of each of the first hash buckets in the hash table, and obtain a first hash signature; For each of the model update parameters, based on a preset second function set and the model update parameter, a second hash bucket is determined from the hash buckets contained in each of the hash tables corresponding to the model update parameter, the model update parameter is mapped to the second hash bucket, and the indexes of each of the second hash buckets in the hash table are combined to obtain a second hash signature.

3. The method according to claim 2, characterized in that The querying of first candidate parameters from each of the model update parameters based on each of the first hash signatures and the query point parameters, and constructing a first candidate parameter set consisting of each of the first candidate parameters, includes: For each of the first hash signatures, insert it into each of the first hash buckets of the corresponding model update parameter mapping; For each of the hash buckets, calculating the density of the first hash signature according to the number of the first hash signatures in the hash bucket; Based on the preset first function set and the query point parameter, determine a third hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameters, map the query point parameter to the third hash bucket, combine the indexes of each of the third hash buckets in the hash table, and obtain a third hash signature; From each of the hash buckets, determine the hash bucket corresponding to the first hash signature consistent with the third hash signature as a target hash bucket, and if the density of the first hash signature in the target hash bucket is greater than a density threshold, and the target hash bucket is mapped with the corresponding model update parameter, determine the model update parameter corresponding to the target hash bucket as a first candidate parameter; A first candidate set is constructed based on each of the first candidate parameters.

4. The method according to claim 2, characterized in that: The querying of second candidate parameters from each of the model update parameters based on each of the second hash signatures and the query point parameters, and constructing a second candidate parameter set consisting of each of the second candidate parameters, includes: For each of the second hash signatures, insert it into each of the second hash buckets of the corresponding model update parameter mapping; For each of the hash buckets, calculating the density of the second hash signatures according to the number of the first hash signatures in the hash bucket; Based on the preset second function set and the query point parameter, determine a fourth hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameters, map the query point parameter to the fourth hash bucket, combine the indexes of each of the fourth hash buckets in the hash table, and obtain a fourth hash signature; From each of the hash buckets, determine the hash bucket corresponding to the first hash signature consistent with the fourth hash signature as a target hash bucket, and if the density of the first hash signature in the target hash bucket is greater than a density threshold, and the target hash bucket is mapped with the corresponding model update parameter, determine the model update parameter corresponding to the target hash bucket as a second candidate parameter; A second candidate set is constructed based on each of the second candidate parameters.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Respectively updating the global model using the model update parameters in the first candidate set and the second candidate set, to obtain a first global model and a second global model respectively; Based on preset weights, the first global model and the second global model are weightedly combined to obtain an updated global model.

6. A federated learning aggregation device based on locality sensitive hashing, characterized in that: Applied to a central server, the device comprises: A receiving unit, configured to receive the model update parameters sent by each client, and calculate the mean of each of the model update parameters to obtain the query point parameter; A mapping unit, configured to perform hash mapping using a preset first function set and a preset second function set for each of the model update parameters, respectively, to obtain a first hash signature and a second hash signature correspondingly; the first function set includes a plurality of Euclidean distance local sensitive hash functions; the second function set includes a plurality of cosine similarity local sensitive hash functions; a first query unit, configured to query first candidate parameters from each of the model update parameters based on each of the first hash signatures and the query point parameter, and construct a first candidate parameter set consisting of each of the first candidate parameters; a second query unit, configured to query second candidate parameters from each of the model update parameters based on each of the second hash signatures and the query point parameter, and construct a second candidate parameter set consisting of each of the second candidate parameters; A weighting unit, configured to perform weighted averaging on the intersection of the first candidate set and the second candidate set to obtain a global model update parameter; An updating unit is used to update the global model based on the global model update parameters.

7. The device according to claim 6, characterized in that Each of the clients corresponds to a plurality of hash tables, each of the hash tables includes a plurality of hash buckets, and the hash tables are pre-constructed based on a plurality of Euclidean distance local sensitive hash functions and a plurality of cosine similarity local sensitive hash functions; The mapping unit is specifically used for: For each of the model update parameters, based on a preset first function set and the model update parameter, determine a first hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameter, map the model update parameter to the first hash bucket, combine the indexes of each of the first hash buckets in the hash table, and obtain a first hash signature; For each of the model update parameters, based on a preset second function set and the model update parameter, a second hash bucket is determined from the hash buckets contained in each of the hash tables corresponding to the model update parameter, the model update parameter is mapped to the second hash bucket, and the indexes of each of the second hash buckets in the hash table are combined to obtain a second hash signature.

8. The device according to claim 7, characterized in that The first query unit is specifically configured to: For each of the first hash signatures, insert it into each of the first hash buckets of the corresponding model update parameter mapping; For each of the hash buckets, calculating the density of the first hash signature according to the number of the first hash signatures in the hash bucket; Based on the preset first function set and the query point parameter, determine a third hash bucket from the hash buckets included in each of the hash tables corresponding to the model update parameters, map the query point parameter to the third hash bucket, combine the indexes of each of the third hash buckets in the hash table, and obtain a third hash signature; From each of the hash buckets, determine the hash bucket corresponding to the first hash signature consistent with the third hash signature as a target hash bucket, and if the density of the first hash signature in the target hash bucket is greater than a density threshold, and the target hash bucket is mapped with the corresponding model update parameter, determine the model update parameter corresponding to the target hash bucket as a first candidate parameter; A first candidate set is constructed based on each of the first candidate parameters.

9. A storage medium, characterized in that: Used to store a computer program, which, when executed, is specifically used to implement the federated learning aggregation method based on local sensitive hashing as described in any one of claims 1 to 5.

10. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program, specifically to implement the federated learning aggregation method based on local sensitive hashing as described in any one of claims 1 to 5.

Images