Federated learning aggregation verification method and system

Abstract

Embodiments of the present application disclose a federated learning aggregation verification method and system, wherein the method comprises: each training party negotiates to obtain an initialized sub-model parameter; each training party performs local training according to local training data and the initialized sub-model parameter to obtain a local model update value; each training party calculates a local model update commitment value according to the local model update value and a randomly generated local commitment factor; each two training parties negotiate to generate a mask value, and each training party respectively disturbs the local model update value and the local commitment factor according to the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value; a first verification value is calculated according to all the local model update commitment values; a second verification value is calculated according to all the local model update disturbance values and the local commitment factor disturbance values, and whether the first verification value and the second verification value are consistent is compared; if consistent, the training is continued, otherwise the training is stopped. The data security in federated learning is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of privacy computing, in particular to a federated learning aggregation verification method and system. BACKGROUND

[0002] Federated learning is a technology for realizing collaborative training of a model while protecting the privacy of local data. The basic idea is that each participant holds original data locally and does not directly upload the data, but generates a model update value through local training, and then uploads the update value to an aggregation node for global model updating. This approach can avoid centralized storage and transmission of original data, thereby reducing the risk of privacy leakage.

[0003] However, for a classic and conventional horizontal federated learning scenario, an aggregation computing service provider is generally required to assist each data provider to complete the model aggregation and updating process. Obviously, this has some security risks: (1) the aggregation computing service provider can obtain the model update information of each data provider, which not only leaks the model data but also threatens the privacy security of the data; (2) the aggregation computing service provider may maliciously modify the model update information submitted by each data provider, giving incorrect model aggregation calculation results.

[0004] There is currently no effective solution to the above problems in the prior art. SUMMARY

[0005] To solve the above problems, the present application provides a federated learning aggregation verification method and system, which generates model update values locally by each training party and combines commitment algorithms and mask disturbance mechanisms for decentralized aggregation, and then verifies the consistency of the first verification value and the second verification value, thereby ensuring the correctness of the aggregation result while protecting data privacy, to overcome the privacy leakage and tampering risks of traditional centralized aggregation.

[0006] To achieve the above purpose, the present application provides a federated learning aggregation verification method, comprising: each training party obtains an initialized sub-model parameter according to a private data negotiation algorithm; each training party performs local training according to local training data and the initialized sub-model parameter to obtain a local model update value; each training party calculates a local model update commitment value using a commitment algorithm according to the local model update value and a locally randomly generated local commitment factor; each two training parties negotiate to generate a mask value, and each training party respectively disturbs the corresponding local model update value and the local commitment factor according to the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value; a first verification value is calculated according to the local model update commitment values of all training parties; a second verification value is calculated according to the local model update disturbance values and the local commitment factor disturbance values of all training parties, and the first verification value and the second verification value are compared for consistency, and if consistent, the training continues, otherwise the training stops.

[0007] Further optionally, the first training party, the second training party and the third training party are included.

[0008] The negotiation of the mask value by each two training parties includes: the first training party and the second training party generate a first mask value according to a private data negotiation algorithm; the second training party and the third training party generate a second mask value according to the private data negotiation algorithm; and the first training party and the third training party generate a third mask value according to the private data negotiation algorithm.

[0009] Further optionally, each training party respectively disturbs the corresponding local model update value and the local commitment factor according to the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value, including: the first training party calculates a first difference value of the first mask value and the third mask value, calculates a sum of the first model update value and the first difference value to obtain a first model update disturbance value; and calculates a sum of the first commitment factor and the first difference value to obtain a first commitment factor disturbance value; the second training party calculates a second difference value of the second mask value and the first mask value, and calculates a sum of the second model update value and the second difference value to obtain a second model update disturbance value; and calculates a sum of the second commitment factor and the second difference value to obtain a second commitment factor disturbance value; and the third training party calculates a third difference value of the third mask value and the second mask value, and calculates a sum of the third model update value and the third difference value to obtain a third model update disturbance value; and calculates a sum of the third commitment factor and the third difference value to obtain a third commitment factor disturbance value; wherein the first model update value, the second model update value and the third model update value are local model update values of the first training party, the second training party and the third training party respectively; and the first commitment factor, the second commitment factor and the third commitment factor are local commitment factors of the first training party, the second training party and the third training party respectively.

[0010] Further optionally, the calculation of the first check value according to the local model update commitment values of all training parties includes: obtaining a first model update commitment value of the first training party, a second model update commitment value of the second training party and a third model update commitment value of the third training party; and multiplying the first model update commitment value, the second model update commitment value and the third model update commitment value to obtain the first check value; wherein the local model update commitment value is obtained by multiplying an exponential operation result of a first preset generator and a local model update value, and an exponential operation result of a second preset generator and a local commitment factor.

[0011] Further optionally, the second check value is calculated according to the local model update disturbance value and the local commitment factor disturbance value of all training parties, including: calculating the sum of the first model update disturbance value, the second model update disturbance value and the third model update disturbance value to obtain a global model update value; calculating the sum of the first commitment factor disturbance value, the second commitment factor disturbance value and the third commitment factor disturbance value to obtain a global commitment factor value; performing an exponential operation on the global model update value according to the first preset generator to obtain a first intermediate result; performing an exponential operation on the global commitment factor value according to the second preset generator to obtain a second intermediate result; and multiplying the first intermediate result and the second intermediate result to obtain the second check value.

[0012] In another aspect, the present application also provides a federated learning aggregation check system, including: a parameter initialization module, configured to obtain an initialized sub-model parameter according to a private data negotiation algorithm by each training party; a training module, configured to perform local training according to local training data and the initialized sub-model parameter by each training party to obtain a local model update value; a commitment value calculation module, configured to calculate a local model update commitment value according to the local model update value and a locally randomly generated commitment factor by each training party using a commitment algorithm; a disturbance module, configured to generate a mask value by each two training parties, and each training party respectively disturbs the corresponding local model update value and the local commitment factor according to the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value; a check value calculation module, configured to calculate a first check value according to the local model update commitment value of all training parties; and a verification module, configured to calculate a second check value according to the local model update disturbance value and the local commitment factor disturbance value of all training parties, and compare whether the first check value and the second check value are consistent, if consistent, continue training, otherwise stop training.

[0013] Further optionally, the training parties include a first training party, a second training party and a third training party.

[0014] The disturbance module includes: a first mask calculation submodule, configured to generate a first mask value by the first training party and the second training party according to a private data negotiation algorithm; a second mask calculation submodule, configured to generate a second mask value by the second training party and the third training party according to a private data negotiation algorithm; and a third mask calculation submodule, configured to generate a third mask value by the first training party and the third training party according to a private data negotiation algorithm.

[0015] Further optionally, the disturbance module comprises: a first disturbance submodule, configured to calculate a first difference value between the first mask value and the third mask value, calculate a sum of the first model update value and the first difference value to obtain a first model update disturbance value, and calculate a sum of the first commitment factor and the first difference value to obtain a first commitment factor disturbance value; a second disturbance submodule, configured to calculate a second difference value between the second mask value and the first mask value, calculate a sum of the second model update value and the second difference value to obtain a second model update disturbance value, and calculate a sum of the second commitment factor and the second difference value to obtain a second commitment factor disturbance value; and a third disturbance submodule, configured to calculate a third difference value between the third mask value and the second mask value, calculate a sum of the third model update value and the third difference value to obtain a third model update disturbance value, and calculate a sum of the third commitment factor and the third difference value to obtain a third commitment factor disturbance value; wherein the first model update value, the second model update value and the third model update value are local model update values of the first training party, the second training party and the third training party respectively; and the first commitment factor, the second commitment factor and the third commitment factor are local commitment factors of the first training party, the second training party and the third training party respectively.

[0016] Further optionally, the verification value calculation module comprises: a commitment value acquisition submodule, configured to acquire a first model update commitment value of the first training party, a second model update commitment value of the second training party and a third model update commitment value of the third training party; and a first verification value calculation submodule, configured to multiply the first model update commitment value, the second model update commitment value and the third model update commitment value to obtain the first verification value; wherein the local model update commitment value is obtained by multiplying an exponential operation result of a first preset generator and the local model update value and an exponential operation result of a second preset generator and the local commitment factor.

[0017] Further optionally, the verification module comprises: a global model update value calculation submodule, configured to calculate a sum of the first model update disturbance value, the second model update disturbance value and the third model update disturbance value to obtain a global model update value; a global commitment factor value calculation submodule, configured to calculate a sum of the first commitment factor disturbance value, the second commitment factor disturbance value and the third commitment factor disturbance value to obtain a global commitment factor value; a first exponential operation submodule, configured to perform an exponential operation on the global model update value according to the first preset generator to obtain a first intermediate result; a second exponential operation submodule, configured to perform an exponential operation on the global commitment factor value according to the second preset generator to obtain a second intermediate result; and a second verification value calculation submodule, configured to multiply the first intermediate result and the second intermediate result to obtain the second verification value.

[0018] The technical scheme has the following beneficial effects: the local computing model update value is calculated, and the commitment algorithm is used to generate the commitment result, so that the authenticity and non-tamperability of the information submitted by each training party are ensured; the mask is generated through negotiation between two parties and is subjected to disturbance processing, the masks can offset each other in the aggregation process, so that the single-party information leakage is avoided, and the accuracy of the aggregation result is ensured; finally, through the consistency comparison of the first check value and the second check value, the abnormality or tampering can be effectively found, so that the federated learning process can obtain a correct and reliable global model while ensuring data privacy. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0020] Figure 1 is a flowchart of the federated learning aggregation verification method provided by the embodiment of the present application;

[0021] Figure 2 is a flowchart of the mask value generation method provided by the embodiment of the present application;

[0022] Figure 3 is a flowchart of the disturbance method provided by the embodiment of the present application;

[0023] Figure 4 is a flowchart of the first check value calculation method provided by the embodiment of the present application;

[0024] Figure 5 is a flowchart of the second check value calculation method provided by the embodiment of the present application;

[0025] Figure 6 is a structural schematic diagram of the federated learning aggregation verification system provided by the embodiment of the present application;

[0026] Figure 7 is a structural schematic diagram of the disturbance module for mask generation provided by the embodiment of the present application;

[0027] Figure 8 is a structural schematic diagram of the disturbance module for disturbance provided by the embodiment of the present application;

[0028] Figure 9 is a structural schematic diagram of the check value calculation module provided by the embodiment of the present application;

[0029] Figure 10 is a structural schematic diagram of the verification module provided by the embodiment of the present application.

[0030] Reference signs: 100 - parameter initialization module; 200 - training module; 300 - commitment value calculation module; 400 - disturbance module; 4001 - first mask calculation sub-module; 4002 - second mask calculation sub-module; 4003 - third mask calculation sub-module; 4004 - first disturbance sub-module; 4005 - second disturbance sub-module; 4006 - third disturbance sub-module; 500 - verification value calculation module; 5001 - commitment value acquisition sub-module; 5002 - first verification value calculation sub-module; 600 - verification module; 6001 - global model update value calculation sub-module; 6002 - global commitment factor value calculation sub-module; 6003 - first exponential operation sub-module; 6004 - second exponential operation sub-module; 6005 - second verification value calculation sub-module. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0032] To solve the problems of easy leakage and easy tampering of data in the existing horizontal federated learning scene, the embodiments of the present application provide a federated learning aggregation verification method, Figure 1 which is a flowchart of the federated learning aggregation verification method provided by the embodiments of the present application, as Figure 1 shown, the method comprises:

[0033] S1, each training party obtains an initialization sub-model parameter according to a private data negotiation algorithm.

[0034] Since the data of each training party is local and not shared with each other, to avoid directly exchanging sensitive information, the embodiments adopt a private data negotiation algorithm to realize the secure generation of the initialization parameter.

[0035] Specifically, each training party negotiates to obtain consistent initial model parameters, i.e. initialization sub-module parameters , based on a secure multi-party computation protocol through an interactive manner without leaking their own original data. This process guarantees the correctness and consistency of parameter initialization, and at the same time avoids potential security risks caused by setting parameters by a single party.

[0036] S2, each training party performs local training according to the local training data and the initialization sub-model parameter, and obtains a local model update value.

[0037] After each training party completes the acquisition of the initialized sub-model parameter, the training party conducts local training based on the local training data held by the training party. Specifically, each training party iteratively optimizes the initialized sub-model using local data samples until a preset iteration number or a convergence condition is reached, thereby obtaining a local model update value.

[0038] S3, each training party calculates a local model update commitment value using a commitment algorithm based on the local model update value and a randomly generated local commitment factor.

[0039] To ensure the security and tamper resistance of the local model update value during aggregation, after obtaining the local model update value, each training party randomly selects a local commitment factor and uses the commitment algorithm to calculate a local model update commitment value by taking the commitment factor and the local model update value as inputs. After obtaining the local model update commitment value, each training party discloses the local model update commitment value.

[0040] The commitment algorithm described above is a cryptographic commitment method based on double generators, which is typically represented as the product of the result of the exponentiation of a first preset generator and the local model update value, and the result of the exponentiation of a second preset generator and the local commitment factor. The local model update commitment value can provide verifiability of the correctness of the subsequent aggregation result without revealing the model update value and the commitment factor, thereby ensuring that the update information submitted by each training party is authentic and effective and cannot be tampered with.

[0041] S4, each two training parties negotiate to generate a mask value, and each training party respectively disturbs the corresponding local model update value and local commitment factor based on the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value.

[0042] Each two training parties negotiate to generate a mask value, that is, any two training parties interactively generate a shared mask value through a private data negotiation algorithm, and save the corresponding calculation result locally.

[0043] After obtaining the mask value, each training party disturbs its local model update value using the local mask value to obtain a local model update disturbance value, and discloses the local model update disturbance value. At the same time, the local commitment factor is disturbed using the local mask value to obtain a local commitment factor disturbance value, and the local commitment factor disturbance value is disclosed.

[0044] It should be noted that the disturbance processing method should satisfy that the aggregation of the local model update disturbance values of all training parties cancels out all the mask values to obtain the aggregation value of the pure local model update parameter. The same applies to the local commitment factor disturbance value. Preferably, the disturbance processing method for the local commitment factor using the mask value is consistent with the disturbance processing method for the local model update value.

[0045] S5, calculating a first check value according to the local model update commitment values of all training parties.

[0046] The local model update commitment values generated by each training party are collected and aggregated to obtain a first check value. The first check value can reflect the correctness of the model update process as a whole without leaking the original data of each training party, and serve as a benchmark for subsequent consistency verification.

[0047] S6, calculating a second check value according to the local model update disturbance values and local commitment factor disturbance values of all training parties, and comparing the first check value with the second check value. If they are consistent, continue training, otherwise stop training.

[0048] The local model update disturbance values and local commitment factor disturbance values generated and disclosed by each training party are taken as input to obtain a second check value through aggregation. The second check value corresponds logically to the overall effect of the real model update values and commitment factors of each training party, but the mask factors contained therein can be offset during aggregation, thereby restoring a clean aggregation result.

[0049] After obtaining the second check value, the second check value is compared with the first check value. If they are consistent, it means that the model update values and commitment factors submitted by each training party are real and valid, and the aggregation result is correct and reliable, so the next round of training can be entered. At this time, the result of aggregating all local model update disturbance values can be used as the initial value for the next iteration of all training parties. If they are inconsistent, it means that there is data anomaly or tampering, and the training is immediately stopped to avoid further propagation of incorrect results.

[0050] As an optional implementation, the participants of the training include a first training party, a second training party, and a third training party.

[0051] The first training party locally stores first training data The second training party locally stores second training data The third training party locally stores third training data . The three parties negotiate to obtain an initialized sub-model parameter .

[0052] The first training party , based on the first training data and the initialized sub-model value parameter , locally trains to obtain a first model update value ;

[0053] The second training party , based on the second training data and initializing the sub-model parameters , locally training to obtain a second model update value ;

[0054] a third training party , based on third training data and initializing the sub-model parameters , locally training to obtain a third model update value .

[0055] Figure 2 is a flowchart of the mask value generation method provided by the embodiment of the application, as shown in Figure 2 , each two training parties negotiate to generate a mask value, including:

[0056] S401, the first training party and the second training party generate a first mask value according to a private data negotiation algorithm;

[0057] S402, the second training party and the third training party generate a second mask value according to a private data negotiation algorithm;

[0058] S403, the first training party and the third training party generate a third mask value according to a private data negotiation algorithm.

[0059] After each training party obtains the local model update value, it is necessary to negotiate to generate a mask value to disturb the local model update value, so as to ensure data security.

[0060] the first and second training parties , based on a private data negotiation algorithm, negotiate to obtain a first mask value ;

[0061] the second and third training parties , based on a private data negotiation algorithm, negotiate to obtain a second mask value ;

[0062] the first and third training parties , based on a private data negotiation algorithm, negotiate to obtain a third mask value .

[0063] As an optional implementation, Figure 3 is a flowchart of the disturbance method provided by the embodiment of the application, as shown in Figure 3 , each training party respectively disturbs the corresponding local model update value and local commitment factor according to the local mask value, to obtain a local model update disturbance value and a local commitment factor disturbance value, including:

[0064] The first training party calculates a first difference value of the first mask value and the third mask value, calculates a sum of the first model update value and the first difference value to obtain a first model update disturbance value, and calculates a sum of the first commitment factor and the first difference value to obtain a first commitment factor disturbance value.

[0065] The first training party disturbs the first model update value by using the local first mask value and the third mask value to obtain the first model update disturbance value, and the specific formula is as follows:

[0066]

[0067] wherein, the first model update disturbance value is, the first model update value is, the first mask value is, the third mask value is.

[0068] The first training party disturbs the first commitment factor by using the local first mask value and the third mask value to obtain the first commitment factor disturbance value, and the specific formula is as follows:

[0069]

[0070] wherein, the first commitment factor disturbance value is, the first commitment factor is.

[0071] The second training party calculates a second difference value of the second mask value and the first mask value, calculates a sum of the second model update value and the second difference value to obtain a second model update disturbance value, and calculates a sum of the second commitment factor and the second difference value to obtain a second commitment factor disturbance value.

[0072] The second training party disturbs the second model update value by using the local second mask value and the first mask value to obtain the second model update disturbance value, and the specific formula is as follows:

[0073]

[0074] wherein, the second model update disturbance value is, the second model update value is, the first mask value is, the second mask value is.

[0075] The second training party disturbs the second commitment factor by using the local second mask value and the first mask value to obtain the second commitment factor disturbance value, and the specific formula is as follows:

[0076]

[0077] wherein, is a second commitment factor, is a second commitment factor.

[0078] S406, the third training party calculates a third difference value of the third mask value and the second mask value, and calculates a sum of the third model update value and the third difference value to obtain a third model update disturbance value; calculates a sum of the third commitment factor and the third difference value to obtain a third commitment factor disturbance value; wherein the first model update value, the second model update value and the third model update value are local model update values of the first training party, the second training party and the third training party respectively; the first commitment factor, the second commitment factor and the third commitment factor are local commitment factors of the first training party, the second training party and the third training party respectively.

[0079] The third training party disturbs the third model update value by using the local third mask value and the second mask value to obtain the third model update disturbance value, which is specifically as follows:

[0080]

[0081] wherein, is a third model update disturbance value, is a third model update value, is a third mask value, is a second mask value.

[0082] The third training party disturbs the third commitment factor by using the local third mask value and the second mask value to obtain the third commitment factor disturbance value, which is specifically as follows:

[0083]

[0084] wherein, is a third commitment factor disturbance value, is a third commitment factor.

[0085] The model update disturbance values and the commitment factor disturbance values obtained by the above training parties are all disclosed.

[0086] As an optional implementation, Figure 4 is a flowchart of the verification value calculation method provided by the embodiment of the application, as shown in Figure 4 The first verification value is calculated according to the local model update commitment values of all training parties, including:

[0087] S501, obtaining a first model update commitment value of a first training party, a second model update commitment value of a second training party and a third model update commitment value of a third training party;

[0088] S502, multiply the first model update commitment value, the second model update commitment value and the third model update commitment value to obtain a first check value; wherein the local model update commitment value is obtained by multiplying the exponential operation result of the first preset generator and the local model update value, and the exponential operation result of the second preset generator and the local commitment factor.

[0089] The first model update commitment value is obtained based on a commitment algorithm by the first commitment factor and the first model update value, and is specifically as follows:

[0090]

[0091] wherein, is the first model update commitment value, g is the first preset generator, and h is the second preset generator.

[0092] The second model update commitment value is obtained based on a commitment algorithm by the second commitment factor and the second model update value, and is specifically as follows:

[0093]

[0094] wherein, is the second model update commitment value.

[0095] The third model update commitment value is obtained based on a commitment algorithm by the third commitment factor and the third model update value, and is specifically as follows:

[0096]

[0097] wherein, is the third model update commitment value.

[0098] The first check value is the result of multiplying the first model update value, the second model update value and the third model update value, and is specifically as follows:

[0099]

[0100] The first check value is used for subsequent security check.

[0101] As an optional implementation, Figure 5 is a flowchart of a second check value calculation method provided by an embodiment of the application, as shown in Figure 5 The second check value is calculated according to the local model update disturbance value and the local commitment factor disturbance value of all training parties, and includes:

[0102] S601, calculate the sum of the first model update disturbance value, the second model update disturbance value and the third model update disturbance value to obtain a global model update value.

[0103] According to the disclosed first model update disturbance value , the second model update disturbance value and the third model update disturbance value , the global model update value is calculated .

[0104] 602, the sum of the first commitment factor disturbance value, the second commitment factor disturbance value and the third commitment factor disturbance value is calculated to obtain the global commitment factor value.

[0105] According to the first commitment factor disturbance value disclosed , the second commitment factor disturbance value and the third commitment factor disturbance value , the global commitment factor value is calculated .

[0106] S603, according to the first preset generating element, the global model update value is exponentially operated to obtain the first intermediate result.

[0107] According to the first preset generating element, the global model update value is calculated to obtain the first intermediate result .

[0108] S604, according to the second preset generating element, the global commitment factor value is exponentially operated to obtain the second intermediate result.

[0109] According to the second preset generating element, the global commitment factor is calculated to obtain the second intermediate result .

[0110] S605, the first intermediate result is multiplied by the second intermediate result to obtain the second verification value.

[0111] According to the first intermediate result and the second intermediate result , the second verification value is calculated .

[0112] Then, whether the equation is established is verified, if the intermediate data is not tampered, the equation is established, that is, it is proved that the aggregation result is valid, and the global model update value is used as the initial value of the model of each training party for next iteration; if the equation is not established, it is proved that the aggregation result is wrong, and the iteration is stopped.

[0113] The embodiment of the application also provides a federated learning aggregation verification system, Figure 6 is a structural schematic diagram of the federated learning aggregation verification system provided by the embodiment of the application, as Figure 6 shown, the system comprises:

[0114] The parameter initialization module 100 is used for obtaining the initialized sub-model parameter according to the private data negotiation algorithm by each training party.

[0115] Since the data of each training party is local and not shared with each other, to avoid directly exchanging sensitive information, the embodiment adopts a private data negotiation algorithm to realize the secure generation of the initial parameters.

[0116] Specifically, based on a secure multi-party computation protocol, each training party negotiates consistent initial model parameters, i.e., initialization sub-module parameters, through interaction without revealing the original data of each party. This process ensures the correctness and consistency of parameter initialization, while avoiding potential security risks caused by a single party setting parameters.

[0117] The training module 200 is configured to perform local training according to the local training data and the initialization sub-model parameters, and obtain local model update values.

[0118] After obtaining the initialization sub-model parameters, each training party performs local training based on the local training data held by each party. Specifically, each training party iteratively optimizes the initialization sub-model using local data samples until a preset iteration number or a convergence condition is reached, thereby obtaining local model update values.

[0119] The commitment value calculation module 300 is configured to calculate local model update commitment values using a commitment algorithm according to the local model update values and randomly generated local commitment factors.

[0120] To ensure the security and tamper resistance of the local model update values during aggregation, after obtaining the local model update values, each training party randomly selects a local commitment factor, and uses the commitment algorithm to calculate the local model update commitment values by taking the commitment factor and the local model update values as inputs. After obtaining the local model update commitment values, each training party discloses the local model update commitment values.

[0121] The commitment algorithm described above is a cryptographic commitment method based on double generators, which is usually represented as the product of the result of the exponentiation of the first preset generator and the local model update value, and the result of the exponentiation of the second preset generator and the local commitment factor. The local model update commitment value can provide verifiability of the correctness of the subsequent aggregation result without revealing the model update value and the commitment factor, thereby ensuring that the update information submitted by each training party is real and effective and cannot be tampered with.

[0122] The disturbance module 400 is configured to negotiate a mask value for each two training parties, and each training party respectively disturbs the corresponding local model update value and local commitment factor according to the local mask value, to obtain a local model update disturbance value and a local commitment factor disturbance value.

[0123] Each pair of training parties negotiates a mask value, that is, any two training parties interact to generate a shared mask value through a private data negotiation algorithm, and save the corresponding calculation result locally.

[0124] After obtaining the mask value, each training party uses the local mask value to disturb the local model update value to obtain a local model update disturbance value, and discloses the local model update disturbance value. At the same time, the local mask value is used to disturb the local commitment factor respectively to obtain a local commitment factor disturbance value, and the local commitment factor disturbance value is disclosed.

[0125] It should be noted that the disturbance processing mode should satisfy that the aggregation of the local model update disturbance values of all training parties cancels out all mask values to obtain the aggregation value of the pure local model update parameter. The local commitment factor disturbance value is the same. Preferably, the disturbance processing mode of the mask value on the local commitment factor is consistent with the disturbance processing mode of the local model update value.

[0126] The verification value calculation module 500 is configured to calculate a first verification value according to the local model update commitment values of all training parties.

[0127] The local model update commitment values generated by each training party are collected, and all local model update commitment values are aggregated to obtain a first verification value. The first verification value can reflect the correctness of the model update process as a whole without leaking the original data of each training party, and serve as a benchmark for subsequent consistency verification.

[0128] The verification module 600 is configured to calculate a second verification value according to the local model update disturbance values and the local commitment factor disturbance values of all training parties, compare the first verification value with the second verification value, and if they are consistent, continue training, otherwise stop training.

[0129] The local model update disturbance values and the local commitment factor disturbance values generated and disclosed by each training party are inputted, and a second verification value is obtained through aggregation operation. The second verification value corresponds to the overall effect of the real model update value and the commitment factor of each training party in logic, but the mask factor contained therein can be cancelled out in the aggregation process, so that the clean aggregation result is restored.

[0130] After obtaining the second verification value, the second verification value is compared with the first verification value. If they are consistent, it means that the model update value and the commitment factor submitted by each training party are real and effective, and the aggregation result is correct and reliable, and the next round of training can be entered, at this time the result of the aggregation of all local model update disturbance values can be used as the initial value for the next iteration of all training parties; if they are inconsistent, it means that there is data anomaly or tampering, and the training is immediately stopped to avoid further propagation of the error result.

[0131] As an optional implementation method, the participants in the training include a first training party, a second training party, and a third training party.

[0132] First Training Method The first training data is stored locally. Second training method The second training data is stored locally. The third training method Third training data is stored locally. The three parties negotiated to obtain the initialization parameters for the sub-model. .

[0133] First Training Method Based on the first training data and initializing sub-model value parameters The first model update value is obtained through local training. ;

[0134] Second training method Based on the second training data and initializing sub-model parameters The second model update value is obtained through local training. ;

[0135] Third training method Based on third training data and initializing sub-model parameters The third model update value is obtained through local training. .

[0136] Figure 7 This is a schematic diagram of the scrambling module for mask generation provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the disturbance module 400 includes:

[0137] The first mask calculation submodule 4001 is used by the first training party and the second training party to generate the first mask value according to the private data negotiation algorithm.

[0138] The second mask calculation submodule 4002 is used by the second training party and the third training party to generate the second mask value according to the private data negotiation algorithm.

[0139] The third mask calculation submodule 4003 is used by the first training party and the third training party to generate the third mask value based on the private data negotiation algorithm.

[0140] After each training party obtains the local model update values, they need to negotiate to generate a mask value to scramble the local model update values ​​in order to ensure data security.

[0141] First and second training methods , the first mask value is obtained through negotiation based on a private data negotiation algorithm ;

[0142] the second and third training parties , the second mask value is obtained through negotiation based on a private data negotiation algorithm ;

[0143] the first and third training parties , the third mask value is obtained through negotiation based on a private data negotiation algorithm .

[0144] As an optional implementation, Figure 8 is a structural schematic diagram of the disturbance module for disturbance provided by the embodiment of the application, as shown in the figure, Figure 8 The disturbance module 400 includes:

[0145] The first disturbance submodule 4004 is configured to calculate a first difference value between the first mask value and the third mask value, calculate a sum of the first model update value and the first difference value to obtain a first model update disturbance value, and calculate a sum of the first commitment factor and the first difference value to obtain a first commitment factor disturbance value.

[0146] The first training party disturbs the first model update value by using the local first mask value and the third mask value to obtain the first model update disturbance value, and the specific formula is as follows:

[0147]

[0148] , wherein, the first model update disturbance value, the first model update value, the first mask value, the third mask value.

[0149] The first training party disturbs the first commitment factor by using the local first mask value and the third mask value to obtain the first commitment factor disturbance value, and the specific formula is as follows:

[0150]

[0151] , wherein, the first commitment factor disturbance value, the first commitment factor.

[0152] The second disturbance submodule 4005 is configured to calculate a second difference value between the second mask value and the first mask value, calculate a sum of the second model update value and the second difference value to obtain a second model update disturbance value, and calculate a sum of the second commitment factor and the second difference value to obtain a second commitment factor disturbance value.

[0153] The second training party uses the local second mask value and the first mask value to disturb the second model update value to obtain a second model update disturbance value, specifically as follows:

[0154]

[0155] wherein, the second model update disturbance value is, the second model update value is, the first mask value is, the second mask value is.

[0156] The second training party uses the local second mask value and the first mask value to disturb the second commitment factor to obtain a second commitment factor disturbance value, specifically as follows:

[0157]

[0158] wherein, the second commitment factor disturbance value is, the second commitment factor is.

[0159] The third disturbance submodule 4006 is configured to calculate a third difference value between a third mask value and a second mask value, and calculate a sum of the third difference value and a third model update value to obtain a third model update disturbance value; calculate a sum of the third difference value and a third commitment factor to obtain a third commitment factor disturbance value; wherein the first model update value, the second model update value and the third model update value are local model update values of the first training party, the second training party and the third training party respectively; and the first commitment factor, the second commitment factor and the third commitment factor are local commitment factors of the first training party, the second training party and the third training party respectively.

[0160] The third training party uses the local third mask value and the second mask value to disturb the third model update value to obtain a third model update disturbance value, specifically as follows:

[0161]

[0162] wherein, the third model update disturbance value is, the third model update value is, the third mask value is, the second mask value is.

[0163] The third training party uses the local third mask value and the second mask value to disturb the third commitment factor to obtain a third commitment factor disturbance value, specifically as follows:

[0164]

[0165] wherein, is a third commitment factor. is a third commitment factor.

[0166] The model update disturbance value and the commitment factor disturbance value obtained by each training party are disclosed.

[0167] As an optional implementation, Figure 9 is a structural diagram of a verification value calculation module provided by an embodiment of the application, as shown in the figure, Figure 9 the verification value calculation module 500 comprises:

[0168] The commitment value acquisition submodule 5001 is configured to acquire a first model update commitment value of a first training party, a second model update commitment value of a second training party, and a third model update commitment value of a third training party.

[0169] The first verification value calculation submodule 5002 is configured to multiply the first model update commitment value, the second model update commitment value, and the third model update commitment value to obtain a first verification value; wherein the local model update commitment value is obtained by multiplying an exponential operation result of a first preset generator and a local model update value, and an exponential operation result of a second preset generator and a local commitment factor.

[0170] The first model update commitment value is obtained by the first commitment factor and a first model update value based on a commitment algorithm, and is specifically as follows:

[0171]

[0172] wherein, is the first model update commitment value, g is the first preset generator, and h is the second preset generator.

[0173] The second model update commitment value is obtained by the second commitment factor and a second model update value based on the commitment algorithm, and is specifically as follows:

[0174]

[0175] wherein, is the second model update commitment value.

[0176] The third model update commitment value is obtained by the third commitment factor and a third model update value based on the commitment algorithm, and is specifically as follows:

[0177]

[0178] wherein, is the third model update commitment value.

[0179] The first check value is a result of multiplication of the first model update value, the second model update value and the third model update value, as shown in the following formula:

[0180]

[0181] The first check value is used for subsequent security check.

[0182] As an optional implementation, Figure 10 is a structural diagram of the verification module provided by the embodiment of the application, as shown in Figure 10 The verification module 600 includes:

[0183] The global model update value calculation sub-module 6001 is configured to calculate a sum of the first model update disturbance value, the second model update disturbance value and the third model update disturbance value to obtain the global model update value.

[0184] According to the disclosed first model update disturbance value , the second model update disturbance value and the third model update disturbance value , the global model update value is calculated.

[0185] The global commitment factor value calculation sub-module 6002 is configured to calculate a sum of the first commitment factor disturbance value, the second commitment factor disturbance value and the third commitment factor disturbance value to obtain the global commitment factor value.

[0186] According to the disclosed first commitment factor disturbance value , the second commitment factor disturbance value and the third commitment factor disturbance value , the global commitment factor value is calculated.

[0187] The first exponent operation sub-module 6003 is configured to perform an exponent operation on the global model update value according to the first preset generator to obtain a first intermediate result.

[0188] The first intermediate result is calculated according to the first preset generator and the global model update value.

[0189] The second exponent operation sub-module 6004 is configured to perform an exponent operation on the global commitment factor value according to the second preset generator to obtain a second intermediate result.

[0190] The second intermediate result is calculated according to the second preset generator and the global commitment factor.

[0191] The second check value calculation sub-module 6005 is configured to multiply the first intermediate result and the second intermediate result to obtain the second check value.​​

[0192] According to the first intermediate result With the second intermediate result Calculate the second check value .

[0193] Then verify the equation If the intermediate data is not tampered with, the equation is established, that is, the aggregation result is valid, and the global model update value is used As the initial value of the model of each training party for the next iteration; if the equation is not established, it proves that the aggregation result is wrong, and the iteration is stopped.

[0194] The above technical solutions have the following beneficial effects: by calculating the model update value locally and generating the commitment result using the commitment algorithm, the authenticity and tamper resistance of the information submitted by each training party are ensured; by negotiating with each other to generate a mask and perform disturbance processing, the masks can cancel each other out during aggregation, thereby avoiding the leakage of single-party information and ensuring the accuracy of the aggregation result; finally, by comparing the consistency of the first check value and the second check value, abnormal or tampered conditions can be effectively found, ensuring that the federated learning process obtains a correct and reliable global model while protecting data privacy.

[0195] The specific embodiments of the above invention further illustrate the purpose, technical solutions and beneficial effects of the invention. It should be understood that the above content is only a specific embodiment of the invention and is not used to limit the protection scope of the invention. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the invention should be included in the protection scope of the invention.

Claims

1. A federated learning aggregation verification method, characterized in that, The method comprises the following steps: Each training party negotiates an initial sub-model parameter according to private data; Each training party performs local training according to local training data and the initial sub-model parameter to obtain a local model update value; Each training party calculates a local model update commitment value by using a commitment algorithm according to the local model update value and a randomly generated local commitment factor; Each two training parties negotiate to generate a mask value, and each training party respectively disturbs the corresponding local model update value and the local commitment factor according to the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value; wherein, the two training parties include a first training party, a second training party and a third training party; the negotiation of the mask value by each two training parties comprises: the first training party and the second training party generate a first mask value according to a private data negotiation algorithm; the second training party and the third training party generate a second mask value according to the private data negotiation algorithm; and the first training party and the third training party generate a third mask value according to the private data negotiation algorithm; Each training party respectively disturbs the corresponding local model update value and the local commitment factor according to the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value, which comprises: the first training party calculates a first difference value between the first mask value and the third mask value, calculates a sum of the first model update value and the first difference value to obtain a first model update disturbance value; and calculates a sum of the first commitment factor and the first difference value to obtain a first commitment factor disturbance value; the second training party calculates a second difference value between the second mask value and the first mask value, calculates a sum of the second model update value and the second difference value to obtain a second model update disturbance value; and calculates a sum of the second commitment factor and the second difference value to obtain a second commitment factor disturbance value; and the third training party calculates a third difference value between the third mask value and the second mask value, calculates a sum of the third model update value and the third difference value to obtain a third model update disturbance value; and calculates a sum of the third commitment factor and the third difference value to obtain a third commitment factor disturbance value; wherein, the first model update value, the second model update value and the third model update value are the local model update values of the first training party, the second training party and the third training party respectively; and the first commitment factor, the second commitment factor and the third commitment factor are the local commitment factors of the first training party, the second training party and the third training party respectively; A first check value is calculated according to the local model update commitment values of all training parties; A second check value is calculated according to the local model update disturbance values and the local commitment factor disturbance values of all training parties, and whether the first check value and the second check value are consistent is compared; if yes, the training is continued; otherwise, the training is stopped.

2. The federated learning aggregation verification method according to claim 1, characterized in that, The calculation of the first check value according to the local model update commitment values of all training parties comprises: The first model update commitment value of the first training party, the second model update commitment value of the second training party and the third model update commitment value of the third training party are obtained; The first model update commitment value, the second model update commitment value and the third model update commitment value are multiplied to obtain the first check value. The local model update commitment value is obtained by multiplying the exponential operation result of the first preset generator and the local model update value and the exponential operation result of the second preset generator and the local commitment factor.

3. The federated learning aggregation verification method according to claim 2, characterized in that, The second check value is calculated according to the local model update disturbance value and the local commitment factor disturbance value of all training parties, and includes: The sum of the first model update disturbance value, the second model update disturbance value and the third model update disturbance value is calculated to obtain the global model update value; The sum of the first commitment factor disturbance value, the second commitment factor disturbance value and the third commitment factor disturbance value is calculated to obtain the global commitment factor value; The global model update value is subjected to exponential operation according to the first preset generator to obtain a first intermediate result; The global commitment factor value is subjected to exponential operation according to the second preset generator to obtain a second intermediate result; The first intermediate result is multiplied by the second intermediate result to obtain the second check value.

4. A federated learning aggregation verification system, comprising: It includes: The parameter initialization module is used for each training party to obtain the initialized sub-model parameter according to the private data negotiation algorithm; The training module is used for each training party to perform local training according to the local training data and the initialized sub-model parameter to obtain the local model update value; The commitment value calculation module is used for each training party to calculate the local model update commitment value by using the commitment algorithm according to the local model update value and the randomly generated local commitment factor; The disturbance module is used for each two training parties to negotiate a mask value, and each training party respectively disturbs the corresponding local model update value and local commitment factor according to the local mask value to obtain a local model update disturbance value and a local commitment factor disturbance value; wherein, the training parties include a first training party, a second training party and a third training party; the disturbance module comprises: a first mask calculation submodule, used for the first training party and the second training party to generate a first mask value according to a private data negotiation algorithm; a second mask calculation submodule, used for the second training party and the third training party to generate a second mask value according to the private data negotiation algorithm; a third mask calculation submodule, used for the first training party and the third training party to generate a third mask value according to the private data negotiation algorithm; a first disturbance submodule, used for the first training party to calculate a first difference value of the first mask value and the third mask value, calculate a sum of a first model update value and the first difference value to obtain a first model update disturbance value, and calculate a sum of a first commitment factor and the first difference value to obtain a first commitment factor disturbance value; a second disturbance submodule, used for the second training party to calculate a second difference value of the second mask value and the first mask value, calculate a sum of a second model update value and the second difference value to obtain a second model update disturbance value, and calculate a sum of a second commitment factor and the second difference value to obtain a second commitment factor disturbance value; and a third disturbance submodule, used for the third training party to calculate a third difference value of the third mask value and the second mask value, calculate a sum of a third model update value and the third difference value to obtain a third model update disturbance value, and calculate a sum of a third commitment factor and the third difference value to obtain a third commitment factor disturbance value; wherein, the first model update value, the second model update value and the third model update value are local model update values of the first training party, the second training party and the third training party respectively; and the first commitment factor, the second commitment factor and the third commitment factor are local commitment factors of the first training party, the second training party and the third training party respectively; The verification value calculation module is used for calculating a first verification value according to the local model update commitment values of all the training parties; The verification module is used for calculating a second verification value according to the local model update disturbance values and the local commitment factor disturbance values of all the training parties, comparing whether the first verification value and the second verification value are consistent, and if consistent, continuing training, otherwise, stopping training.

5. The federated learning aggregation verification system of claim 4, wherein, The verification value calculation module comprises: The commitment value acquisition submodule is used for acquiring a first model update commitment value of the first training party, a second model update commitment value of the second training party and a third model update commitment value of the third training party; The first verification value calculation submodule is used for multiplying the first model update commitment value, the second model update commitment value and the third model update commitment value to obtain the first verification value; Wherein, the local model update commitment value is obtained by multiplying an exponential operation result of a first preset generator and a local model update value, and an exponential operation result of a second preset generator and a local commitment factor.

6. The federated learning aggregation verification system of claim 5, wherein, The verification module comprises: The global model update value calculation submodule is used for calculating a sum of the first model update disturbance value, the second model update disturbance value and the third model update disturbance value to obtain a global model update value; The global commitment factor value calculation submodule is configured to calculate a sum of the first commitment factor disturbance value, the second commitment factor disturbance value and the third commitment factor disturbance value to obtain a global commitment factor value; The first exponential operation submodule is configured to perform exponential operation on the global model update value according to the first preset generator to obtain a first intermediate result; The second exponential operation submodule is configured to perform exponential operation on the global commitment factor value according to the second preset generator to obtain a second intermediate result; The second check value calculation submodule is configured to multiply the first intermediate result and the second intermediate result to obtain the second check value.

Images