Data processing method and device

Abstract

The invention provides a data processing method and device. When the method is applied to a target data provider in a federated learning-based data processing system, the target data provider, other data providers in the processing system and a target data user form a federated learning cluster; the method comprises the steps that a target data provider obtains a to-be-trained global model issued by a target data user; performing model training on the global model based on the local data to obtain a trained local model, further generating an integrity commitment for the local model, and sending the integrity commitment to a target data user; evaluating a quality score of the local model based on a reference model disclosed by the target data user, further generating an authenticity commitment for the quality score, and sending the quality score and the authenticity commitment to the target data user; and cooperating with the target data user to verify whether the quality score is true based on the authenticity commitment in an interactive verification mode.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technologies, and in particular, to a data processing method and apparatus. Background Art

[0002] Currently, Artificial Intelligence (AI) technology has penetrated all aspects of daily life. Among them, in the process of training an AI model, a Data Acquirer (DA) can access the data owned by a Data Provider (DP) that meets the needs of the data acquirer, and thus train a high-quality AI model through these data. To promote data circulation and at the same time reduce the risk of privacy leakage caused by data transactions, a model can be trained through, for example, federated learning technology. Federated learning can complete model training by sharing the parameter gradients of the model without exchanging the original data, thereby achieving the goal of "the original data does not leave the domain, and the data can be used but not seen".

[0003] However, although the federated learning method can protect data privacy in the model training stage, there is still a risk of privacy leakage in the model evaluation stage after the model training is completed. Summary of the Invention

[0004] A data processing method and apparatus provided by the embodiments of this specification.

[0005] According to a first aspect of the embodiments of this specification, a data processing method is provided, which is applied to a data processing system based on federated learning. The data processing system includes a plurality of data providers and a plurality of data users; the method includes: A target data user combines a cluster for federated learning with multiple target data providers, and publishes a global model to be trained to the target data providers in the cluster; Each target data provider performs model training on the global model based on local data, obtains a trained local model, and further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user; After receiving the integrity commitments sent by the respective target data providers, the target data user discloses a benchmark model to the respective target data providers; wherein the benchmark model is used to evaluate the local models trained by the respective target data; Each target data provider evaluates the quality score of the trained local model based on the benchmark model, and further generates a authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user; After receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments; and further determines the target quality scores with high scores from the verified quality scores, and obtains the target local model from the target data provider that sent the target quality scores. The target data user verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

[0006] Optionally, generating the integrity commitment for the local model includes: Calculating the integrity commitment of the local model based on a preset multi-party security protocol; wherein, the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider.

[0007] Optionally, the target data user verifying whether the target local model is complete based on the integrity commitment of the target local model includes: The target data user calculates Whether it holds; if it holds, it is determined that the target local model is complete; Wherein, is the round of federated learning in which the current training is located, i is the serial number of the target data user, is the local model obtained from the target data provider, is the integrity commitment sent by the target data provider, is the session key generated by the multi-party security protocol, is the global key held by the target data user.

[0008] Optionally, the target data user discloses the benchmark model to the respective target data providers, and obtains it through the following method: Determine whether the current round of model training is the first round of federated learning; If so, the target data user trains the model based on local data to obtain the benchmark model; if not, the target data user obtains the local model from the data provider with the highest verified quality score in the previous round to use the local model as the benchmark model for the current round of model training.

[0009] Optionally, the target data provider evaluates the quality score of the local model based on the benchmark model, including: The target data provider calculates the cosine distance of the model parameters between the benchmark model and the local model, and takes the cosine distance as the quality score of the local model.

[0010] Optionally, generating a authenticity commitment for the quality score includes: The target data provider generates an authenticity commitment for the quality score through the formula ; where is the value of the authenticity commitment, is the model vector of the local model of the target data party, is the second basis, is the model vector of the reference model, is the first basis, represents the quality score of the local model of the target data party, represents a random number selected by the target data provider.

[0011] Optionally, after receiving the quality scores and authenticity commitments sent by each target data provider, the target data user verifies whether the quality scores are authentic based on the authenticity commitments, including: After receiving the quality scores and authenticity commitments sent by each target data provider, the target data user adopts an interactive verification method to verify whether the quality scores are authentic based on the authenticity commitments.

[0012] Optionally, the method of adopting an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment includes: The target data provider obtains the reference model , the local model , the first basis and the second basis ; and performs multiple rounds of interactive verification using the following steps until the length of S1, the target data provider splits into containing the first half of the model parameters and containing the second half of the model parameters, splits into containing the first half of the model parameters and containing the second half of the model parameters, splits into containing the first half of the element points and containing the second half of the element points, and splits the second basis into containing the first half of the element points and containing the second half of the element points; S2, the target data provider calculates , , , , and further send and to the target data user; S3. The target data generator randomly generates a challenge value x and calculates , , , , ; and further send x, , , , and to the target data provider; where is the inverse element of the challenge value x; S4. The target data provider proves whether holds; if it holds, then determine , , and as the new , , and , and repeat S1; S5. When the length of is equal to 1, the target data provider sends the with a length of 1 to the target data user; S6. The target data user calculates , , with a length of 1 calculated according to S3, and calculates . If , it is determined that the verification passes, otherwise the verification fails.

[0013] Optionally, the first basis and the second basis include two sets of independent element point sets selected from the elliptic curve group defined on ; where the number of element points in the element point set is the same as the number of model parameters in the global model.

[0014] Optionally, determining the target quality score with a high score from the verified quality scores includes: The target data user performs joint sampling on all verified quality scores and calculates the probability value ; where D is all verified quality scores, and t is the sampling temperature for adjusting the output distribution; Determine the quality score exceeding the probability value as the target quality score of high score.

[0015] According to the second aspect of the embodiments of this specification, a data processing method is provided, which is applied to a target data user in a data processing system based on federated learning. The target data user and multiple target data providers in the processing system form a federated learning cluster; the method includes: The target data user publishes the global model to be trained to the target data providers in the cluster. Receive the integrity commitments for the local models returned by each target data provider; wherein, the local model is the local model obtained by the target data provider training the global model based on local data. Disclose the benchmark model to each target data provider; wherein, the benchmark model is used to evaluate the local models sent by each target data provider. Receive the quality scores obtained by evaluating the layout models based on the benchmark model sent by each target data provider, and the authenticity commitments for the quality scores. Verify whether the quality scores are true based on the authenticity commitments, and further determine the target quality scores of high score from the verified quality scores, and obtain the target local models from the target data providers that send the target quality scores. Verify whether the target local models are complete based on the integrity commitments of the target local models, and further aggregate the verified target local models into a new global model.

[0016] According to the third aspect of the embodiments of this specification, a data processing method is provided, which is applied to a target data provider in a data processing system based on federated learning. The target data provider and other data providers and target data users in the processing system form a federated learning cluster; the method includes: The target data provider obtains the global model to be trained published by the target data user. Train the global model based on local data to obtain a trained local model, and further generate an integrity commitment for the local model, and send the integrity commitment to the target data user. Evaluate the quality score of the local model based on the benchmark model disclosed by the target data user, and further generate an authenticity commitment for the quality score, and send the quality score and the authenticity commitment to the target data user. Cooperate with the target data user to verify whether the quality score is true based on the authenticity commitment in an interactive verification manner.

[0017] According to a fourth aspect of the embodiments of the present specification, a data processing device is provided, which is applied to a target data user in a data processing system based on federated learning. The target data user and multiple target data providers in the processing system form a federated learning cluster; the device includes: A first publishing unit, where the target data user publishes a global model to be trained to the target data providers in the cluster; A first receiving unit, which receives integrity commitments for the local models returned by each target data provider; wherein, the local model is a local model obtained by the target data provider training the global model based on local data; A second publishing unit, which discloses a benchmark model to each of the target data providers; wherein, the benchmark model is used to evaluate the local models sent by each of the target data providers; A second receiving unit, which receives the quality scores obtained by evaluating the layout model based on the benchmark model sent by each of the target data providers, and authenticity commitments for the quality scores; An authenticity verification unit, which verifies whether the quality scores are authentic based on the authenticity commitments, further determines high-score target quality scores from the verified quality scores, and obtains target local models from the target data providers that send the target quality scores; An integrity verification unit, which verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

[0018] According to a fifth aspect of the embodiments of the present specification, a data processing device is provided, which is applied to a target data provider in a data processing system based on federated learning. The target data provider and other data providers and target data users in the processing system form a federated learning cluster; the device includes: A model acquisition unit, where the target data provider acquires the global model to be trained published by the target data user; A model training unit, which trains the global model based on local data to obtain a trained local model, and further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user; A model evaluation unit, which evaluates the quality score of the local model based on the benchmark model disclosed by the target data user, and further generates an authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user; The authenticity verification unit, in cooperation with the target data user, adopts an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment.

[0019] According to a sixth aspect of the embodiments of the present specification, there is provided a computing device, including: A processor; A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the data processing method as described in any one of the previous items.

[0020] The embodiments of the present specification provide a data processing solution. By zero-knowledge proof, it is ensured that the data provider can participate in the evaluation of the local model without disclosing the model details during the model evaluation stage, thereby avoiding the risk of privacy leakage and ensuring the security and fairness of the data provider and the data user. Specifically, after training a local model based on federated learning, the data provider first calculates an integrity commitment for the local model to ensure that the local model will not be tampered with during the evaluation process; then evaluates the local model through a benchmark model provided by the data user to obtain the quality score of the local model; while the data user verifies the quality score through an interactive zero-knowledge proof verification method to ensure the authenticity of the evaluation result, and selects a high-quality local model for fusion according to the quality score. Before fusion, the integrity commitment of the local model is verified to ensure the integrity of the local model. In this way, through the dual commitments of integrity commitment and authenticity commitment, the reliability and consistency of the local model are ensured, and privacy protection during the model evaluation process is achieved. Description of the Drawings

[0021] Figure 1 Is the architecture diagram of the data processing system provided by an embodiment of the present specification; Figure 2 Is the flowchart of the data processing method provided by an embodiment of the present specification; Figure 3 Is the schematic diagram of the zero-knowledge protocol provided by an embodiment of the present specification; Figure 4 Is the schematic diagram of the process of the verification method during interaction provided by an embodiment of the present specification; Figure 5 Is the schematic flowchart of the target data user provided by an embodiment of the present specification; Figure 6 Is the schematic flowchart of the target data provider provided by an embodiment of the present specification; Figure 7 Is the hardware structure diagram of the data processing device provided by an embodiment of the present specification; Figure 8Module of the data processing device provided by an embodiment of this specification; Figure 9 Module of the data processing device provided by an embodiment of this specification. Detailed implementation manners

[0022] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0023] The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit this specification. The singular forms "a", "the", and "said" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0025] In the existing data processing solutions applied to federated learning as described above, although data privacy can be protected during the model training phase, there is still a risk of privacy leakage during the model evaluation phase after the model training is completed. Therefore, a data processing solution that can meet privacy protection during model evaluation is needed.

[0026] In the existing data trading methods applied to federated learning, the model quality of the data provider is evaluated without effectively protecting the data privacy of the data provider. Zero-knowledge proof technology is a cryptographic technology. The prover can convince the verifier of the correctness of the assertion without revealing any detailed information about the assertion. In recent years, with the development of algorithms such as Zero-Knowledge Succinct Non-interactive Arguments of Knowledge (zk-SNARK), some research has proposed that the rights and interests protection in machine learning systems can be achieved based on verifiable computing methods. Currently, there is little research work using this technology in model quality evaluation.

[0027] In view of this, the present invention proposes a brand-new data processing solution. Through zero-knowledge proof, it ensures that in the model evaluation stage, the data provider can participate in the evaluation of the local model without disclosing model details, thereby avoiding the risk of privacy leakage and ensuring the security and fairness of both the data provider and the data user.

[0028] Please refer to the following Figure 1 system architecture diagram of the data processing system based on federated learning shown. The system may include several data providers and several data users. Among them, the data user can publish the global model to be trained to the data providers participating in federated learning, and the data providers locally store the local data for training the model in federated learning.

[0029] A data user and multiple data providers can form a cluster for federated learning. To distinguish from data providers and data users outside the cluster, as Figure 1 shown, the data user within the cluster can be called the target data user, and the data provider within the cluster can be called the target data provider. Generally, there can be multiple different target data providers within the cluster, such as Figure 1 the target data provider 1 to the target data provider 6 shown.

[0030] During the federated learning process, after the target data provider uses the local data to train the global model published by the target data user to obtain the trained local model, it first calculates the integrity commitment for the local model to ensure that the local model will not be tampered with during the evaluation process; then it evaluates the local model through the benchmark model provided by the data user to obtain the quality score of the local model; while the data user verifies the quality score through the verification method of interactive zero-knowledge proof to ensure the authenticity of the evaluation result, and selects high-quality local models for fusion according to the quality score. Before fusion, it verifies the integrity commitment of the local model to ensure the integrity of the local model.

[0031] Through the above solution, in a data processing system based on federated learning, through the dual commitments of integrity commitment and authenticity commitment, while effectively evaluating the quality of the local models trained by data providers, it ensures both the reliability and consistency of the local models and the privacy security of the local models.

[0032] Next, please refer to Figure 2 the schematic diagram of the data processing method flow shown below. This method can be applied to the aforementioned data processing system based on federated learning, and the method may include the following steps: Step 210, the target data user combines a cluster for federated learning with multiple target data providers, and publishes the global model to be trained to the target data providers in the cluster.

[0033] Suppose there are n target data providers in total, and the number of model parameters of the global model published by the target data user is m. Considering that in practical applications, different global models have different numbers of model parameters, a standard number can be defined. When the number of model parameters is less than the standard number, the insufficient part can be filled with 0.

[0034] Exemplarily, the standard number can be set to , where q can be any integer. In this way, when m is less than , 0s can be appended after the existing model parameters until the number of model parameters reaches ones.

[0035] The target data user and the target data providers can construct a multi-party secure protocol through pre-negotiation, such as a zero-knowledge proof protocol.

[0036] Exemplarily, the above negotiation can be completed using a generator such as VOLE (Vector Oblivious Linear Evaluation). Because the VOLE generator has a low communication complexity, it is often used to construct secure multi-party computing protocols.

[0037] In addition, the target data user and the target data providers can also determine two sets of bases required for the multi-party secure protocol according to the VOLE generator. The two sets of bases can be, for example, a set of two independent element points selected from the elliptic curve group defined on ; where the number of element points in the set is the same as the number of model parameters in the above global model (m). Exemplarily, the two sets of bases can be denoted as: the first base and the second base

[0038] Step 220: Each target data provider trains the global model based on local data to obtain a trained local model, and further generates an integrity commitment for the local model.

[0039] In this embodiment, each target data provider can calculate the integrity commitment of its respective trained local model based on the multi-party security protocol determined through the foregoing negotiation.

[0040] Assume that the current round of training is the t-th round of federated learning. The global model released by the target data user can be denoted as . The following will take the i-th target data provider as an example to introduce the steps that each target data provider needs to execute (i.e., the execution process of each target data provider is the same).

[0041] After obtaining the global model , the target data provider uses local data to perform model training on the basis of this global model to obtain the trained local model ; then as Figure 3 shown, the target data provider and the target data user can jointly execute the multi-party security protocol through the VOLE generator. In this way, the target data provider can obtain the integrity commitment for the local model generated by this multi-party security protocol , while the target data user can obtain the session key for the result of this negotiation (i.e., the integrity commitment ) generated by this multi-party security protocol .

[0042] The integrity commitment and the session key generated by this multi-party security protocol satisfy the following linear relationship:

[0043] where is the global key held by the target data user (only known to the target data user). This linear relationship is also the basis for the target data user to verify the integrity commitment in the subsequent steps, and this part of the verification process will be introduced in the subsequent steps.

[0044] Step 221: Each target data provider sends its respective generated integrity commitment to the target data user.

[0045] After the target data provider obtains the integrity commitment , it can send this integrity commitment to the target data user.

[0046] Step 230: After the target data user receives the integrity commitments sent by the respective target data providers, the target data user discloses the benchmark model to the respective target data providers; wherein, the benchmark model is used to evaluate the local models trained from the respective target data.

[0047] When the target data user receives the integrity commitments sent by all the target data providers it can disclose the benchmark model for this round of training .

[0048] The benchmark model can be obtained in the following manner: If the quality of the local data of the target data user is relatively high, or this round of model training is the first round of federated learning training, then the target data user can perform one round of model training based on the local data to obtain the benchmark model Otherwise, the target data user needs to obtain the local model of the data provider that passed the verification and had the highest model quality score in the previous round and use this local model as the benchmark model for this round of model training .

[0049] It should be noted that in the scenario of data trading, the target data user needs to purchase from the data provider to obtain the benchmark model

[0050] Step 240: The respective target data providers evaluate the quality scores of the local models trained based on the benchmark model and further generate authenticity commitments for the quality scores.

[0051] Exemplarily, the process of calculating the quality score can be: The target data provider converts the benchmark model and the local model into model vectors, calculates the cosine distance between the model vector of the benchmark model and the model vector of the local model, and uses the cosine distance as the quality score of the local model.

[0052] Still taking the target data provider as an example below: The target data provider after receiving the benchmark model provided by the target data user can calculate the quality score d of the locally trained local model through the following formula locally.

[0053]

[0054] Among them, is the model vector of the reference model, is the model vector of the local model.

[0055] Since , , so the above formula can be further expanded and converted into the following formula:

[0056] Through the above formula, the quality score obtained by the reference model for evaluating the local model can be calculated.

[0057] After calculating the quality score, the target data provider also needs to calculate the authenticity commitment for this quality score.

[0058] Exemplarily, the calculation process of this authenticity commitment is as follows: The target data provider generates an authenticity commitment for the said quality score through the formula ; Among them, is the value of the authenticity commitment, is the model vector of the local model of the target data party, is the second basis, is the model vector of the reference model, is the first basis, represents the quality score of the local model of the target data party, represents a random number selected by the target data provider.

[0059] Still taking the target data provider as an example: Substituting the serial number i of the target data provider into the above formula for authenticity commitment, it can be:

[0060] Among them, is the local model of the i-th target data party 's model vector.

[0061] Step 241, each of the target data providers sends the quality score and the authenticity commitment generated by itself to the target data user.

[0062] The target data provider sends the quality score d and the authenticity commitment C to the target data user together.

[0063] In step 250, after the target data user receives the quality scores and authenticity commitments sent by each target data provider, it verifies whether the quality scores are authentic based on the authenticity commitments; and further determines the target quality scores with high scores from the verified quality scores, and obtains the target local model from the target data provider that sent the target quality scores.

[0064] After the target data user receives the authenticity commitment sent by the target data provider, it can use the aforementioned first basis, denoted as to supplement the authenticity commitment, and this process can be expressed as .

[0065] The verification process of the authenticity commitment will be introduced in detail below.

[0066] In an exemplary embodiment, verifying whether the quality score is authentic based on the authenticity commitment may include: Adopting an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment.

[0067] In this embodiment, the interactive verification method requires the cooperation of both the target data user and the target data provider for multiple rounds of interactive verification.

[0068] Below, to simplify the notation, let the reference model be , and the local model be .

[0069] The target data provider obtains the reference model , the local model , the first basis and the second basis ; uses the following steps for multiple rounds of interactive verification until the length of is equal to 1, and then executes S5: S1, the target data provider splits into containing the first half of the model parameters and containing the second half of the model parameters, splits into containing the first half of the model parameters and containing the second half of the model parameters, splits into containing the first half of the element points and into containing the first half of the element points and containing the second half of the element points; S2, the target data provider calculates , , , , and further send and to the target data user; S3. The target data generator randomly generates a challenge value x, and calculates , , , , ; and further send x, , , , and to the target data provider; where is the inverse element of the challenge value x; S4. The target data provider proves whether holds; if it holds, then determine , , and as the new , , and , and repeat S1; if it does not hold, determine that the verification fails;

[0070] S5. When the length of is equal to 1, the target data provider sends the with a length of 1 to the target data user; S6. The target data user calculates , , with a length of 1 calculated in S3, and calculates . If , determine that the verification passes, otherwise the verification fails.

[0071] Next, please refer to the schematic diagram of the interactive verification process shown in Figure 4 . Through each round of interaction, the benchmark model , the local model , the first basis and the second basis are shortened to half of the previous ones, and when the length of is 1, v1 can be directly sent to the target data user. Since the challenge value x sent by the target data user in each round of interaction is random, when the quality score d is not calculated according to the multi-party security protocol, it cannot pass the verification in the last round.

[0072] Further, for the quality scores that pass the verification, the target data user determines the high-score target quality scores from the verified quality scores, and obtains the target local model from the target data provider that sends the target quality scores.

[0073] In an exemplary embodiment, the target data user may perform joint sampling on all the quality scores that pass the verification and calculate the probability value ; where D is all the quality scores that pass the verification, and t is the sampling temperature for adjusting the output distribution; Determine the quality scores exceeding the probability value as the high-score target quality scores.

[0074] In this embodiment, the target data user collects all the quality scores that pass the verification and then selects a group of target local models with the highest scores through joint sampling. First, use the sampling temperature t to adjust the output distribution of the target local model. When t < 1.0, it can increase the robustness of sampling, and when t > 1.0, it can increase the randomness of sampling. Select the target local models with a probability exceeding P as the finally aggregated models. Alternatively, randomly select a preset number K of models from the target local models with a probability exceeding P as the finally aggregated models.

[0075] Step 260, the target data user verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

[0076] Before aggregating all the target local models, the target data user also needs to verify whether these target local models are complete. At this time, it is necessary to use the integrity commitment received by the target data user in the previous step 230. And as introduced in the previous step 220, the target data provider and the target data user can jointly execute a multi-party security protocol: The target data provider can obtain the integrity commitment generated by this multi-party security protocol for the local model while the target data user can obtain the session key generated by this multi-party security protocol for the negotiation result of this time (i.e., the integrity commitment ). ). .

[0077] The integrity commitment and the session key generated by this multi-party security protocol satisfy the following linear relationship:

[0078] Where, It is the global key held by the target data user (known only to the target data user).

[0079] This linear relationship is the basis for the target data user to verify the integrity of the target local model based on the integrity commitment of the target local model at this time. Specifically: The target data user uses the formula to verify whether the target local model is complete; where is the round of federated learning in which this training is located, i is the serial number of the target data user, is the local model obtained from the target data provider, is the integrity commitment sent by the target data provider, is the session key generated by the multi-party security protocol, is the global key held by the target data user.

[0080] If it holds (i.e., satisfies ), it can be determined that the target local model is complete; If it does not hold (does not satisfy ), it can be determined that the target local model is incomplete.

[0081] Furthermore, the target data user aggregates the verified target local models into a new global model trained in this round (such as the aforementioned round t). Since federated learning usually requires multiple rounds of learning, when a new global model is obtained at the end of this round of training, this new global model can be used as the global model for the next round of training, and the above embodiments are repeatedly executed to train the model of the (t + 1)-th round of federated learning.

[0082] After introducing the method embodiments of the above data processing system, the method embodiments of the target data user and the target data provider are introduced respectively with the target data user and the target data provider as the execution entities.

[0083] Figure 5 The data processing method shown is applied to the target data user in a data processing system based on federated learning. The target data user and multiple target data providers in the processing system form a federated learning cluster. The method includes: Step 510, the target data user publishes the global model to be trained to the target data providers in the cluster; Step 520, receive the integrity commitments for the local models returned by each target data provider; where the local model is the local model obtained by the target data provider through model training on the global model based on local data; Step 530: Disclose the benchmark model to each of the target data providers, where the benchmark model is used to evaluate the local models sent by each of the target data providers. Step 540: Receive the quality scores obtained by evaluating the layout model based on the benchmark model sent by each of the target data providers, as well as the authenticity commitments for the quality scores. Step 550: Verify whether the quality scores are authentic based on the authenticity commitments, further determine the target quality scores with high scores from the verified quality scores, and obtain the target local models from the target data providers that sent the target quality scores. Step 560: Verify whether the target local models are complete based on the integrity commitments of the target local models, and further aggregate the verified target local models into a new global model.

[0084] This embodiment is Figure 2 a method embodiment with the target data user in the foregoing embodiment as the executing entity. The above steps 510 to 560 can refer to the foregoing embodiment and will not be elaborated here.

[0085] Figure 6 The data processing method shown is applied to a target data provider in a data processing system based on federated learning. The target data provider and other data providers and target data users in the processing system form a federated learning cluster. The method includes: Step 610: The target data provider obtains the global model to be trained published by the target data user. Step 620: Perform model training on the global model based on local data to obtain the trained local model, and further generate an integrity commitment for the local model, and send the integrity commitment to the target data user. Step 630: Evaluate the quality score of the local model based on the benchmark model publicly disclosed by the target data user, and further generate an authenticity commitment for the quality score, and send the quality score and the authenticity commitment to the target data user. Step 640: Collaborate with the target data user to verify whether the quality score is authentic based on the authenticity commitment in an interactive verification manner.

[0086] This embodiment is Figure 2 a method embodiment with the target data provider in the foregoing embodiment as the executing entity. The above steps 610 to 640 can refer to the foregoing embodiment and will not be elaborated here.

[0087] Corresponding to the foregoing embodiments of the data processing method, this specification also provides embodiments of a data processing apparatus. The apparatus embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful apparatus, it is formed by the processor of the device where it is located reading the corresponding computer program in the non-volatile memory into the memory for operation. At the hardware level, as Figure 7 shown, it is a hardware structure diagram of the device where the data processing apparatus in this specification is located. In addition to Figure 7 the processor, network interface, memory, and non-volatile memory shown, the device where the apparatus is located in the embodiment usually includes other hardware according to the actual communication function, which will not be elaborated here.

[0088] Please refer to Figure 8 , which is a module diagram of the data processing apparatus provided by an embodiment of this specification. The apparatus corresponds to Figure 5 the embodiment shown, and is applied to the target data user in the data processing system based on federated learning. The target data user and multiple target data providers in the processing system form a federated learning cluster; the apparatus includes: A first publishing unit 810, where the target data user publishes the global model to be trained to the target data providers in the cluster; A first receiving unit 820, which receives the integrity commitments for the local models returned by each target data provider; wherein, the local model is the local model obtained by the target data provider training the global model based on local data; A second publishing unit 830, which discloses the benchmark model to each target data provider; wherein, the benchmark model is used to evaluate the local models sent by each target data provider; A second receiving unit 840, which receives the quality scores obtained by evaluating the layout model based on the benchmark model sent by each target data provider, and the authenticity commitments for the quality scores; An authenticity verification unit 850, which verifies whether the quality scores are authentic based on the authenticity commitments, further determines the high-score target quality scores from the quality scores that pass the verification, and obtains the target local models from the target data providers that send the target quality scores; An integrity verification unit 860, which verifies whether the target local models are complete based on the integrity commitments of the target local models, and further aggregates the target local models that pass the verification into a new global model.

[0089] Please refer to Figure 9 , which is a module diagram of the data processing apparatus provided by an embodiment of this specification. The apparatus corresponds to Figure 6The illustrated embodiment is applied to a target data provider in a data processing system based on federated learning. The target data provider, together with other data providers and the target data user in the processing system, forms a federated learning cluster. The apparatus includes: A model acquisition unit 910, where the target data provider acquires the global model to be trained published by the target data user. A model training unit 920, which trains the global model based on local data to obtain a trained local model, and further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user. A model evaluation unit 930, which evaluates the quality score of the local model based on the benchmark model publicly disclosed by the target data user, and further generates a authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user. An authenticity verification unit 940, which collaborates with the target data user and adopts an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment.

[0090] The system, apparatus, module or unit illustrated in the above embodiment can be specifically implemented by a computer chip or entity, or by a product with a certain function. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0091] For the implementation process of the functions and roles of each unit in the above apparatus, refer to the implementation process of the corresponding steps in the above method for details, which will not be repeated here.

[0092] For the apparatus embodiment, since it basically corresponds to the method embodiment, refer to the partial description of the method embodiment for the relevant parts. The apparatus embodiment described above is only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Those of ordinary skill in the art can understand and implement it without creative work.

[0093] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments.

[0094] Those skilled in the art will readily conceive of other embodiments of the present specification after considering the specification and practicing the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations of the present specification, which follow the general principles of the present specification and include known common general knowledge or conventional technical means in the technical field not disclosed in the present specification. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present specification are pointed out by the following claims.

[0095] It should be understood that the present specification is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present specification is only limited by the appended claims.

Claims

1. A data processing method, characterized in that, Applied to a data processing system based on federated learning, the data processing system includes a number of data providers and a number of data users; the method includes: The target data user combines a cluster for federated learning with multiple target data providers, and publishes a global model to be trained to the target data providers in the cluster; Each target data provider trains the global model based on local data to obtain a trained local model, and further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user; After receiving the integrity commitments sent by the respective target data providers, the target data user discloses a benchmark model to the respective target data providers; wherein, the benchmark model is used to evaluate the local models trained by the respective target data; Each target data provider evaluates the quality score of the trained local model based on the benchmark model, and further generates a authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user; After receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments; and further determines a target quality score with a high score from the verified quality scores, and obtains a target local model from the target data provider that sent the target quality score; The target data user verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

2. The method according to claim 1, wherein The generation of the integrity commitment for the local model includes: Calculating the integrity commitment of the local model based on a preset multi-party security protocol; wherein, the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider.

3. The method according to claim 2, wherein The target data user verifying whether the target local model is complete based on the integrity commitment of the target local model includes: The target data user calculates whether it holds; if it holds, determine that the target local model is complete; Among them, is the round of federated learning in which this training is carried out, and i is the serial number of the target data user. is the local model obtained from the target data provider. is the integrity commitment sent by the target data provider. is the session key generated by the multi-party security protocol. is the global key held by the target data user.

4. The method according to claim 1, characterized in that, The target data user discloses the benchmark model to the respective target data providers, and obtains it through the following method: Determining whether the current round of model training is the first round of federated learning; If so, the target data user trains a model based on local data to obtain a benchmark model; if not, the target data user obtains the local model from the data provider with the highest quality score that passed the verification in the previous round, and uses this local model as the benchmark model for the current round of model training.

5. The method according to claim 1, wherein The target data provider evaluating the quality score of the local model based on the benchmark model includes: The target data provider calculates the cosine distance of the model parameters between the benchmark model and the local model, and uses the cosine distance as the quality score of the local model.

6. The method according to claim 1, characterized in that The generation of the authenticity commitment for the quality score includes: The target data provider generates a authenticity commitment for the quality score through the formula ; Among them, is the value of the authenticity commitment, is the model vector of the local model of the target data party, is the second basis, is the model vector of the reference model, is the first basis, represents the quality score of the local model of the target data party, represents the random number selected by the target data provider.

7. The method according to claim 1, characterized in that After receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments, including: After receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user adopts an interactive verification method to verify whether the quality scores are authentic based on the authenticity commitments.

8. The method according to claim 7, wherein The adopting of the interactive verification method to verify whether the quality scores are authentic based on the authenticity commitments includes: The target data provider obtains a benchmark model , a local model , a first basis and a second basis ; perform multiple rounds of cross-validation using the following steps until the length of is equal to 1, then execute S5: S1, the target data provider will be split into containing the first half of the model parameters and , and will be split into containing the first half of the model parameters and , and will be split into containing the first half of the element points and containing the second half of the element points, and the second basis will be split into containing the first half of the element points and containing the second half of the element points; S2, the target data provider calculates , , , , and further sends and to the target data user; S3. The target data generator randomly generates a challenge value x and calculates , , , , ; and further sends x, , , , and to the target data provider; where is the inverse element of the challenge value x; S4, the target data provider proves whether it holds; if it holds, then , , and are determined as new , , and , and repeat S1; S5, when has a length equal to 1, the target data provider sends with a length of 1 to the target data user; S6, the target data user calculates, according to S3, those with a length of 1 , , , and calculates . If , it is determined that the verification passes; otherwise, the verification fails.

9. The method according to claim 6 or 8, characterized in that, The first substrate and the second substrate Including from the definition in Elliptic curve group on Two independent sets of element points are selected from; wherein the number of element points in the element point set is the same as the number of model parameters in the global model.

10. The method according to claim 1, characterized in that, The determining of the high-score target quality scores from the verified quality scores includes: The target data user performs joint sampling on all the verified quality scores to calculate the probability value ; where D is all the verified quality scores, and t is the sampling temperature for adjusting the output distribution Determine the quality scores exceeding the probability value as the high-score target quality scores.

11. A data processing method, characterized in that, Applied to the target data user in the data processing system based on federated learning, the target data user and multiple target data providers in the processing system form a federated learning cluster; the method includes: The target data user publishes the global model to be trained to the target data providers in the cluster; Receive the integrity commitments for the local models returned by the respective target data providers; wherein, the local model is the local model obtained by the target data provider training the global model based on local data. Disclose the benchmark model to the respective target data providers; wherein, the benchmark model is used to evaluate the local models sent by the respective target data providers. Receive the quality scores obtained by evaluating the layout models based on the benchmark model sent by the respective target data providers, and the authenticity commitments for the quality scores; Verify whether the quality scores are authentic based on the authenticity commitments, further determine the high-score target quality scores from the verified quality scores, and obtain the target local models from the target data providers that sent the target quality scores; Verify whether the target local models are complete based on the integrity commitments of the target local models, and further aggregate the verified target local models into a new global model.

12. A data processing method, characterized in that, Applied to the target data provider in the data processing system based on federated learning, the target data provider and other data providers and target data users in the processing system form a federated learning cluster; the method includes: The target data provider obtains the global model to be trained published by the target data user; Train the global model based on local data to obtain the trained local model, and further generate an integrity commitment for the local model, and send the integrity commitment to the target data user; Evaluate the quality score of the local model based on the benchmark model disclosed by the target data user, and further generate an authenticity commitment for the quality score, and send the quality score and the authenticity commitment to the target data user; Cooperate with the target data user to adopt an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment.

13. A data processing device, characterized in that, Applied to the target data user in the data processing system based on federated learning, the target data user and multiple target data providers in the processing system form a federated learning cluster; the device includes: The first publishing unit, where the target data user publishes the global model to be trained to the target data providers in the cluster; A first receiving unit that receives integrity commitments for the local models returned by each target data provider; wherein the local model is a local model obtained by the target data provider through model training on the global model based on local data. A second publishing unit that discloses a benchmark model to each of the target data providers; wherein the benchmark model is used to evaluate the local models sent by each of the target data providers. A second receiving unit that receives the quality scores obtained by evaluating the layout model based on the benchmark model and the authenticity commitments for the quality scores sent by each of the target data providers. An authenticity verification unit that verifies whether the quality scores are authentic based on the authenticity commitments, further determines the target quality scores with high scores from the quality scores that pass the verification, and obtains the target local models from the target data providers that send the target quality scores. An integrity verification unit that verifies whether the target local models are complete based on the integrity commitments of the target local models, and further aggregates the target local models that pass the verification into a new global model.

14. A data processing device, characterized in that, Applied to a target data provider in a data processing system based on federated learning, the target data provider and other data providers and target data users in the processing system form a federated learning cluster; the device includes: A model acquisition unit, where the target data provider acquires the global model to be trained published by the target data user. A model training unit that performs model training on the global model based on local data to obtain a trained local model, and further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user. A model evaluation unit that evaluates the quality score of the local model based on the benchmark model disclosed by the target data user, and further generates an authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user. An authenticity verification unit that cooperates with the target data user to verify whether the quality score is authentic based on the authenticity commitment in an interactive verification manner.

15. An electronic device, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein the processor is configured to execute the method described in any one of the above claims 1-12.

Images