Data processing method and device

Abstract

This specification provides a data processing method and apparatus. When applied to a target data provider in a data processing system based on federated learning, the target data provider, other data providers, and target data users in the processing system form a federated learning cluster; and the method includes: the target data provider obtains a global model to be trained published by the target data user; performs model training on the global model based on local data to obtain a trained local model, further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user; evaluates the quality score of the local model based on a benchmark model disclosed by the target data user, further generates an authenticity commitment for the quality score, and sends the quality score and authenticity commitment to the target data user; and collaborates with the target data user to verify the authenticity of the quality score based on the authenticity commitment using an interactive verification method.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly, to a data processing method and apparatus. Background Art

[0002] Artificial Intelligence (AI) technology has permeated every aspect of daily life. During AI model training, data users (DAs) can access data from data providers (DPs) that meets their needs, allowing them to train high-quality AI models using this data. To facilitate data circulation and mitigate the privacy risks associated with data transactions, models can be trained using technologies such as federated learning. Federated learning allows model training to be completed by sharing model parameter gradients without exchanging the original data, achieving the goal of "raw data remaining in the domain and data being available but not visible."

[0003] However, although the federated learning method can protect data privacy during the model training phase, there is still a risk of privacy leakage during the model evaluation phase after model training is completed. Summary of the Invention

[0004] The embodiments of this specification provide a data processing method and device.

[0005] According to a first aspect of an embodiment of this specification, a data processing method is provided, which is applied to a data processing system based on federated learning, wherein the data processing system includes several data providers and several data users; the method includes:

[0006] The target data user and multiple target data providers form a federated learning cluster, and publish the global model to be trained to the target data providers in the cluster;

[0007] Each target data provider performs model training on the global model based on local data to obtain a trained local model, and further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user;

[0008] After receiving the integrity commitments sent by the respective target data providers, the target data user discloses the benchmark model to the respective target data providers; wherein the benchmark model is used to evaluate the local models trained with the respective target data;

[0009] Each target data provider evaluates a quality score of the trained local model based on the benchmark model, further generates an authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user;

[0010] After receiving the quality scores and authenticity commitments sent by the target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments; further determines a high target quality score from the verified quality scores, and obtains a target local model from the target data provider that sent the target quality score;

[0011] The target data user verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

[0012] Optionally, generating an integrity commitment for the local model includes:

[0013] The integrity commitment of the local model is calculated based on a preset multi-party security protocol; wherein the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider.

[0014] Optionally, the target data user verifies whether the target local model is complete based on the integrity commitment of the target local model, including:

[0015] The target data is calculated by the user If so, determining whether the target local model is complete;

[0016] in, is the round of federated learning in which this training is conducted, i is the serial number of the target data user, The local model obtained from the target data provider, The integrity commitment sent by the target data provider, a session key generated for the multi-party security protocol, The global key held by the target data consumer.

[0017] Optionally, the target data user discloses the benchmark model to each target data provider, and obtains the benchmark model by:

[0018] Determine whether this round of model training is the first round of federated learning;

[0019] If yes, the target data user performs model training based on the local data to obtain a benchmark model; if no, the target data user obtains a local model from the data provider that passed verification in the previous round and has the highest quality score, and uses the local model as the benchmark model for this round of model training.

[0020] Optionally, the target data provider evaluates the quality score of the local model based on the benchmark model, including:

[0021] The target data provider calculates a cosine distance of model parameters between the reference model and the local model, and uses the cosine distance as a quality score of the local model.

[0022] Optionally, generating an authenticity commitment for the quality score includes:

[0023] The target data provider uses the formula generating a commitment of authenticity for the quality score;

[0024] in, is the value of the authenticity commitment, is the model vector of the local model of the target data provider, is the second substrate, is the model vector of the baseline model, is the first substrate, represents the quality score of the local model of the target data provider, Indicates a random number chosen by the target data provider.

[0025] Optionally, after receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments, including:

[0026] After receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user uses an interactive verification method to verify whether the quality scores are authentic based on the authenticity commitments.

[0027] Optionally, the interactive verification method is used to verify whether the quality score is authentic based on the authenticity commitment, including:

[0028] The target data provider obtains the benchmark model , local model , first substrate and the second substrate ; Use the following steps to perform multiple rounds of interactive verification until When the length of is equal to 1, execute S5:

[0029] S1, the target data provider will Split into a containing the first half of the model parameters and contains the second half of the model parameters ,Will Split into a containing the first half of the model parameters and contains the second half of the model parameters ,Will Split into the first half of the element points and the points containing the second half of the elements , the second substrate Split into the first half of the element points and the points containing the second half of the elements ;

[0030] S2, the target data provider calculates 、 、 、 , and further and Sending to the target data user;

[0031] S3, the target data user randomly generates a challenge value x and calculates 、 、 、 、 ; and further x, 、 、 、 and Sent to the target data provider; wherein, is the inverse of the challenge value x;

[0032] S4, the target data provider proves Is it established? If so, 、 、 and Determined as new 、 、 and , repeat S1;

[0033] S5, in When the length of is equal to 1, the target data provider will Sending to the target data user;

[0034] S6, the target data user calculates the length of 1 according to S3 、 、 ,calculate ,if , confirm that the verification is successful, otherwise the verification fails.

[0035] Optionally, the first substrate and the second substrate Including from the definition in Elliptic curve group on Two independent sets of element points are selected from ; wherein the number of element points in the element point set is the same as the number of model parameters in the global model.

[0036] Optionally, determining a high target quality score from the verified quality scores includes:

[0037] The target data user performs joint sampling on all verified quality scores and calculates the probability value ; Where D is the quality score of all verified values, and t is the sampling temperature for adjusting the output distribution;

[0038] A quality score exceeding the probability value is determined as a high target quality score.

[0039] According to a second aspect of an embodiment of this specification, a data processing method is provided, which is applied to a target data user in a data processing system based on federated learning, wherein the target data user and multiple target data providers in the processing system form a federated learning cluster; the method comprises:

[0040] The target data user publishes the global model to be trained to the target data provider in the cluster;

[0041] Receive integrity commitments for local models returned by each target data provider; wherein the local model is a trained local model obtained by the target data provider training the global model based on local data;

[0042] Disclosing a benchmark model to each target data provider; wherein the benchmark model is used to evaluate the local models sent by each target data provider;

[0043] receiving, from each target data provider, a quality score obtained by evaluating the layout model based on the benchmark model, and a commitment to authenticity of the quality score;

[0044] Verifying whether the quality score is authentic based on the authenticity commitment, further determining a high target quality score from the verified quality scores, and obtaining a target local model from a target data provider that sent the target quality score;

[0045] Whether the target local model is complete is verified based on the integrity commitment of the target local model, and the verified target local models are further aggregated into a new global model.

[0046] According to a third aspect of an embodiment of this specification, a data processing method is provided, which is applied to a target data provider in a data processing system based on federated learning, wherein the target data provider, other data providers, and target data users in the processing system form a federated learning cluster; the method comprises:

[0047] The target data provider obtains the global model to be trained published by the target data user;

[0048] Performing model training on the global model based on local data to obtain a trained local model, further generating an integrity commitment for the local model, and sending the integrity commitment to the target data user;

[0049] Evaluating a quality score of the local model based on a benchmark model disclosed by the target data user, further generating an authenticity commitment for the quality score, and sending the quality score and the authenticity commitment to the target data user;

[0050] Cooperate with the target data user to adopt an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment.

[0051] According to a fourth aspect of an embodiment of this specification, a data processing apparatus is provided, which is applied to a target data user in a data processing system based on federated learning, wherein the target data user and multiple target data providers in the processing system form a federated learning cluster; the apparatus includes:

[0052] A first publishing unit, a target data user publishing a global model to be trained to a target data provider in the cluster;

[0053] A first receiving unit receives integrity commitments for local models returned by each target data provider; wherein the local model is a trained local model obtained by the target data provider training the global model based on local data;

[0054] A second publishing unit, which discloses a benchmark model to each target data provider; wherein the benchmark model is used to evaluate the local models sent by each target data provider;

[0055] a second receiving unit, receiving, from each target data provider, a quality score obtained by evaluating the layout model based on the benchmark model, and an authenticity commitment for the quality score;

[0056] an authenticity verification unit, which verifies whether the quality score is authentic based on the authenticity commitment, further determines a high target quality score from the verified quality scores, and obtains a target local model from a target data provider that sends the target quality score;

[0057] An integrity verification unit verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

[0058] According to a fifth aspect of an embodiment of this specification, a data processing apparatus is provided, which is applied to a target data provider in a data processing system based on federated learning, wherein the target data provider, other data providers, and target data users in the processing system form a federated learning cluster; the apparatus comprises:

[0059] A model acquisition unit, where the target data provider acquires the global model to be trained published by the target data user;

[0060] A model training unit performs model training on the global model based on local data to obtain a trained local model, further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user;

[0061] a model evaluation unit that evaluates a quality score of the local model based on a benchmark model disclosed by the target data user, further generates an authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user;

[0062] The authenticity verification unit cooperates with the target data user to adopt an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment.

[0063] According to a sixth aspect of the embodiments of this specification, there is provided a computing device, including:

[0064] processor;

[0065] a memory for storing instructions executable by the processor;

[0066] The processor is configured to execute the executable instructions to implement the data processing method as described in any of the preceding items.

[0067] The embodiments of this specification provide a data processing solution. Zero-knowledge proof is used to ensure that the data provider can participate in the evaluation of the local model during the model evaluation phase without disclosing the details of the model, thereby avoiding the risk of privacy leakage and ensuring the safety and fairness of the data provider and the data user. Specifically, after the data provider trains the local model based on federated learning, it first calculates the integrity commitment for the local model to ensure that the local model will not be tampered with during the evaluation process; then the local model is evaluated through the benchmark model provided by the data user to obtain the quality score of the local model; and the data user verifies the quality score through interactive zero-knowledge proof verification to ensure the authenticity of the evaluation result, and selects high-quality local models for fusion based on the quality score. Before fusion, the integrity commitment of the local model is verified to ensure the integrity of the local model. In this way, through the dual commitment of integrity commitment and authenticity commitment, the reliability and consistency of the local model are ensured, and privacy protection is achieved during the model evaluation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 An architectural diagram of a data processing system provided in accordance with an embodiment of this specification;

[0069] Figure 2 A flowchart of a data processing method provided in one embodiment of this specification;

[0070] Figure 3 A schematic diagram of a zero-knowledge protocol provided in one embodiment of this specification;

[0071] Figure 4 A schematic diagram of the process of interactive verification provided in one embodiment of this specification;

[0072] Figure 5 A flowchart of a target data user provided in one embodiment of this specification;

[0073] Figure 6 A flowchart of a target data provider provided in one embodiment of this specification;

[0074] Figure 7 A hardware structure diagram of a data processing device provided in one embodiment of this specification;

[0075] Figure 8 A module of a data processing device provided in an embodiment of this specification;

[0076] Figure 9 A module of a data processing device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0077] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0078] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. As used in this specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0079] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information without departing from the scope of this specification. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0080] While existing data processing solutions for federated learning, such as those described above, can protect data privacy during the model training phase, the risk of privacy leakage persists during the model evaluation phase after model training. Therefore, a data processing solution that can ensure privacy protection during model evaluation is needed.

[0081] Existing data trading methods for federated learning fail to effectively protect the privacy of data providers while assessing their model quality. Zero-knowledge proof technology is a cryptographic technique that allows a prover to convince a verifier of the correctness of an assertion without revealing any detailed information about the assertion. In recent years, with the development of algorithms such as zero-knowledge succinct non-interactive arguments of knowledge (zk-SNARK), some research has proposed methods based on verifiable computation to achieve rights protection in machine learning systems. However, limited research currently utilizes this technology for model quality assessment.

[0082] In view of this, the present invention proposes a new data processing solution, which ensures that data providers can participate in the evaluation of local models during the model evaluation phase through zero-knowledge proof without disclosing model details, thereby avoiding the risk of privacy leakage and ensuring the security and fairness of data providers and data users.

[0083] Please refer to the following Figure 1 The system architecture diagram of a data processing system based on federated learning is shown. The system can include several data providers and several data users. The data users can publish the global model to be trained to the data providers participating in federated learning, while the data providers locally store local data for the federated learning training model.

[0084] A data user and multiple data providers can form a federated learning cluster. In order to distinguish data providers and data users from those outside the cluster, Figure 1 As shown in , the data users in the cluster can be called target data users, and the data providers in the cluster can be called target data providers. Generally, there can be multiple different target data providers in the cluster, such as Figure 1 Target data providers 1 to 6 are shown.

[0085] During federated learning, the target data provider can use local data to train the global model published by the target data consumer. After obtaining the trained local model, the provider first calculates the integrity commitment for the local model to ensure that the local model is not tampered with during the evaluation process. The local model is then evaluated using the benchmark model provided by the data consumer to obtain a quality score for the local model.

[0086] Data users verify the quality scores through interactive zero-knowledge proof to ensure the authenticity of the evaluation results, and select high-quality local models for fusion based on the quality scores. Before fusion, they ensure the integrity of the local models by verifying their integrity commitments.

[0087] Through the above scheme, in the data processing system based on federated learning, through the dual commitment of integrity commitment and authenticity commitment, while effectively evaluating the quality of the local model trained by the data provider, it ensures the reliability and consistency of the local model and protects the privacy security of the local model.

[0088] Please refer to the following Figure 2 The data processing method shown in the figure can be applied to the aforementioned federated learning-based data processing system. The method may include the following steps:

[0089] In step 210 , the target data user and multiple target data providers form a federated learning cluster, and publish the global model to be trained to the target data providers in the cluster.

[0090] Assume there are n target data providers and the number of model parameters of the global model published by the target data consumer is m. Considering that different global models have different numbers of model parameters in practical applications, a standard number can be defined. When the number of model parameters is less than the standard number, the shortfall can be padded with zeros.

[0091] For example, the standard quantity can be set to , q can be any integer, so when m is less than When , you can add 0 after the existing model parameters until the number of model parameters reaches indivual.

[0092] The target data user and the target data provider can construct a multi-party security protocol, such as a zero-knowledge proof protocol, through prior negotiation.

[0093] Exemplarily, the above negotiation can be accomplished by using a VOLE (Vector Oblivious Linear Evaluation) generator, which is often used to construct secure multi-party computing protocols due to its low communication complexity.

[0094] In addition, the target data user and the target data provider can also determine two sets of bases required for the multi-party security protocol according to the VOLE generator. The two sets of bases can be, for example, from the Elliptic curve group on The number of element points in the set is the same as the number of model parameters in the global model (m). The basis of these two groups can be denoted as: and the second substrate

[0095] In step 220 , each target data provider performs model training on the global model based on local data to obtain a trained local model, and further generates an integrity commitment for the local model.

[0096] In this embodiment, each target data provider can calculate the integrity commitment of its own trained local model based on the multi-party security protocol determined by the aforementioned negotiation.

[0097] Assuming that this round of training is in the tth round of federated learning, the global model released by the target data user can be recorded as The following will take the i-th target data provider as This section uses the following as an example to describe the steps that each target data provider needs to perform (i.e., the execution process for each target data provider is the same).

[0098] The target data provider obtains the global model Then, use local data In this global model Model training is performed based on , thereby obtaining the trained local model ; Then as Figure 3 As shown in the figure, the target data provider and the target data user can collaboratively execute the multi-party security protocol through the VOLE generator, so that the target data provider can obtain the local model generated by the multi-party security protocol. Commitment to integrity , and the target data user can obtain the negotiation result (i.e. integrity commitment) generated by the multi-party security protocol ) .

[0099] The integrity commitment generated by this multi-party security protocol and session key The following linear relationship is satisfied:

[0100]

[0101] in, This is the global key held by the target data consumer (known only to the target data consumer). This linear relationship is also the basis for the target data consumer to verify the integrity commitment. This part of the verification process will be described in the subsequent steps.

[0102] In step 221, each target data provider sends the integrity commitment generated by the provider to the target data user.

[0103] Target data provider Obtaining a commitment to integrity After that, the integrity commitment can be Sent to the target data consumer.

[0104] In step 230, after receiving the integrity commitments sent by the target data providers, the target data user discloses the benchmark model to the target data providers; wherein the benchmark model is used to evaluate the local models trained by the target data.

[0105] When the target data consumer receives the integrity commitment sent by all target data providers After that, the baseline model of this round of training can be made public .

[0106] The benchmark model can be obtained in the following ways:

[0107] If the target data uses the local data If the data quality is high, or this round of model training is the first round of federated learning, the target data user can Perform a round of model training to obtain a baseline model Otherwise, the target data user needs to pass the verification in the previous round and the data provider with the highest model quality score Get the data provider The local model is used as the baseline model for this round of model training. .

[0108] It should be noted that when it comes to data transaction scenarios, the target data user needs to provide Buy to get the baseline model

[0109] In step 240 , each target data provider evaluates the quality score of the trained local model based on the benchmark model, and further generates an authenticity commitment for the quality score.

[0110] For example, the process of calculating the quality score may be:

[0111] The target data provider converts the baseline model and the local model into model vectors, calculates the cosine distance between the model vector of the baseline model and the model vector of the local model, and uses the cosine distance as the quality score of the local model.

[0112] The following still uses the target data provider For example:

[0113] Target data provider Receive the benchmark model provided by the target data user After that, the trained local model can be calculated locally using the following formula The quality score of d.

[0114]

[0115] in, is the model vector of the baseline model, is the model vector of the local model.

[0116] because , , so the above formula can be further expanded to the following formula:

[0117]

[0118] The above formula can be used to calculate the quality score of the local model evaluated by the baseline model.

[0119] After calculating the quality score, the target data provider also needs to calculate the authenticity commitment for the quality score.

[0120] For example, the calculation process of the authenticity commitment is as follows:

[0121] The target data provider uses the formula generating a commitment of authenticity for the quality score;

[0122] in, is the value of the authenticity commitment, is the model vector of the local model of the target data provider, is the second substrate, is the model vector of the baseline model, is the first substrate, represents the quality score of the local model of the target data provider, Indicates a random number chosen by the target data provider.

[0123] The following still uses the target data provider For example:

[0124] Substituting the target data provider's serial number i into the above authenticity commitment formula can be obtained as follows:

[0125]

[0126] in, is the local model of the i-th target data provider The model vector of .

[0127] In step 241 , each target data provider sends the quality score and the authenticity commitment generated by the provider to the target data user.

[0128] The target data provider sends the quality score d and the authenticity commitment C to the target data user.

[0129] In step 250, after receiving the quality scores and authenticity commitments sent by the target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments; and further determines a high-scoring target quality score from the verified quality scores, and obtains a target local model from the target data provider that sent the target quality score.

[0130] After the target data user receives the authenticity commitment sent by the target data provider, he can use the first basis mentioned above to record To complement the authenticity commitment, this process can be expressed as .

[0131] The verification process of the authenticity commitment is described in detail below.

[0132] In an exemplary embodiment, verifying whether the quality score is authentic based on the authenticity commitment may include:

[0133] An interactive verification method is adopted to verify whether the quality score is authentic based on the authenticity commitment.

[0134] In this embodiment, the interactive verification method requires the target data user and the target data provider to cooperate and perform multiple rounds of interactive verification.

[0135] To simplify the symbols below, let the benchmark model , local model .

[0136] The target data provider obtains the benchmark model , local model , first substrate and the second substrate ; Use the following steps to perform multiple rounds of interactive verification until When the length of is equal to 1, execute S5:

[0137] S1, the target data provider will Split into a containing the first half of the model parameters and contains the second half of the model parameters ,Will Split into a containing the first half of the model parameters and contains the second half of the model parameters ,Will Split into the first half of the element points and the points containing the second half of the elements , the second substrate Split into the first half of the elements and the points containing the second half of the elements ;

[0138] S2, the target data provider calculates 、 、 、 , and further and Sending to the target data user;

[0139] S3, the target data user randomly generates a challenge value x and calculates 、 、 、 、 ; and further x, 、 、 、 and Sent to the target data provider; wherein, is the inverse of the challenge value x;

[0140] S4, the target data provider proves Is it established? If so, 、 、 and Determined as new 、 、 and , repeat S1; if not, it is determined that the verification has failed.

[0141] S5, in When the length of is equal to 1, the target data provider will Sending to the target data user;

[0142] S6, the target data user calculates the length of 1 according to S3 、 、 ,calculate ,if , confirm that the verification is successful, otherwise the verification fails.

[0143] Please refer to the following Figure 4 The interactive verification process diagram shown in the figure is as follows: , local model , first substrate and the second substrate shortened to half of its previous value and When the length of is 1, v1 can be sent directly to the target data user. Since the challenge value x sent by the target data user in each round of interaction is random, when the quality score d is not calculated according to the multi-party security protocol, it will not pass the final round of verification.

[0144] Furthermore, for the verified quality scores, the target data user determines a high target quality score from the verified quality scores, and obtains a target local model from the target data provider that sent the target quality score.

[0145] In an exemplary embodiment, the target data user may perform joint sampling on all verified quality scores and calculate the probability value ; Where D is the quality score of all verified values, and t is the sampling temperature for adjusting the output distribution;

[0146] A quality score exceeding the probability value is determined as a high target quality score.

[0147] In this embodiment, the target data user collects all the quality scores that have passed the verification. Finally, a set of target local models with the highest scores is selected through joint sampling. The sampling temperature t is first used to adjust the output distribution of the target local models. When t < 1.0, this increases sampling robustness, while when t > 1.0, this increases sampling randomness. Target local models with a probability exceeding P are selected as the final aggregated model. Alternatively, a preset number K of models are randomly selected from the target local models with a probability exceeding P as the final aggregated model.

[0148] In step 260 , the target data user verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

[0149] Before aggregating all target partial models, the target data user needs to verify the integrity of these target partial models. This requires the integrity commitment received by the target data user in step 230. Furthermore, as described in step 220, the target data provider and target data user can collaborate to execute a multi-party security agreement:

[0150] The target data provider can obtain the local model generated by the multi-party security protocol Commitment to integrity , and the target data user can obtain the negotiation result (i.e. integrity commitment) generated by the multi-party security protocol ) .

[0151] The integrity commitment generated by this multi-party security protocol and session key The following linear relationship is satisfied:

[0152]

[0153] in, A global key held by the target data consumer (known only to the target data consumer).

[0154] This linear relationship is the basis for the target data user to verify whether the target local model is complete based on the integrity commitment of the target local model. Specifically:

[0155] The target data user uses the formula Verifying whether the target local model is complete;

[0156] in, is the round of federated learning in which this training is conducted, i is the serial number of the target data user, The local model obtained from the target data provider, The integrity commitment sent by the target data provider, a session key generated for the multi-party security protocol, The global key held by the target data consumer.

[0157] If it holds (i.e. satisfies ), it can be determined that the target local model is complete;

[0158] If not (does not satisfy ), it can be determined that the target local model is incomplete.

[0159] Furthermore, the target data user aggregates the verified target local models into a new global model trained in this round (such as the aforementioned round t); since federated learning usually requires multiple rounds of learning, when a new global model is obtained after the current round of training, the new global model can be used as the global model for the next round of training, and the above embodiment can be repeated to train the model training for the t+1 round of federated learning.

[0160] After introducing the method embodiments of the above-mentioned data processing system, the following introduces respective method embodiments with the target data user and the target data provider as the execution entities.

[0161] Figure 5 The data processing method shown is applied to a target data user in a data processing system based on federated learning. The target data user and multiple target data providers in the processing system form a federated learning cluster. The method includes:

[0162] Step 510: The target data user publishes the global model to be trained to the target data provider in the cluster;

[0163] Step 520: Receive integrity commitments for local models returned by each target data provider; wherein the local models are trained by the target data provider on the global model based on local data.

[0164] Step 530: Publicize the benchmark model to each target data provider; wherein the benchmark model is used to evaluate the local models sent by each target data provider;

[0165] Step 540 , receiving quality scores obtained by evaluating the layout model based on the benchmark model and authenticity commitments for the quality scores, sent by the respective target data providers;

[0166] Step 550: Verify whether the quality score is authentic based on the authenticity commitment, further determine a high target quality score from the verified quality scores, and obtain a target local model from a target data provider that sent the target quality score.

[0167] Step 560 : Verify whether the target local model is complete based on the integrity commitment of the target local model, and further aggregate the verified target local models into a new global model.

[0168] This embodiment is based on the aforementioned Figure 2 In the embodiment, the target data user is the method embodiment of the execution subject. The above steps 510 to 560 can refer to the previous embodiment and will not be repeated here.

[0169] Figure 6 The data processing method shown is applied to a target data provider in a data processing system based on federated learning. The target data provider, other data providers, and target data users in the processing system form a federated learning cluster. The method includes:

[0170] Step 610: The target data provider obtains the global model to be trained published by the target data user;

[0171] Step 620: Perform model training on the global model based on the local data to obtain a trained local model, further generate an integrity commitment for the local model, and send the integrity commitment to the target data user;

[0172] Step 630: Evaluate the quality score of the local model based on the benchmark model disclosed by the target data user, further generate an authenticity commitment for the quality score, and send the quality score and the authenticity commitment to the target data user;

[0173] Step 640: Cooperate with the target data user to adopt an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment.

[0174] This embodiment is based on the aforementioned Figure 2 In the embodiment of the method in which the target data provider is the execution subject, the above steps 610 to 640 can refer to the above embodiment and will not be described in detail here.

[0175] Corresponding to the aforementioned data processing method embodiment, this specification also provides an embodiment of a data processing device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the device in which it is located reading the corresponding computer program in the non-volatile memory into the memory and running it. From the hardware level, such as Figure 7 The figure shows a hardware structure diagram of the device where the data processing device in this manual is located, except Figure 7 In addition to the processor, network interface, memory and non-volatile memory shown, the device in which the apparatus in the embodiment is located may also include other hardware according to the actual communication function, which will not be described in detail.

[0176] See Figure 8 , is a block diagram of a data processing device provided in an embodiment of this specification, the device corresponds to Figure 5 The embodiment shown is applied to a target data user in a data processing system based on federated learning, wherein the target data user and multiple target data providers in the processing system form a federated learning cluster; the apparatus includes:

[0177] A first publishing unit 810, a target data user publishes a global model to be trained to a target data provider in the cluster;

[0178] A first receiving unit 820 receives integrity commitments for local models returned by each target data provider; wherein the local model is a trained local model obtained by the target data provider training the global model based on local data;

[0179] A second publishing unit 830 is configured to publish a benchmark model to each target data provider; wherein the benchmark model is used to evaluate the local models sent by each target data provider;

[0180] A second receiving unit 840 receives quality scores obtained by evaluating the layout model based on the benchmark model and authenticity commitments for the quality scores, sent by the respective target data providers;

[0181] an authenticity verification unit 850 that verifies whether the quality score is authentic based on the authenticity commitment, further determines a high target quality score from the verified quality scores, and obtains a target local model from a target data provider that sends the target quality score;

[0182] The integrity verification unit 860 verifies whether the target local model is complete based on the integrity commitment of the target local model, and further aggregates the verified target local models into a new global model.

[0183] See Figure 9 , is a block diagram of a data processing device provided in an embodiment of this specification, the device corresponds to Figure 6 The embodiment shown is applied to a target data provider in a data processing system based on federated learning, wherein the target data provider, other data providers, and target data users in the processing system form a federated learning cluster; the apparatus includes:

[0184] The model acquisition unit 910 is used for the target data provider to acquire the global model to be trained published by the target data user;

[0185] The model training unit 920 performs model training on the global model based on the local data to obtain a trained local model, further generates an integrity commitment for the local model, and sends the integrity commitment to the target data user;

[0186] A model evaluation unit 930 evaluates a quality score of the local model based on a benchmark model disclosed by the target data user, further generates an authenticity commitment for the quality score, and sends the quality score and the authenticity commitment to the target data user;

[0187] The authenticity verification unit 940 cooperates with the target data user to verify whether the quality score is authentic based on the authenticity commitment using an interactive verification method.

[0188] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0189] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0190] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0191] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0192] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this specification and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present invention being indicated by the following claims.

[0193] It should be understood that the present description is not limited to the exact structure that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: Applied to a data processing system based on federated learning, the data processing system includes several data providers and several data users; the method includes: The target data user and multiple target data providers form a federated learning cluster, and publish the global model to be trained to the target data providers in the cluster; Each target data provider trains the global model based on local data to obtain a trained local model, and further calculates the integrity commitment of the local model based on a preset multi-party security protocol, and sends the integrity commitment to the target data user; wherein the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider; After receiving the integrity commitments sent by the target data providers, the target data user discloses the benchmark model to the target data providers; wherein the benchmark model is used to evaluate the local models trained by the target data providers; Each target data provider evaluates the quality score of the trained local model based on the benchmark model, and further uses the formula Generate an authenticity commitment for the quality score, and send the quality score and the authenticity commitment to the target data user; wherein, is the value of the authenticity commitment, is the model vector of the local model of the target data provider, is the second substrate, is the model vector of the baseline model, is the first substrate, represents the quality score of the local model of the target data provider, Indicates the random number selected by the target data provider; After receiving the quality scores and authenticity commitments sent by the target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments; further determines a high target quality score from the verified quality scores, and obtains a target local model from the target data provider that sent the target quality score; The target data is calculated by the user Is it true? If so, determine that the target local model is complete, and further aggregate the complete target local model into a new global model; where t is the round of federated learning in which this training is located, i is the serial number of the target data user, The local model obtained from the target data provider, The integrity commitment sent by the target data provider, a session key generated for the multi-party security protocol, The global key held by the target data consumer.

2. The method according to claim 1, characterized in that The target data user discloses the benchmark model to each target data provider and obtains it in the following manner: Determine whether this round of model training is the first round of federated learning; If yes, the target data user performs model training based on the local data to obtain a benchmark model; if no, the target data user obtains a local model from the data provider that passed verification in the previous round and has the highest quality score, and uses the local model as the benchmark model for this round of model training.

3. The method according to claim 1, characterized in that The target data provider evaluates the quality score of the local model based on the benchmark model, including: The target data provider calculates a cosine distance of model parameters between the reference model and the local model, and uses the cosine distance as a quality score of the local model.

4. The method according to claim 1, wherein After receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user verifies whether the quality scores are authentic based on the authenticity commitments, including: After receiving the quality scores and authenticity commitments sent by the respective target data providers, the target data user uses an interactive verification method to verify whether the quality scores are authentic based on the authenticity commitments.

5. The method according to claim 4, characterized in that The interactive verification method is used to verify whether the quality score is authentic based on the authenticity commitment, including: The target data provider obtains the benchmark model , local model , first substrate and the second substrate ; Use the following steps to perform multiple rounds of interactive verification until When the length of is equal to 1, execute S5: S1, the target data provider will Split into a containing the first half of the model parameters and contains the second half of the model parameters ,Will Split into a containing the first half of the model parameters and contains the second half of the model parameters ,Will Split into the first half of the element points and the points containing the second half of the elements , the second substrate Split into the first half of the element points and the points containing the second half of the elements ; S2, the target data provider calculates 、 、 、 , and further and Sending to the target data user; S3, the target data user randomly generates a challenge value x and calculates 、 、 、 、 ; and further x, 、 、 、 and Sent to the target data provider; wherein, is the inverse of the challenge value x; S4, the target data provider proves Is it established? If so, 、 、 and Determined as new 、 、 and , repeat S1; S5, in When the length of is equal to 1, the target data provider will Sending to the target data user; S6, the target data user calculates the length of 1 according to S3 、 、 ,calculate ,if , confirm that the verification is successful, otherwise the verification fails.

6. The method according to claim 1 or 5, characterized in that The first substrate and the second substrate Including from the definition in Elliptic curve group on Two independent sets of element points are selected from ; wherein the number of element points in the element point set is the same as the number of model parameters in the global model.

7. The method according to claim 1, characterized in that Determining a high target quality score from the verified quality scores includes: The target data user performs joint sampling on all verified quality scores and calculates the probability value ; Where D is the quality score of all verified values, and t is the sampling temperature for adjusting the output distribution; A quality score exceeding the probability value is determined as a high target quality score.

8. A data processing method, characterized in that: The method is applied to a target data user in a data processing system based on federated learning, wherein the target data user and multiple target data providers in the processing system form a federated learning cluster; the method includes: The target data user publishes the global model to be trained to the target data provider in the cluster; Receive integrity commitments returned by each target data provider for calculating a local model based on a preset multi-party security protocol; wherein the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider, and the local model is a trained local model obtained by the target data provider through model training of the global model based on local data; Disclosing a benchmark model to each target data provider; wherein the benchmark model is used to evaluate the local models sent by each target data provider; Receive the quality scores of the layout models evaluated based on the benchmark model sent by each target data provider, and calculate the quality scores of the layout models based on the benchmark model through the formula Generating an authenticity commitment for the quality score; wherein, is the value of the authenticity commitment, is the model vector of the local model of the target data provider, is the second substrate, is the model vector of the baseline model, is the first substrate, represents the quality score of the local model of the target data provider, Indicates the random number selected by the target data provider; Verifying whether the quality score is authentic based on the authenticity commitment, further determining a high target quality score from the verified quality scores, and obtaining a target local model from a target data provider that sent the target quality score; calculate Is it true? If so, determine that the target local model is complete, and further aggregate the complete target local model into a new global model; where t is the round of federated learning in which this training is located, i is the serial number of the target data user, The local model obtained from the target data provider, The integrity commitment sent by the target data provider, a session key generated for the multi-party security protocol, The global key held by the target data consumer.

9. A data processing method, characterized in that: The method is applied to a target data provider in a data processing system based on federated learning, wherein the target data provider, other data providers, and target data users in the processing system form a federated learning cluster; the method comprises: The target data provider obtains the global model to be trained published by the target data user; Performing model training on the global model based on local data to obtain a trained local model, further calculating an integrity commitment for the local model based on a preset multi-party security protocol, and sending the integrity commitment to the target data user; wherein the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider; The quality score of the local model is evaluated based on the benchmark model disclosed by the target data user, and further calculated by formula Generate an authenticity commitment for the quality score, and send the quality score and the authenticity commitment to the target data user; wherein, is the value of the authenticity commitment, is the model vector of the local model of the target data provider, is the second substrate, is the model vector of the baseline model, is the first substrate, represents the quality score of the local model of the target data provider, Indicates the random number selected by the target data provider; Cooperating with the target data user, using an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment; When the quality score is true and is a high target quality score, the target local model is sent to the target data user so that the target data user calculates Is it true? If so, determine that the target local model is complete, and further aggregate the complete target local model into a new global model; where t is the round of federated learning in which this training is located, i is the serial number of the target data user, The local model obtained from the target data provider, The integrity commitment sent by the target data provider, a session key generated for the multi-party security protocol, The global key held by the target data consumer.

10. A data processing device, characterized in that: The device is applied to a target data user in a data processing system based on federated learning, wherein the target data user and multiple target data providers in the processing system form a federated learning cluster; the device comprises: A first publishing unit, a target data user publishing a global model to be trained to a target data provider in the cluster; A first receiving unit receives integrity commitments returned by each target data provider for calculating a local model based on a preset multi-party security protocol; wherein the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider, and the local model is a trained local model obtained by the target data provider through model training of the global model based on local data; A second publishing unit, which discloses a benchmark model to each target data provider; wherein the benchmark model is used to evaluate the local models sent by each target data provider; The second receiving unit receives the quality scores sent by each target data provider and obtained by evaluating the layout model based on the benchmark model, and Generating an authenticity commitment for the quality score; wherein, is the value of the authenticity commitment, is the model vector of the local model of the target data provider, is the second substrate, is the model vector of the baseline model, is the first substrate, represents the quality score of the local model of the target data provider, Indicates the random number selected by the target data provider; an authenticity verification unit, which verifies whether the quality score is authentic based on the authenticity commitment, further determines a high target quality score from the verified quality scores, and obtains a target local model from a target data provider that sends the target quality score; Integrity verification unit, calculation Is it true? If so, determine that the target local model is complete, and further aggregate the complete target local model into a new global model; where t is the round of federated learning in which this training is located, i is the serial number of the target data user, The local model obtained from the target data provider, The integrity commitment sent by the target data provider, a session key generated for the multi-party security protocol, The global key held by the target data consumer.

11. A data processing device, characterized in that: The device is applied to a target data provider in a data processing system based on federated learning, wherein the target data provider, other data providers, and target data users in the processing system form a federated learning cluster; the device comprises: A model acquisition unit, where the target data provider acquires the global model to be trained published by the target data user; a model training unit, which performs model training on the global model based on local data to obtain a trained local model, further calculates an integrity commitment for the local model based on a preset multi-party security protocol, and sends the integrity commitment to the target data user; wherein the multi-party security protocol is pre-negotiated and determined between the target data user and the target data provider; The model evaluation unit evaluates the quality score of the local model based on the benchmark model disclosed by the target data user, and further calculates the quality score of the local model through the formula Generate an authenticity commitment for the quality score, and send the quality score and the authenticity commitment to the target data user; wherein, is the value of the authenticity commitment, is the model vector of the local model of the target data provider, is the second substrate, is the model vector of the baseline model, is the first substrate, represents the quality score of the local model of the target data provider, Indicates the random number selected by the target data provider; an authenticity verification unit, cooperating with the target data user, using an interactive verification method to verify whether the quality score is authentic based on the authenticity commitment; When the quality score is true and is a high target quality score, the target local model is sent to the target data user so that the target data user calculates Is it true? If so, determine that the target local model is complete, and further aggregate the complete target local model into a new global model; where t is the round of federated learning in which this training is located, i is the serial number of the target data user, The local model obtained from the target data provider, The integrity commitment sent by the target data provider, a session key generated for the multi-party security protocol, The global key held by the target data consumer.

12. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 9.

Images