A federated learning-based trusted model training method

Abstract

The application discloses a trusted model training method based on federated learning, and belongs to the field of machine learning. In view of the problem of large proof overhead in the prior art, in the application, a task initiator issues a task and generates public parameters in an initialization stage, a plurality of clients cooperate to generate and issue crs, the problem of trapdoor leakage is solved, the client preprocesses a local data set, commits a data subset and uploads the data subset to an aggregation server; in a proof stage, the client trains a local model, and respectively converts convolution, pooling and activation layers in the training process into matrix multiplication operations, performs zk-SNARK proof, and generates trusted evidence; in a verification stage, data authenticity, data integrity and training integrity are verified respectively, and the three characteristics ensure that the local training process is realizable. The technical scheme of the application optimizes the online proof efficiency and storage overhead of federated learning without affecting accuracy.

Description

Technical Field

[0001] This invention relates to a reliable model training method based on federated learning, belonging to the field of machine learning. Background Technology

[0002] Federated learning (FL) is a distributed machine learning framework that enables collaborative training among multiple parties to obtain high-quality global models without sharing data, thus providing default privacy protection. While FL offers privacy protection for participating clients, its unique architecture also introduces several security challenges. To protect privacy, the FL model training process is only visible to the client itself, which provides an environment for malicious clients to launch attacks.

[0003] Federated learning has shortcomings in defending against malicious attacks. Some studies have pointed out that malicious clients may not use genuine local datasets, or fail to perform local training tasks as required, using poisoned data to train local models, or uploading outdated or fake model update parameters. These improper behaviors can affect the correctness of the local model, compromise its accuracy, or hinder the convergence of the global model. Especially in real-world applications based on federated learning, it can lead to untimely or incorrect model judgments, or even serious economic losses.

[0004] Existing federated learning frameworks rarely consider the scenario of untrusted nodes. Most verifiable federated learning schemes verify the correctness of the aggregation results, as the aggregation server may be compromised or maliciously attacked. Most existing work verifies corrections through proofs, signatures, and blockchains of the aggregation results. However, they all rely on the assumption that the gradients submitted by participating nodes are correct, but none consider whether the real local dataset is used during model training. Taking a cross-institutional federated learning scenario as an example, dishonest clients might use only a portion of their local data to update their local model, and then they could cheat rewards with a smaller dataset and less training work. Furthermore, the authenticity of the training dataset is a prerequisite for the usability of federated training results, which can be combined with the solutions mentioned above to establish model trustworthiness in federated learning with malicious clients. To our knowledge, very few works have explored this issue.

[0005] Existing federated learning schemes typically use trusted execution environments based on trusted hardware to ensure the trustworthiness of the local execution process. However, it is difficult to provide a unified trusted execution environment for all clients with heterogeneous hardware resources. Recently, a method based on zk-SNARK proofs to ensure the trustworthiness of the local training process has been proposed. By employing software-based zk-SNARK and various cryptographic techniques, it achieves proofs and privacy protection for local training in federated learning, ensuring training integrity—that is, the client can correctly execute the training task as expected. This prevents lazy clients from incompletely executing the training process or malicious clients from providing false model updates, thus achieving fair and trustworthy federated learning.

[0006] Existing methods that directly use zk-SNARK to generate proofs result in a sharp performance drop because the proof time becomes a bottleneck for zk-SNARK-based proofs. Some work has focused on improving the efficiency of zk-SNARK proofs by optimizing the proofs generated during neural network training in trusted model training. Convolutional computations involve a large number of multiplication gates, making convolution operations the dominant performance bottleneck. Furthermore, while zk-SNARK is suitable for handling addition and multiplication gates, how to transform it to handle nonlinear layers in neural networks is a challenge. Therefore, optimizing neural network computations to reduce proof overhead is a key challenge in achieving trusted local training in federated learning. Summary of the Invention

[0007] To address the problems existing in the prior art, this application proposes a trusted model training method based on federated learning, which achieves trusted and secure global model aggregation in malicious environments without affecting accuracy.

[0008] The technical solution adopted in this invention is a trusted model training method based on federated learning, including an initialization phase, a proof phase, and a verification phase, specifically including the following steps:

[0009] The initialization phase includes,

[0010] The task initiator publishes the FL task and generates common parameters;

[0011] The client accepts the task, downloads common parameters, and preprocesses the dataset;

[0012] The aggregation server (AS) distributes the global model.

[0013] The proof phase includes,

[0014] The client trains locally, obtains local gradient updates, and generates credible proofs based on the training process.

[0015] The client packages and sends the data commitment, credible evidence, and local gradient to the aggregation server AS.

[0016] The verification phase includes,

[0017] The aggregation server AS receives messages, queries the Bloom filter, verifies data commitments, and verifies credible evidence.

[0018] In the optimized federated learning-based trusted model training method described above, during the initialization phase, the task initiator publishes the federated learning task, announces the initial global model θ0, generates public parameters pp and sends them to the aggregation server AS; multiple nodes collaborate to generate and publish crs.

[0019] Client P j A subset of commitments from the local dataset is sent to the aggregation server AS. The aggregation server AS stores the commitments in a Bloom filter and then exposes the public parameter pp to each participating client P. j ;

[0020] Client P j After receiving the common parameter pp, use the modulus sequence H = {h1, h2, ..., h} within it. n-m+1 The sample matrix X in the local dataset D is preprocessed, including the Chinese Remainder Theorem and the im2col transformation, and then the processed matrix E is obtained.

[0021] In the optimized federated learning-based trustworthy model training method described above, during the proof phase,

[0022] In each iteration of the federated learning task, participating client P j A new round of local model is trained using a subset of local data, and a local proof is generated for the convolutional neural network operations during training using the preprocessed matrix E. This proof includes proofs for convolutional and nonlinear layers, generating credible evidence to demonstrate that the client correctly executed the local training operations and that the local model generation process is credible.

[0023] Participating client P j The commitments and credible evidence of this training subset are sent together to the aggregation server AS.

[0024] In the optimized, federated learning-based trust model training method described above, during the validation phase...

[0025] The aggregation server AS receives client P j After sending a message, first check whether the commitment of the corresponding data subset is stored in the Bloom filter to verify the authenticity of the client's data;

[0026] Aggregation server AS to client P j Verify the integrity of the client's training by using credible evidence;

[0027] The aggregation server AS verifies the client's commitment to the complete dataset and the commitment to a subset of the data, thus validating the data integrity of the client's participation in federated learning.

[0028] The optimized, federated learning-based trusted model training method described above involves the task initiator publishing the federated learning task, releasing the initial global model θ0, generating public parameters pp, and sending them to the aggregation server AS. The specific process includes...

[0029] The task initiator distributes federated learning tasks, initializes the global model θ0 and common parameters pp, and selects a large prime number p to generate a prime group of order p. Where p > 2 λ λ is a security parameter, and a bilinear mapping is generated containing three groups G1, G2, G... T Satisfying e(G1,G2)→G T , where g and h are generators of G1 and G2, respectively;

[0030] Next, select the desired number of training epochs T, the dimension of the sample matrix n×n, and the dimension of the convolution kernel m×m, and construct an m... 2 ×m 2 Given a singular square matrix U, we finally calculate a modulus sequence H = {h1, h2, ..., h...} n-m+1};

[0031] The task initiator will share public parameters Send to aggregation server AS;

[0032] The specific process of multiple nodes collaboratively generating and publishing CRS includes,

[0033] Suppose there are ρ nodes collaboratively generating a CRS. Each node first converts the constraints of its local proof from an NP-hard language into the corresponding relation. Then, each node selects 6 random numbers x. i α i ,β i γ i δ i , z i ∈Z q The calculation is as follows:

[0034] Each node broadcasts (α) i ,β i ,γ i ,δ i ,x i ,z i It receives broadcasts from other ρ-1 nodes and aggregates the parameters of all nodes as follows:

[0035]

[0036] Next, each node calculates... as follows,

[0037]

[0038] Finally, It is made public to all nodes.

[0039] The optimized, federated learning-based trust model training method described above includes the following specific preprocessing steps for the sample matrix X in the local dataset D:

[0040] Client P j After downloading the public parameters pp, use the modulus sequence H = {h1, h2, ..., h} within it. n-m+1 Preprocess the sample matrix X in the local dataset D. i,j =CRT(x i,j ,x i+1,j ,...,x i+n-m,j ),

[0041]

[0042] Client P j Subset of local data D j The promise of comm(D) j )=U×E j Send to aggregation server AS, aggregation server AS will promise comm(D) j Stored in the Bloom filter BF.

[0043] The optimized, federated learning-based trustworthy model training method described above, in each iteration of the federated learning task, involves client P... j The specific process of training a new local model using a subset of local data includes:

[0044] Assume client P j The global model θ was obtained in the t-th training round. t-1 Client P j In dataset D j Train a local model on a subset of the dataset to extract the convolution kernel matrix W and the feature matrix E;

[0045] A local proof is generated using the preprocessed matrix E for the convolutional neural network operations during training. The specific process includes...

[0046] Client P j Generate credible evidence using an optimized zk-SNARK evidence generation algorithm;

[0047] Client Pj First, the convolutional layer operation is proven, and the client P... j The batch convolution operation is constructed as multiple matrix multiplications W×E=Z, and the multiple matrix multiplications are combined by setting an uncertain variable Q, such that ΣQ i W i ×ΣQ i E i =ΣQ i Z i Established;

[0048] According to the QMP-based zk-SNARK definition, the left-hand input is... Right-line input is Output line is Client P j Set φ to (Q1, Q2, ..., Q s ), set ω to (Q s+1 Q s+2 ,...,Q N And generate credible evidence by executing the algorithm π←Prove(crs,φ,ω), client P j Random in Choose r and s, and calculate the evidence as follows:

[0049]

[0050] Client P j Evidence is given that π = (A, B, C, D) makes the equation e(A, B) = e(g) true. α ,h β )·e(D,h γ )·e(C,h δ ) was established;

[0051] For nonlinear layers such as max pooling layers and activation function layers, 0 / 1 matrices are used to simulate nonlinear operations and nonlinear operations are converted into matrix operations;

[0052] In the max pooling layer, based on the position of the maximum element in each pooling region, the corresponding elements of the max pooling operation matrix PM are set to 1, and the remaining elements are set to 0. This transforms the nonlinear max pooling operation into a matrix multiplication operation with the output matrix Z of the previous layer.

[0053] In the activation layer, a ReLU operation matrix RM with 0 and 1 elements is constructed, where 1 corresponds to the element greater than 0 in the input ReLU activation layer. This matrix is ​​then multiplied with the feature matrix Z output from the convolutional layer. The evidence generation algorithm based on QMP designed above is then used for processing.

[0054] Client P j After completing model training and evidence generation, subset commitment Comm(D ji ), trusted evidence π, and local gradient updates are packaged and sent to the aggregation server AS.

[0055] Optimized, for the above trusted model training method based on federated learning, after the aggregation server AS receives the message sent by the client P j , it first queries whether the commitment of the corresponding data subset is stored in the Bloom filter to verify the authenticity of the client's data. The specific process is as follows:

[0056] Data authenticity verification: The aggregation server AS uses ξ unbiased hash functions of the Bloom filter to calculate the hash of the data subset commitment and modulo the length L of the Bloom filter to store the hash of the subset commitment in the Bloom filter. Specifically, bit k ←H k (hash(Comm(D ji ))) mod L, and determines whether the corresponding bit is 1 in the Bloom filter;

[0057] Each bit will be misjudged with a misjudgment rate of ò. If the results of all ξ queries are 1, then there is a very high probability of 1 - ò ξ to confirm that the Bloom filter contains the commitment of this subset, that is, the data authenticity verification is successful.

[0058] The aggregation server AS verifies the trusted evidence of the client P j to verify the training integrity of the client. The specific process is as follows:

[0059] Data integrity verification: Assume that after t rounds of training, the global model converges and the model accuracy meets the requirements;

[0060] When t < T, it means that the global model meets the requirements of the task initiator before all committed subsets are used up, and the federated learning task is completed. At this time, we require all clients to upload the remaining commitments for data integrity verification;

[0061] When t = T, directly perform data integrity verification;

[0062] When t > T, it means that after all subsets have been trained, the global model still does not meet the requirements. We use the same data subset to train the next T rounds, and perform an integrity verification every T rounds until the model converges;

[0063] The aggregation server calculates the challenge value η = hash(Comm(D)), and verifies Check if the equation is true. If it is true, the data integrity verification is successful, and we get b2 = 1.

[0064] The aggregation server AS verifies the client's commitment to the complete dataset and the commitment to a subset of the data, thus validating the data integrity of the client's participation in federated learning. The specific process is as follows:

[0065] For training integrity verification, the aggregation server AS executes the algorithm b3←Verify(crs,φ,π), resolves π as (A,B,C,D), and calculates the equation e(A,B)=e(g) α ,h β )·e(D,h γ )·e(C,h δ If the condition is met, the training completeness is passed.

[0066] Finally, verification If b is 1, it proves that the client's local model training is being performed correctly; otherwise, the client is considered malicious, and its gradient updates are discarded.

[0067] The beneficial effects of this application are as follows:

[0068] In the technical solution of this application, a credible proof of the integrity of local training for federated learning is designed and implemented, and a proof of credible model training on the client side in a malicious environment is generated without affecting accuracy.

[0069] By using the Chinese Remainder Theorem to preprocess the dataset, the online evidence generation time is significantly reduced; the nonlinear layer operations in the neural network are optimized, reducing the proof overhead for generating credible evidence.

[0070] This solves the problem of excessive overhead in convolution operations in convolutional neural network models.

[0071] A lightweight verification of data authenticity and integrity is implemented in a client / server architecture; a circuit improvement based on zero-knowledge concise non-interactive knowledge argumentation is used to improve the proof efficiency of batch proofs.

[0072] The technical solution of this application also utilizes a lightweight commitment scheme and Bloom filter, which greatly improves the communication and storage overhead of federated learning. Attached Figure Description

[0073] Figure 1 This is a system model diagram of the trusted model training method based on federated learning proposed in this invention. Detailed Implementation

[0074] The technical features of the present invention will be further illustrated below with reference to specific embodiments.

[0075] This application proposes a method for training a trusted model based on federated learning in a malicious environment, such as... Figure 1 As shown, the entity mainly includes the task initiator, the client, and the aggregation server (AS), and its specific process mainly includes the following steps.

[0076] 1) The task initiator publishes the FL task and generates common parameters;

[0077] 2) The client accepts the task, downloads common parameters, and preprocesses the dataset;

[0078] 3) Aggregator server AS distributes the global model;

[0079] 4) The client trains locally, obtains local gradient updates, and generates credible proofs based on the training process;

[0080] 5) The client packages the data commitment, credible evidence, and local gradient and sends them to the aggregation server AS;

[0081] 6) The aggregation server AS receives the message, queries the Bloom filter, verifies the data commitment, and verifies the credible evidence.

[0082] We will now describe the specific process in three stages: the initialization stage, the proof stage, and the verification stage.

[0083] Initialization phase:

[0084] The task initiator distributes federated learning tasks, initializes the global model θ0 and common parameters pp, and selects a large prime number p to generate a prime group of order p. Where p > 2 λ λ is a security parameter, and a bilinear mapping is generated containing three groups G1, G2, G... T Satisfying e(G1,G2)→G T Where g and h are the generators of G1 and G2 respectively, then the desired number of training epochs T, the dimension of the sample matrix n×n, and the dimension of the convolution kernel m×m are selected to construct an m 2 ×m 2 Given a singular square matrix U, we finally calculate a modulus sequence H = {h1, h2, ..., h...} n-m+1 The task initiator will provide public parameters. Send to aggregation server AS.

[0085] Suppose there are ρ nodes collaboratively generating a CRS. Each node first converts the constraints of its local proof from an NP-hard language into the corresponding relation. Then, each node selects 6 random numbers x. i α i ,βi γ i δ i , z i ∈Z q The calculation is as follows:

[0086] Each node broadcasts (α) i ,β i ,γ i ,δ i ,x i ,z i It receives broadcasts from other ρ-1 nodes and aggregates the parameters of all nodes into a single data structure.

[0087]

[0088] Next, each node calculates... as follows,

[0089]

[0090] Finally, It is made public to all nodes.

[0091] Client P j After downloading the public parameters pp, use the modulus sequence H = {h1, h2, ..., h} within it. n-m+1 Preprocess the sample matrix X in the local dataset D. i,j =CRT(x i,j ,x i+1,j ,...,x i+n-m,j ), as shown below:

[0092]

[0093] Client P j Subset of local data D j The promise of comm(D) j )=U×E j Send to aggregation server AS, aggregation server AS will promise comm(D) j Stored in the Bloom filter BF.

[0094] Proof Phase: Each node performs the following operations to generate credible evidence:

[0095] Local model training: Assume P j The global model θ was obtained in the t-th training round. t-1 P j In dataset P j The local model is trained on a subset of the dataset to extract the convolution kernel matrix W and the feature matrix E.

[0096] Local evidence generation: P j Confidential evidence is generated using an optimized zk-SNARK evidence generation algorithm. j First, the operation of the convolutional layer is proven, P j The batch convolution operation is constructed as multiple matrix multiplications W×E=Z, and the multiple matrix multiplications are combined by setting an uncertain variable Q, such that ΣQ i W i ×ΣQ i E i =ΣQ i Z i Established. According to the QMP-based zk-SNARK definition, the left-hand input is... Right-line input is Output line is Client P j Set φ to (Q1, Q2, ..., Q s ), set ω to (Q s+1 Q s+2 ,...,Q N And generate credible evidence by executing the algorithm π←Prove(crs,φ,ω), client P j Random in Choose r and s, and calculate the evidence as follows:

[0097]

[0098] Client P j Evidence is given that π = (A, B, C, D) makes the equation e(A, B) = e(g) true. α ,h β )·e(D,h γ )·e(C,h δ It was established.

[0099] For nonlinear layers such as max pooling layers and activation function layers, we use 0 / 1 matrices to simulate nonlinear operations and convert nonlinear operations into matrix operations. In the max pooling layer, based on the position of the maximum element in each pooling region, we set the corresponding elements of the max pooling operation matrix PM to 1 and the remaining elements to 0. This transforms the nonlinear max pooling operation into a matrix multiplication operation with the output matrix Z of the previous layer. In the activation layer, we construct a ReLU operation matrix RM with 0 and 1 elements, where 1 corresponds to an element greater than 0 in the input ReLU activation layer. We then multiply this matrix by the feature matrix Z output from the convolutional layer. The evidence was then processed using the QMP-based optimized evidence generation algorithm designed above.

[0100] Client P j After completing model training and evidence generation, subset commitment Comm(D ji ), trusted evidence π, and local gradient updates are packed and sent to the aggregation server AS.

[0101] Verification phase: The aggregation server performs the following operations.

[0102] Data authenticity verification: The aggregation server AS uses ξ unbiased hash functions of the Bloom filter to calculate the hash of the data subset commitment and modulo the length L of the Bloom filter to store the hash of the subset commitment in the Bloom filter. Specifically, bit k ←H k (hash(Comm(D ji ))) mod L, and determines whether the corresponding bit is 1 in the Bloom filter. Each bit will be misjudged with a misjudgment rate of ò. If the results of all ξ queries are 1, then there is a very high probability of 1 - ò ξ to confirm that the Bloom filter contains the commitment of this subset, that is, the data authenticity verification is successful.

[0103]

[0104] Data integrity verification: Assume that after t rounds of training, the global model converges and the model accuracy meets the requirements. When t < T, it means that the global model meets the requirements of the task initiator before all committed subsets are used up, and the federated learning task is completed. At this time, we require all clients to upload the remaining commitments for data integrity verification; when t = T, directly perform data integrity verification; when t > T, it means that after all subsets have been trained, the global model still does not meet the requirements. We use the same data subset to train the next T rounds, and perform integrity verification once every T rounds until the model converges. The aggregation server calculates the challenge value η = hash(Comm(D)) and verifies whether the equation holds. If it holds, the data integrity verification passes, and b2 = 1 is obtained.

[0105] Training integrity verification: The aggregation server AS executes the algorithm b3←Verify(crs, φ, π), parses π as (A, B, C, D), and calculates whether the equation e(A, B) = e(g α , h β )·e(D, h γ )·e(C, h δ ) holds. If it holds, the training integrity passes. Finally, verify If b is 1, it proves that the client's local model training is actually executed. Otherwise, it is determined that the client is malicious and its gradient update is discarded.

[0106] The dataset preprocessing algorithm is as follows:

[0107] Input: an n×n sample matrix X and a sequence of moduli H = {h1, h2, ..., hn} n-m+1}

[0108] Output: Preprocessed matrix E

[0109] 1. Client P calculates the Chinese Remainder Theorem on the sample matrix X.

[0110] 2. For j = 1 to n, client P randomly selects column vector (x 1,j ,x 2,j ,...,x n,j ) T

[0111] 3. For i = 1 to m, the client uses the modulo sequence to calculate e. i,j =CRT(x i,j ,x i+1,j ,...,x i+n-m,j )

[0112] 4. Then obtain the coefficient matrix X i ′, where {e i,j} is matrix X′ i Each element.

[0113] 5. In order to transform the m×n coefficient matrix X i X′ is converted into matrix multiplication using an m×m convolution kernel, and then multiplied using the im2col algorithm. i Transform into E i =im2col(X′) i ).

[0114] 6. Return to E i .

[0115] The improved zk-SNARK-based evidence generation algorithm is:

[0116] Input: Processed matrix E; Feature matrix Z; Common reference string crs; Global model parameters θ;

[0117] Output: Credible evidence π;

[0118] The client-side parsing process extracts the model parameters θ to obtain the convolution kernel W and bias b.

[0119] The analytic statement φ is (Q1, Q2, ..., Q s ), the analytical evidence ω is (Q s+1 Q s+2 ,...,Q N);

[0120] For i = 1 to N, such that ΣQ i W i ×ΣQ i E i =ΣQ i Z i The equation holds true;

[0121] Client in domain Randomly select r and s;

[0122] Calculate the equation

[0123]

[0124] Construct evidence π = (A, B, C, D);

[0125] Return π.

[0126] Comparison of the technical solution of this application with other verifiable convolutional neural network solutions:

[0127] Functionally, similar to VeriML, vCNN, and pvCNN, we optimize the proof process of verifiable convolutional neural networks. Building upon this, we further apply it to federated learning scenarios. Our proposed scheme employs a credible model update method, requiring all local model training to generate credible evidence proving the actual execution of convolutional neural network operations. Compared to VeriML, vCNN, and pvCNN, we optimize the convolution operation computation by using the Chinese Remainder Theorem for preprocessing the sample matrix. Furthermore, we improve the QMP-based zk-SNARK scheme for arithmetic circuit operations, significantly reducing the online proof time. Theoretically, our online proof time comparison with other schemes is shown in the table below:

[0128] Table 1. Analysis of Online Proof Time

[0129] VeriML <![CDATA[O(N·M·n 2 m 2 )]]> vCNN <![CDATA[O(N·M(n 2 +m 2 ))]]> pvCNN <![CDATA[O(N·M·n 2 )]]> TMT-FL <![CDATA[O(N·n 2 )]]>

[0130] Based on computational cost analysis, the CRS generation time and proof time of other schemes are shown in the table below:

[0131] Table 2 Proof Time

[0132] 16×16 1020s 602s 198s 123s 20×20 1950s 884s 346s 148s 24×24 3160s 1331s 508s 158s 28×28 4400s 1782s 648s 264s

[0133] Table 3 CRS Generation Time

[0134] 16×16 1530s 682s 331s 268s 20×20 2925s 963s 611s 334s 24×24 4740s 1486s 913s 378s 28×28 6588s 1895s 1331s 651s

[0135] The computational cost of this scheme is lower than that of other schemes in both the setup and proof phases, and it is highly efficient. In particular, compared with Groth16, the CRS generation cost and proof cost increase significantly as the dimension of the sample matrix increases.

[0136] From a storage cost analysis perspective, the storage overhead of clients using different matrix dimensions is shown in the table below:

[0137] Table 4 Client Storage Overhead

[0138] 16×16 404.97kb 239.01kb 93.40kb 194.23kb 20×20 629.26kb 285.81kb 239.04kb 301.81kb 24×24 903.40kb 380.52kb 537.96kb 433.30kb 28×28 1227.39kb 497.09kb 1053.51kb 588.69kb

[0139] As shown in the table, with the increase of matrix dimension, the storage overhead of this scheme is less than that of Groth16 and pvCNN schemes, and slightly higher than that of vCNN scheme. In particular, the storage size increases at a much faster rate with the increase of matrix dimension than the pvCNN scheme.

[0140] The solution proposed in this application will now be described in conjunction with a specific application.

[0141] In smart healthcare scenarios: Detecting a rare disease in a hospital is extremely difficult. A single medical institution's sample database contains few records, making it impossible to train a stable and accurate predictive model. Furthermore, the sample information contains patients' private information and cannot be directly disclosed. To prevent malicious users from constructing false gradients to disrupt the global model, or from forging gradients to obtain global model results due to unwillingness to use private data for training, [further measures are needed].

[0142] To address this situation, the proposed solution is applicable to multiple medical institutions jointly training a high-accuracy predictive model for a specific disease. Each institution continuously refines the global model based on its own sample data and provides credible evidence of the training process. This process continues until the task issuer's requirements are met. This approach is highly effective for disease research and diagnosis, and there is no concern about model corruption or backdoors being embedded. It is fair to all participating institutions.

[0143] In smart finance: The financial industry faces challenges such as information asymmetry, incomplete user credit records, high costs and difficulty in fully covering manual verification by banks, as well as insufficient risk control capabilities and loan users' risk identification abilities. These challenges present both opportunities and challenges for financial institutions in risk control.

[0144] In response to this situation, the proposed solution is applicable to multiple institutions using multi-dimensional data to build a joint financial risk model. On one hand, federated learning technology can train a more diverse global model while ensuring client data remains locally. Therefore, multi-dimensional data from multiple institutions can construct a more accurate risk control model. On the other hand, the proposed solution also prevents malicious clients from using poisoning attacks to damage the global risk control model and cause economic losses. Similarly, it prevents malicious users from using a portion of the dataset instead of the entire dataset for training and from freely obtaining the aggregated results, ensuring the authenticity and reliability of local gradient updates on each client, thereby ensuring a more accurate global risk control model.

[0145] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A reliable model training method based on federated learning, characterized in that: It includes an initialization stage, a proof stage, and a verification stage; The initialization stage includes The task initiator publishes the FL task and generates public parameters; The client accepts the task, downloads common parameters, and preprocesses the dataset; the aggregation server... Distribute the global model; The proof stage includes The client conducts local training to obtain local gradient updates and generates a credible proof based on the training process, including: Using the preprocessed matrix E to generate a local proof for the convolutional neural network operations during the training process, including: For the max pooling layer and the activation function layer (non-linear layer), use a 0 / 1 matrix to simulate the non-linear operations and convert the non-linear operations into matrix operations; In the max pooling layer, based on the position of the maximum element in each pooling region, the corresponding elements of the max pooling operation matrix PM are set to 1, and the remaining elements are set to 0. This transforms the non-linear max pooling operation into a matrix multiplication operation with the output matrix Z of the previous layer. ; In the activation layer, a ReLU operation matrix RM with elements of 0 and 1 is constructed, where 1 corresponds to the element greater than 0 in the input ReLU activation layer. This matrix is ​​then multiplied by the feature matrix Z output from the convolutional layer. The evidence generation algorithm based on QMP is used for processing. The client packages the data commitment, credible evidence, and local gradients and sends them to the aggregation server. ; The verification stage includes Aggregator server Received client After sending the message, query whether the commitment of the corresponding data subset is stored in the Bloom filter to verify the authenticity of the client's data. The process is as follows: Data authenticity verification, aggregation server Using Bloom filters An unbiased hash function is used to compute the hash of the data subset commitment, and the hash of the subset commitment is stored in the Bloom filter with the length L congruent to it. Determine whether the corresponding bit is 1 in the Bloom filter; each bit will be... The misjudgment rate is incorrect if all [cases are detected]. If all the query results are 1, then there is The probability is used to confirm the commitment that the Bloom filter contains the subset, i.e., the data authenticity verification is successful. ; Aggregator server For the client Verify the integrity of the client's training using credible evidence; including: Data integrity verification. Assume that after t rounds of training, the global model converges and the model accuracy meets the requirements; When t < T, all clients are required to upload the remaining commitments for data integrity verification; When t = T, directly conduct data integrity verification; When t > T, use the same data subset to train for the next T rounds, and perform integrity verification once every T rounds until the model converges; Aggregator server Calculate the challenge value ,verify Check if the equation is true. If it is true, the data integrity verification passes. ; Aggregator server The process involves verifying the client's commitment to the complete dataset and the commitment to a subset of the data. Training integrity verification is performed by the aggregation server AS through the execution of the algorithm. Analysis for Calculate the equation If the condition is met, the training completeness is passed. Finally, verification If b is 1, it proves that the client's local model training is actually performed; otherwise, the client is considered malicious and its gradient update is discarded.

2. The trusted model training method based on federated learning according to claim 1, characterized in that: During the initialization phase, the task initiator publishes the federated learning task and releases the initial global model. Generate common parameters And send it to the aggregation server. Multiple nodes collaborate to generate and publish. ; Client Send a subset of the local dataset to the aggregation server. Aggregator server After storing the commitment in the Bloom filter, the public parameters will be... Open to all participating clients ; Client Upon receiving public parameters Then, use the modulus sequence. For local datasets The sample matrix in Preprocessing is performed, including operations such as the Chinese Remainder Theorem and... Transformation, then obtain the processed matrix. .

3. The trusted model training method based on federated learning according to claim 1, wherein: In the proof stage In each iteration of the federated learning task, participating clients Train a new round of local model using a subset of local data, and use the preprocessed matrix. Generate a local proof for the convolutional neural network operations during training, including proofs for convolutional layers and nonlinear layers, to generate credible evidence that the client correctly executed the local training operations and that the local model generation process is credible. Participating Client The commitments and credible evidence from this training subset will be sent together to the aggregation server. .

4. The trusted model training method based on federated learning according to claim 2, wherein: The task initiator publishes the federated learning task and releases the initial global model. Generate common parameters And send it to the aggregation server. The specific process includes, The task initiator distributes federated learning tasks and initializes the global model. and common parameters The task initiator chooses a large prime number. Generate a prime group ,in , λ It is a security parameter, and generates a bilinear mapping containing three groups. G 1 , G 2 , G T satisfy e ( G 1 , G 2 → G T ,in g, h They are G 1 , G 2 Generators; Next, select the desired number of training rounds. Dimensions of the sample matrix convolution kernel dimension Construct a strange square Finally, a modulus sequence is calculated. ; The task initiator will share public parameters Send to aggregation server ; Multiple nodes collaborate to generate and publish. The specific process includes, Assume there is ρ Each node generates collaboratively. Each node first converts the constraints of its local proof from an NP-complete language into the corresponding relation, and then each node selects 6 random numbers. x i , α i , β i , γ i , δ i , z i ∈Z q The calculation is as follows: Each node broadcasts ( α i , β i , γ i , δ i , x i , z i ), receiving from other ρ - 1 The node broadcasts the data and aggregates the parameters of all nodes as follows: ； Next, each node calculates... as follows, 、 ； Finally, It is made public to all nodes.

5. The trusted model training method based on federated learning according to claim 2, wherein: For local datasets The sample matrix in The specific preprocessing steps include: Client Download public parameters Then, use the modulus sequence. For local datasets The sample matrix in Preprocessing , ； Client Subset of local data promise Send to aggregation server Aggregator server Will promise Stored in the Bloom filter BF.

6. The trusted model training method based on federated learning according to claim 3, wherein: In each iteration of the federated learning task, participating clients The specific process of training a new local model using a subset of local data. includes Assuming the client In the The global model was obtained during the training rounds. Client In the dataset Train a local model on a subset of the dataset to extract the convolution kernel matrix. and characteristic matrix ; Using the preprocessed matrix The specific process of generating a local proof for the convolutional neural network operations during training. includes Client Generate credible evidence using an optimized zk-SNARK evidence generation algorithm; Client First, the convolutional layer operation is proven, and then the client... Construct batch convolution operations as multiple matrix multiplications. And by setting uncertain variables Combine multiple matrix multiplications into a single system, such that Established; According to the QMP-based zk-SNARK definition, the left-hand input is... The right-hand input is The output line is Client set up for ,set up for And generate credible evidence by executing the algorithm. Client Random in Select The evidence is calculated as follows: ； Client Obtain evidence This makes the equation Established; Client After completing model training and evidence generation, subset commitments will be made. credible evidence and local gradient update Package and send to the aggregation server. .

Images