A data sharing method and apparatus based on blockchain and federated learning

Abstract

This invention discloses a data sharing method and apparatus based on blockchain and federated learning. In the federated learning network, a task initiator node initiates a federated learning task to the blockchain network; the blockchain network broadcasts this task to the federated learning network; federated nodes that confirm their eligibility to participate in the task provide feedback to the blockchain network; the blockchain network sends the global model to the participating federated learning nodes; each federated learning node iteratively updates its local model; after the local update, it sends the local model back to the blockchain network; the blockchain network executes the corresponding smart contract based on the task type; iterative training is performed according to the set number of training rounds; and the blockchain network sends the finally trained global model back to the task initiator. This invention can improve the robustness of training the global model in federated learning, enabling the training of personalized models for multiple nodes; and it ensures the transparency of incentives through blockchain technology, thereby increasing the enthusiasm of participants.

Description

Technical Field

[0001] This invention relates to the field of federated learning in privacy computing, and more particularly to a data sharing method and apparatus based on blockchain and federated learning. Background Technology

[0002] To design a machine learning framework that enables AI systems to more efficiently and accurately share their data while meeting data privacy, security, and regulatory requirements, the industry has proposed federated learning technology. Federated learning refers to multiple clients collaborating to solve machine learning problems under the coordination of a central server. During this process, each client's raw data is kept locally and not exchanged or transmitted. This technical solution maximizes data privacy and security because participants do not need to share data but only transmit local model parameters. Furthermore, training models collaboratively by multiple parties improves model performance compared to training models using local datasets alone.

[0003] However, in real-world scenarios, data distribution often differs. Whether it's the distribution of data features, labels, or the amount of data, these can vary across different scenarios. In other words, the data from each party is typically neither independent nor identically distributed. Existing federated learning suffers from the problem of model uniqueness. All participants train their local models based on the same global model, and the final aggregated global model is the same. Therefore, using a single model for all participants does not achieve optimal performance.

[0004] The common topology of federated learning is centralized, which places enormous pressure on network bandwidth and communication load in practical applications. Furthermore, centralized topologies are not robust; attacks on the central node, power outages, or network disruptions can severely impact the entire federated learning process. Using a blockchain network to replace the central server can solve this problem. The consensus algorithm and tamper-proof characteristics of blockchain itself can also improve the security of federated learning to a certain extent. At the same time, blockchain technology can ensure the transparency of incentives, increasing the enthusiasm of participants. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a data sharing method and apparatus based on blockchain and federated learning.

[0006] The objective of this invention is achieved through the following technical solution: a data sharing method based on blockchain and federated learning, comprising the following steps:

[0007] S1. The task initiator node in the federated learning network initiates a federated learning task to the blockchain network. The information of the federated learning task includes: task type, relevant data types used for training, initial global model, and number of training rounds.

[0008] S2. The blockchain network broadcasts federated learning task information to the federated learning network to find nodes that can participate in this task;

[0009] S3. Confirm that the federated nodes participating in this task will feed back the relevant data volume of the local federated learning task to the blockchain network;

[0010] S4. The blockchain network uploads the initial global model to the InterPlanetary File System (IPFS) and sends the corresponding hash address to the federated learning nodes participating in the task.

[0011] S5. The federated learning nodes participating in this task download the initial global model. Each node iteratively updates the parameters of the initial global model based on the local data related to the task type to obtain a local model.

[0012] S6. After local updates, upload the local model to IPFS and send the corresponding hash address to the blockchain network;

[0013] S7. The blockchain network executes corresponding smart contracts according to the task type, and evaluates, aggregates, updates reputation values, and distributes rewards for the local models of each node.

[0014] S8. Nodes on the blockchain network execute the consensus mechanism and write verified transactions into blocks;

[0015] S9. Repeat S4-S8 according to the number of training rounds; obtain the final global model that integrates the local data of each node, and upload it to IPFS;

[0016] S10. The blockchain network sends the hash address corresponding to the final global model to the task initiator.

[0017] Furthermore, in step 1, the task types include the following two types: single-node request to train a global model or multi-node collaborative training of a personalized model;

[0018] The cost paid by a single node to request training of the global model includes: the cost of executing smart contracts on the blockchain network and the block reward for consensus nodes, as well as the training cost of federated learning nodes;

[0019] The multi-node collaborative training of personalized models involves these nodes jointly paying for blockchain network fees, including smart contract execution fees and consensus node block production rewards.

[0020] Furthermore, in step 4, the model of interaction between the blockchain network and the federated learning network is stored in the InterPlanetary File System (IPFS), and only the corresponding hash address is transmitted, which reduces the pressure on communication and reduces latency.

[0021] Furthermore, in step 5, if the training task type is a single-node request to train a global model, a robust global model is trained. Since the data between federated learning client nodes are not independent and identically distributed, adding a proximate term constrains the update of the local model, eliminating client drift during training and making the loss function of the global model converge more stably. Therefore, the function to be optimized by task participant k during local iteration is:

[0022]

[0023] Let D be the amount of data on the local machine of each task participant. k The data is represented in the form of (x i ,y i ), x i Let y be a feature of the i-th data point. i w is the label for the i-th data item; t-1 The global model is obtained by aggregation during round t-1 of global communication. This represents the intermediate value of the local model during update iterations. Let L(*) be the local model obtained after node k updates locally during round t of global communication, where L(*) is the loss function and μ is the coefficient of the proximal term.

[0024] Furthermore, in step 5, if the training task type is multi-node collaborative training of a personalized model, then a personalized model is trained for each node participating in the task; therefore, the function to be optimized by task participant k during local iteration is:

[0025] Local model update function:

[0026]

[0027] Among them, the amount of data locally on task participant k is D. k The data is represented in the form of (x i ,y i ), x i Let y be a feature of the i-th data point. i Let i be the label of the i-th data; This represents the intermediate value of the local model during update iterations. Let L(*) be the local model obtained by node k after local update during round t of global communication, and L(*) be the loss function.

[0028] Personalized model update function:

[0029]

[0030] Among them, the amount of data locally on task participant k is D. k The data is represented in the form of (x i ,y i ), x i Let y be a feature of the i-th data point. i Let i be the label of the i-th data; This represents the intermediate value of the local model during update iterations. Let L(*) be the personalized model obtained by node k after local update during t rounds of global communication, and let m be the loss function. t-1 The global model is obtained by aggregation during the t-1 round of global communication; 'a' is a dynamic parameter that is adjusted according to the needs of each participant, and finally each federated learning node obtains a personalized model.

[0031] Furthermore, in step 7, the blockchain network executes a smart contract to evaluate the quality of the local model, considering only the case where the task type is a single node requesting to train the global model; if multiple nodes are collaboratively training a personalized model, then there is no need to pay for the training fees of the federated learning nodes, i.e., no quality evaluation is required.

[0032] If the task initiator provides a partial test dataset, the local models of each federated learning node are first evaluated using the test dataset to obtain evaluation scores. The evaluation metric is the mean absolute error, which is the ratio of the number of incorrect predictions made by the local model on the test dataset to the total number of predictions in the dataset. An evaluation score is then calculated based on the similarity between the local model and the previous global model. Add a dynamic term for adjustment, where γ is a hyperparameter and its value is set according to the specific situation;

[0033]

[0034]

[0035] Where D test To determine the size of the test dataset, the data is represented as (x i ,y i ), x i Let y be a feature of the i-th data point. i Let i be the label of the i-th data; Let w be the local model obtained by node k after local updates during round t of global communication. t-1 The global model is obtained by aggregation in round t-1; t is the current round of global communication, and T is the total number of rounds of global communication for this task.

[0036] The overall assessment score is:

[0037]

[0038] If the task initiator does not provide a test dataset, the training process will be divided into two stages: a pre-training stage and a post-training stage. In the pre-training stage, the blockchain network calculates an evaluation score based on the similarity between the local model and the previous round's global model. At this point, the overall evaluation score After a certain number of rounds of global iteration and stabilization, the network enters the later training stage. Based on the prediction models of each federated node, the blockchain network uses knowledge distillation to generate a pseudo-test dataset that conforms to the global data distribution. The federated learning nodes are then evaluated on this dataset, and a comprehensive evaluation score is used. β is the assessment score and Weight aggregation parameters;

[0039] Assuming there are n federated learning nodes participating in the task, the combined evaluation score (Eva) of these nodes is calculated. k Perform the conversion:

[0040]

[0041] Furthermore, in step 7, the blockchain network executes a smart contract to aggregate local models. If the task type is a single node requesting training of a global model, the comprehensive evaluation score Eva will be used. k The weights of the local model are aggregated:

[0042]

[0043] Among them, Eva k The comprehensive evaluation score for node k is... Let w be the local model obtained by node k after local updates during round t of global communication. t This is the global model obtained by aggregation during round t of global communication;

[0044] If the task type involves multi-node collaborative training of a personalized model, then weighted aggregation is performed based on the amount of data:

[0045]

[0046] Where D k The amount of local data for the task participants. This represents the total amount of data for this task.

[0047] Let m be the local model obtained by node k after local updates during round t of global communication. t This is the global model obtained by aggregation during round t of global communication.

[0048] Furthermore, in step 7, the blockchain network executes a smart contract to update the reputation value and distribute rewards to the federated learning nodes participating in the task, only considering the case where the task type is a single node requesting to train a global model; the blockchain network updates the reputation value based on the comprehensive evaluation score, and then distributes rewards based on the reputation value.

[0049]

[0050] in Let $\frac{k}{t}$ be the reputation value obtained by node $k after updating during round $t$ of global communication. Eva represents the reputation value of node k during round t-1 of global communication. k REP assigns a fixed value to the reputation value set for the blockchain network, which is the overall evaluation score for node k.

[0051]

[0052] in INC represents the reputation value obtained by node k after updating during round t of global communication. t This is the reward used for allocation during round t of global communication. This represents the reward allocated to node k during round t of global communication.

[0053] Secondly, the present invention also provides a data sharing device based on blockchain and federated learning, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it implements the aforementioned data sharing method based on blockchain and federated learning.

[0054] Thirdly, the present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the aforementioned data sharing method based on blockchain and federated learning.

[0055] The beneficial effects of this invention are as follows: Depending on the type of federated learning task, this invention trains a robust global model for a single node or a personalized model for multiple nodes, solving the problem of non-independent and identically distributed data among parties in federated learning; it utilizes a blockchain network to replace a central server to solve the problem of high communication load in centralized topology networks and improves the robustness of the federated learning network topology; it ensures the openness and transparency of incentives through blockchain technology, increasing the enthusiasm of participants; and it enhances the security of federated learning by utilizing the consensus algorithm and tamper-proof characteristics of blockchain itself. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 The present invention provides a flowchart of a data sharing method based on blockchain and federated learning.

[0058] Figure 2 A structural diagram of a data sharing device based on blockchain and federated learning provided by this invention. Detailed Implementation

[0059] The invention will be further illustrated below with reference to examples and accompanying drawings:

[0060] like Figure 1 As shown, one embodiment of the present invention is applied in a banking scenario. A bank wants to use machine learning technology to build an anti-money laundering model, but the amount of data within the bank is insufficient to train a usable model. Therefore, the bank needs to collaborate with other banks to model the model. Based on this scenario, it is assumed that there are 5 banks in the federated learning network, represented by P1, P2, P3, P4, and P5, and 5 blockchain nodes in the blockchain network, represented by B1, B2, B3, B4, and B5. Nodes in both networks are registered in the InterPlanetary File System (IPFS), and simultaneously, nodes on the federated learning network are registered and initialized in the blockchain network.

[0061] In the federated learning network, Bank P5, which has the aforementioned scenario requirements, initiates a task to the blockchain network to jointly build an anti-money laundering model with other banks in the network. Task information includes: task type, the data used in this task is anti-money laundering related data, the logistic regression model used, and the number of training rounds. The anti-money laundering data is represented as (x... i ,y i ), x i This represents user characteristic data (deposits, loans, and other credit data), y i The data is labeled and used to determine whether each user's transaction behavior is suspicious. The anti-money laundering model established in this task is a binary classification task. The logistic regression model used can determine whether a user's behavior is money laundering based on the input user feature information.

[0062] The task type may be one of the following two: (1) Bank P5 requests to train an anti-money laundering model for itself, which is a single node request to train a global model. (2) Bank P5 sends task information to find other banks that also have this need, which is a multi-node collaborative training of a personalized model. According to the task type, the fees paid by a single node requesting to train a global model include: the cost of executing smart contracts on the blockchain network and the block reward for consensus nodes, and the training cost of federated learning nodes; the costs paid by multiple nodes collaboratively training a personalized model include the cost of executing smart contracts and the block reward for consensus nodes.

[0063] The blockchain network broadcasts its learning network to the federation, seeking banks that can participate in jointly building an anti-money laundering model. Banks confirmed to participate will contribute their local anti-money laundering related data (D). k Feedback is sent to the blockchain network. In this example, it is assumed that bank nodes P1, P2, P3, and P4 are federated learning nodes that can participate in this task.

[0064] The blockchain network uploads the global logistic regression model to the InterPlanetary File System (IPFS) and sends the corresponding hash address to participating banks P1, P2, P3, P4, and P5. The model interacting between the blockchain network and the federated learning network is stored in the IPFS interplanetary file system; only the corresponding hash address is transmitted, which reduces communication load and latency.

[0065] Banks P1, P2, P3, P4, and P5 download the global model based on the hash address. Each bank chooses a strategy for iterative updates locally based on the training task type. If the training task type is a single-node request to train the global model, a robust global model is trained. Since the data among the banks is not independent and identically distributed, to eliminate client drift during training, a proximate term is added to constrain the updates of the local model, making the loss function of the global model converge more stably. Therefore, the function to be optimized by participant k in the joint anti-money laundering model task during local iteration is:

[0066]

[0067] Let D be the amount of data on the local machine of each task participant. k The data is represented in the form of (x i ,y i ), x i Let y be a feature of the i-th data point. i Let w be the label for the i-th data element. t-1 The global model is obtained by aggregation during round t-1 of global communication. This represents the intermediate value of the local model during update iterations. Let L(*) be the local model obtained after node k updates locally during round t of global communication, where L(*) is the loss function and μ is the coefficient of the proximal term.

[0068] If the training task type is multi-node collaborative training of a personalized model, where multiple federated client nodes collaboratively train the personalized model, then the federated learning client trains the personalized model for each node participating in the joint construction of the anti-money laundering model task. Therefore, the function to be optimized by task participant k during local iteration is:

[0069] Local model update function:

[0070]

[0071] Among them, the amount of data locally on task participant k is D. k The data is represented in the form of (x i ,y i ), x i Let y be a feature of the i-th data point. i Let be the label of the i-th data. This represents the intermediate value of the local model during update iterations. Let L(*) be the local model obtained by node k after local update during round t of global communication, and let L(*) be the loss function.

[0072] Personalized model update function:

[0073]

[0074] Among them, the amount of data locally on task participant k is D. k The data is represented in the form of (x i ,y i ), x i Let y be a feature of the i-th data point. i Let be the label of the i-th data. This represents the intermediate value of the local model during update iterations. Let L(*) be the personalized model obtained by node k after local update during t rounds of global communication, and let m be the loss function. t-1 This is the global model aggregated during round t-1 of global communication. 'a' is a dynamic parameter, and its value is adjusted according to the needs of each bank, ultimately resulting in a personalized model for each bank.

[0075] After local updates, the local model is uploaded to IPFS, and the corresponding hash address is sent to the blockchain network. The blockchain network executes the corresponding smart contracts based on the task type to evaluate, aggregate, update reputation values, and distribute rewards for each bank's local model.

[0076] The blockchain network's quality assessment of local models based on smart contracts only considers the case where the task type is a single node requesting to train the global model. If multiple nodes collaboratively train a personalized model, there is no need to pay training fees to each bank, meaning no quality assessment is required.

[0077] If the task initiating bank P5 provides a partial test dataset, then the local models of each bank are first evaluated using the test dataset to obtain evaluation scores. The evaluation metric is the mean absolute error, which is the ratio of the number of near-errors in predictions made by the local model on the test dataset to the total number of predictions in the dataset. An evaluation score is then calculated based on the similarity between the local model and the previous global model. Too high a similarity between models is detrimental to model training, while too low a similarity prevents stable convergence. Therefore, a dynamic term is added for adjustment, where γ is a hyperparameter that can be set according to specific circumstances.

[0078]

[0079]

[0080] Where D test To determine the size of the test dataset, the data is represented as (x i ,y i ), x i Let y be a feature of the i-th data point. i Let be the label of the i-th data. Let w be the local model obtained by node k after local updates during round t of global communication. t-1 This is the global model obtained from round t-1 aggregation. t is the current round of global communication, and T is the total number of rounds of global communication for this task.

[0081] The overall assessment score is:

[0082] If the task initiating bank P5 does not provide a test dataset, the training process will be divided into two stages: a pre-training stage and a post-training stage. In the pre-training stage, the blockchain network calculates an evaluation score based on the similarity between the local model and the previous round's global model. At this point, the overall evaluation score After several rounds of global iteration and stabilization, the network enters the later training phase. Based on each bank's prediction model, the blockchain network uses knowledge distillation to generate a pseudo-test dataset that conforms to the global data distribution. Then, the local models of each bank are evaluated on this dataset. A comprehensive evaluation score is then calculated. β is the assessment score and Weighted aggregation parameters.

[0083] Five federated learning nodes participated in the joint task of building an anti-money laundering model, and the comprehensive evaluation scores of these banks were converted as follows:

[0084]

[0085] The blockchain network executes smart contracts to aggregate local models. If the task type is a single node requesting training of a global model, the overall evaluation score Eva will be used. k The weights of the local model are aggregated:

[0086]

[0087] Among them, Eva k The comprehensive evaluation score for node k is... Let w be the local model obtained by node k after local updates during round t of global communication. t This is the global model obtained by aggregation during round t of global communication.

[0088] If the task type involves multi-node collaborative training of a personalized model, then weighted aggregation is performed based on the amount of data:

[0089]

[0090] Where D k The amount of local data for the task participants. This represents the total amount of data for this task. Let m be the local model obtained by node k after local updates during round t of global communication. t This is the global model obtained by aggregation during round t of global communication.

[0091] The blockchain network executes smart contracts to update the reputation values ​​and distribute rewards to participating banks, considering only the case where the task type is a single node request to train a global model. The blockchain network updates the reputation value based on a comprehensive evaluation score, and then distributes rewards based on the reputation value.

[0092]

[0093] in Let $\frac{k}{t}$ be the reputation value obtained by node $k after updating during round $t$ of global communication. Eva represents the reputation value of node k during round t-1 of global communication. k REP assigns a fixed value to the reputation value set for the blockchain network, which is the overall evaluation score for node k.

[0094]

[0095] in INC represents the reputation value obtained by node k after updating during round t of global communication.t This is the reward used for allocation during round t of global communication. This represents the reward allocated to node k during round t of global communication.

[0096] Nodes on the blockchain network execute a consensus mechanism, writing verified transactions into blocks. The blockchain network uses Fabric, with functions to be executed by the network written into smart contracts. Model storage utilizes the IPFS interplanetary file system. The smart contracts include functions such as local model evaluation, global model aggregation, reputation value updates, reward distribution, and pseudo-data generation based on knowledge distillation. The blockchain network's consensus mechanism is a reputation-based Proof-of-Stake (PoS) consensus mechanism. Assuming that in a certain round of consensus, three nodes (B1, B2, and B3) are randomly selected from five blockchain nodes, their reputation values ​​are compared. If B1 has the highest reputation value, then B1 becomes the accounting node and receives the block reward. The remaining nodes B2, B3, B4, and B5 perform verification.

[0097] The above training steps are repeated according to the set number of training rounds. The blockchain network uploads the finally trained global model to IPFS and sends the corresponding hash address to the task initiator bank P5.

[0098] Corresponding to the aforementioned embodiment of a data sharing method based on blockchain and federated learning, the present invention also provides an embodiment of a data sharing device based on blockchain and federated learning.

[0099] See Figure 2 This invention provides a data sharing device based on blockchain and federated learning, comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements a data sharing method based on blockchain and federated learning as described in the above embodiments.

[0100] The present invention provides an embodiment of a data sharing device based on blockchain and federated learning, which can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 2 The diagram shown is a hardware structure diagram of any device with data processing capabilities, which is a data sharing device based on blockchain and federated learning provided by the present invention. (Except for...) Figure 2In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0101] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0102] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0103] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a data sharing method based on blockchain and federated learning as described in the above embodiments.

[0104] The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0105] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A data sharing method based on blockchain and federated learning, characterized in that, The steps are as follows: S1. In the federated learning network, the task initiator node initiates a federated learning task to the blockchain network. The federated learning task information includes: task type, relevant data types used for training, initial global model, and number of training rounds. The task type includes the following two types: single node requesting training of the global model or multi-node collaborative training of a personalized model. S2. The blockchain network broadcasts federated learning task information to the federated learning network to find nodes that can participate in this task; S3. Confirm that the federated nodes participating in this task will feed back the relevant data volume of the local federated learning task to the blockchain network; S4. The blockchain network uploads the initial global model to the InterPlanetary File System (IPFS) and sends the corresponding hash address to the federated learning nodes participating in the task. S5. The federated learning nodes participating in this task download the initial global model. Each node iteratively updates the parameters of the initial global model based on the local data related to the task type to obtain a local model. If the training task type is a single-node request to train a global model, a robust global model will be trained. Therefore, the function to be optimized by task participant k during local iteration is: Let the local data volume of task participant k be . The data is represented in the form of ( ), For the features of the i-th data, Let i be the label of the i-th data; The global model is obtained by aggregation during round t-1 of global communication. This represents the intermediate value of the local model during update iterations. Let L(*) be the local model obtained by node k after local updates during round t of global communication, and let L(*) be the loss function. These are the coefficients of the proximal terms; If the training task type is multi-node collaborative training of a personalized model, then a personalized model is trained for each participating node; therefore, the function to be optimized by task participant k during local iteration is: Local model update function: Among them, the amount of data locally on task participant k is The data is represented in the form of ( ), For the features of the i-th data, Let i be the label of the i-th data; This represents the intermediate value of the local model during update iterations. Let L(*) be the local model obtained by node k after local update during round t of global communication, and L(*) be the loss function. Personalized model update function: Among them, the amount of data locally on task participant k is The data is represented in the form of ( ), For the features of the i-th data, Let i be the label of the i-th data; This represents the intermediate value of the local model during update iterations. Let L(*) be the personalized model obtained by node k after local updates during round t of global communication, and let L(*) be the loss function. This is the global model obtained by aggregation during round t-1 of global communication; These are dynamic parameters, adjusted according to the needs of each participant. The value is adjusted so that each federated learning node obtains a personalized model. S6. After local updates, upload the local model to IPFS and send the corresponding hash address to the blockchain network; S7. The blockchain network executes corresponding smart contracts according to the task type, and evaluates, aggregates, updates reputation values, and distributes rewards for the local models of each node. S8. Nodes on the blockchain network execute the consensus mechanism and write verified transactions into blocks; S9. Repeat S4-S8 according to the number of training rounds; obtain the final global model that integrates the local data of each node, and upload it to IPFS; S10. The blockchain network sends the hash address corresponding to the final global model to the task initiator.

2. The data sharing method based on blockchain and federated learning according to claim 1, characterized in that: In step 1, the fees paid by a single node for requesting to train the global model include: the blockchain network's smart contract execution fees and consensus node block production rewards, as well as the training fees of federated learning nodes; the blockchain network fees, including smart contract execution fees and consensus node block production rewards, are jointly paid by these nodes for collaborative training of personalized models.

3. The data sharing method based on blockchain and federated learning according to claim 1, characterized in that: In step 4, the model of interaction between the blockchain network and the federated learning network is stored in the InterPlanetary File System (IPFS), and only the corresponding hash address is transmitted to reduce communication pressure and latency.

4. The data sharing method based on blockchain and federated learning according to claim 2, characterized in that: In step 7, the blockchain network executes a smart contract to evaluate the quality of the local model, which only considers the case where the task type is a single node requesting to train the global model; if it is a multi-node collaborative training of a personalized model, there is no need to pay the training fee for the federated learning node, that is, no quality evaluation is required. If the task initiator provides a partial test dataset, the local models of each federated learning node are first evaluated using the test dataset to obtain evaluation scores. The evaluation metric is the mean absolute error, which is the ratio of the number of near-errors in predictions made by the local model on the test dataset to the total number of predictions in the dataset. Then, an evaluation score is calculated based on the similarity between the local model and the previous global model. Add dynamic items for adjustment, among which These are hyperparameters; set their values ​​according to the specific circumstances. in The data size of the test dataset is represented in the following format: ( ), For the features of the i-th data, Let i be the label of the i-th data; Let be the local model obtained by node k after local updates during round t of global communication. The global model is obtained by aggregation in round t-1; t is the current round of global communication, and T is the total number of rounds of global communication for this task. The overall assessment score is: If the task initiator does not provide a test dataset, the training process will be divided into two stages: the pre-training stage and the post-training stage. In the early stages, the blockchain network calculates an evaluation score based on the similarity between the local model and the previous global model. At this point, the comprehensive evaluation score After a certain number of rounds of global iteration and stabilization, the training enters the later stage. The blockchain network uses knowledge distillation to generate a pseudo-test dataset that conforms to the global data distribution based on the prediction models of each federated node. The federated learning nodes are then evaluated on this dataset, and a comprehensive evaluation score is used. , To assess scores and Weight aggregation parameters; Assuming there are n federated learning nodes participating in the task, calculate the combined evaluation score of these nodes. Perform the conversion:

5. A data sharing method based on blockchain and federated learning according to claim 2, characterized in that: In step 7, the blockchain network executes a smart contract to aggregate local models. If the task type is a single node request to train a global model, a comprehensive evaluation score will be used. The weights of the local model are aggregated: in, The comprehensive evaluation score for node k is... Let be the local model obtained by node k after local updates during round t of global communication. The global model is obtained by aggregation during round t of global communication, and n is the number of federated learning nodes participating in the task; If the task type involves multi-node collaborative training of a personalized model, then weighted aggregation is performed based on the amount of data: Where D k The amount of local data for the task participants. , which represents the total amount of data for this task; Let be the local model obtained by node k after local updates during round t of global communication. This is the global model obtained by aggregation during round t of global communication.

6. The data sharing method based on blockchain and federated learning according to claim 4, characterized in that: In step 7, the blockchain network executes a smart contract to update the reputation value and distribute rewards to the federated learning nodes participating in the task, only considering the case where the task type is a single node requesting to train a global model; the blockchain network updates the reputation value based on the comprehensive evaluation score, and then distributes rewards based on the reputation value. in Let $\frac{k}{t}$ be the reputation value obtained by node $k after updating during round $t$ of global communication. Let $t_1$ be the reputation value of node $k$ during the (t-1)th round of global communication. The comprehensive evaluation score for node k is... Assign a fixed value to the reputation value set for the blockchain network; in Let $\frac{k}{t}$ be the reputation value obtained by node $k after updating during round $t$ of global communication. This is the reward used for allocation during round t of global communication. This represents the reward allocated to node k during round t of global communication.

7. A data sharing device based on blockchain and federated learning, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it implements a data sharing method based on blockchain and federated learning as described in any one of claims 1-6.

8. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements a data sharing method based on blockchain and federated learning as described in any one of claims 1-6.

Images