Federal learning method and federal learning system based on double-chain block chain, computer system and storage medium

Abstract

The invention relates to the technical field of federated learning, in particular to a federated learning method and system based on a double-chain block chain, a computer system and a storage medium. On one hand, a block chain architecture based on a main chain and an auxiliary chain is designed, through separation verification and data storage, the verification efficiency of the block chain on global model updating and reputation value updating results is effectively improved, and the total overhead of transaction storage is optimized; and on the other hand, a batch of participants with excellent performance and honesty are selected based on a participant selection and management mechanism of real-time available computing resources and training behavior reputation, so that the possibility that low-performance equipment prolongs the training time and has malicious and negative participants is effectively reduced. The invention aims to solve the problem of how to improve the training efficiency of federal learning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning, in particular to a federated learning method based on a double-chain block chain, a federated learning system, a computer system and a storage medium. BACKGROUND

[0002] Artificial Intelligence (AI) and Industrial Internet of Things (IIoT) are currently a research hotspot. Developing AI products for IIoT usually requires large-scale data to train the performance of Machine Learning (ML) models, but in traditional ML mode, each Internet of Things device needs to share the collected raw data to a central server for centralized training, but these raw data often contain a large amount of user privacy information such as geographic location, which is prone to privacy data leakage problems.

[0003] To solve the above problems, Federated Learning (FL) emerges as a distributed ML framework, which does not need to collect raw training data to a central server, and the data owner only needs to upload model parameters after training the model locally, but in the classic FL paradigm, a centralized parameter server is needed, and an unreliable parameter server will still cause data privacy leakage, therefore, related research has adopted the decentralized characteristics of blockchain technology to solve the reliability problem of the central parameter server in traditional FL - in the related technical solution, the Chinese patent with application number 202211413352.0 discloses an asynchronous federated learning architecture system and method based on a block chain, on the one hand, through the block chain module, the central server of the traditional federated learning architecture system is removed, avoiding the trust problem brought by the central server; on the other hand, asynchronous federated learning is adopted, which greatly improves the flexibility and scalability of the whole, to a certain extent, reduces network congestion.

[0004] But this technical solution focuses on improving the security and privacy of federated learning, ignoring the training efficiency of the model in federated learning: the computing performance of different IIoT devices is quite different, but IIoT devices often need to perform multiple tasks including FL, so the real-time available computing resources of the device will change due to the influence of other tasks, thereby affecting the overall training efficiency of FL.

[0005] In view of this, the present application proposes a new architecture of federated learning based on a block chain, aiming to improve the training efficiency of federated learning. SUMMARY

[0006] The main purpose of the present application is to provide a double-chain blockchain-based federated learning method, aiming to solve the problem of how to improve the training efficiency of federated learning.

[0007] To achieve the above-mentioned purpose, the present application provides a double-chain blockchain-based federated learning method, applied to a federated learning system including a double-chain blockchain and multiple nodes, wherein the double-chain blockchain includes a main chain for saving federated learning training data, and a side chain for recording the behavior data of each node in each round of training, and the method comprises the following steps:

[0008] S10, initializing the local model and the reputation value of the initial node;

[0009] S20, estimating the estimated training duration of each initial node, and uploading the estimated training duration to the chain, and selecting target nodes from each initial node according to the reputation value and the estimated training duration by the smart contract in the double-chain blockchain;

[0010] S30, performing a federated learning task action on the local model by the target node, obtaining the current reputation value of the target node after performing the training action, selecting the target node with the highest current reputation value as the leader node, and packaging the updated global model and reputation value by the leader node to generate a new block, and broadcasting the new block to other target nodes for verification;

[0011] S40, if the number of verification passes of the other target nodes is greater than a preset threshold, uploading the new block to the main chain.

[0012] Optionally, in the S20, the step of selecting target nodes from each initial node according to the reputation value and the estimated training duration comprises:

[0013] eliminating the initial node with a reputation value less than a preset reputation threshold;

[0014] sorting the remaining initial nodes after elimination according to the estimated training duration from small to large, and selecting the first k remaining initial nodes as the target nodes.

[0015] Optionally, in the S30, the target node updates its own reputation value after performing the training action on the local model.

[0016] Optionally, the step of updating the reputation value of the target node itself comprises:

[0017] obtaining the actual training duration of the target node after performing the training action on the local model;

[0018] If the estimated training duration is greater than or equal to the actual training duration, the reputation value of the target node is divided by a preset decrease index to obtain a current reputation value of the target node;

[0019] If the estimated training duration is less than the actual training duration, and a sum of the reputation value and a preset reputation value increase step is less than a preset reputation value upper limit, the sum of the reputation value and the preset reputation value increase step is taken as the current reputation value of the target node.

[0020] If the estimated training duration is less than the actual training duration, and the sum of the reputation value and the preset reputation value increase step is greater than or equal to the preset reputation value upper limit, the preset reputation value upper limit is taken as the current reputation value of the target node.

[0021] Optionally, in the S30, the step of verifying the new block by the other target nodes comprises:

[0022] The other target nodes are requested to submit an updated local model update set through the side chain;

[0023] A model aggregation algorithm in a smart contract is executed according to the local model update set, and a local model aggregation calculation result is used to compare and verify the updated global model in the new block;

[0024] If the verification passes, the other target nodes are requested to submit an estimated training duration and an actual training duration through the side chain, a reputation value update algorithm in a smart contract is executed according to the estimated training duration and the actual training duration, and a local reputation value update calculation result is used to perform secondary verification on the reputation value in the new block;

[0025] If the secondary verification passes, it is determined that the verification of the new block passes.

[0026] Optionally, before the S30, the method further comprises:

[0027] S50, obtaining a current block size of each target node;

[0028] S60, adjusting a block size of the main chain and the side chain according to a total number of the target nodes and the current block size.

[0029] Optionally, the S60 comprises:

[0030] The total number of the target nodes and the current block size are input into a trained blockchain delay prediction model to obtain a minimum delay prediction value of the target nodes output by the blockchain delay prediction model;

[0031] if the current delay of the target node is greater than the minimum delay prediction value, determining the current block size as the optimal block size of the target node;

[0032] Otherwise, adjusting the block size of the target node so that the current delay of the target node is greater than the minimum delay prediction value.

[0033] In addition, to achieve the above object, the present application also provides a federated learning system, comprising:

[0034] a network layer comprising a task publisher for publishing a federated learning task and a node for executing the federated learning task;

[0035] a double-chain blockchain layer comprising a main chain for saving federated learning training data and a side chain for recording behavior data of each node at each round of training;

[0036] an application layer for executing the double-chain blockchain-based federated learning method according to any one of claims 1 to 7.

[0037] In addition, to achieve the above object, the present application also provides a computer system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the double-chain blockchain-based federated learning method according to any one of the above.

[0038] In addition, to achieve the above object, the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the double-chain blockchain-based federated learning method according to any one of the above.

[0039] The present application has at least the following beneficial effects:

[0040] 1. A blockchain architecture based on a main chain and a side chain is designed, which effectively improves the verification efficiency of the blockchain for global model update and reputation value update results and optimizes the total transaction storage overhead by separating verification and data storage;

[0041] 2. A participant selection and management mechanism based on real-time available computing resources and training behavior reputation is used to select a batch of participants with good performance and honesty, thereby effectively reducing the problems of low-performance devices prolonging the training time and the existence of malicious and negative participants;

[0042] 3. In order to balance the performance of the blockchain at different stages of FL, a deep learning algorithm is used to dynamically adjust the block size to reduce the average delay of the blockchain at each stage of FL. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 Architecture diagram of a double-chain blockchain involved in embodiments of the present application;

[0044] Figure 2 Flow diagram of a federated learning method based on a double-chain blockchain involved in embodiments of the present application;

[0045] Figure 3 Kan network structure diagram involved in embodiments of the present application;

[0046] Figure 4 Actual local training duration diagram of irregular changes involved in embodiments of the present application;

[0047] Figure 5 Actual local training duration diagram of honest clients involved in embodiments of the present application;

[0048] Figure 6 Actual training duration diagram of each round of introducing dishonest clients involved in embodiments of the present application;

[0049] Figure 7 Reputation value change diagram of a client involved in embodiments of the present application;

[0050] Figure 8 Storage overhead diagram of different nodes involved in embodiments of the present application;

[0051] Figure 9 Contrast test diagram of a single-main-chain architecture node involved in embodiments of the present application;

[0052] Figure 10 Contrast test diagram of a fixed block size involved in embodiments of the present application;

[0053] Figure 11 Architecture diagram of a hardware running environment of a computer system involved in embodiments of the present application;

[0054] Figure 12 Architecture diagram of a federated learning system involved in embodiments of the present application.

[0055] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0056] For a better understanding of the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0057] First embodiment

[0058] In order to ensure the equality of the probability of being selected by the client, the method of random selection is often used in the related literature (Yang Zhanpeng, Shi Yuanming, Zhou Yong, et al. Trustworthy federated learning via blockchain[J]. IEEE Internet of Things Journal, 2022, 10(1): 92-109.). However, in the environment of heterogeneous clients, the method of random selection will seriously reduce the training efficiency of federated learning and the performance of the global model, so the training efficiency of the model in the IIoT environment is still low.

[0059] First, the present embodiment provides a federated learning system including a double-chain blockchain and a plurality of nodes. Referring to Figure 1 , Figure 1 is a schematic diagram of the architecture of the double-chain blockchain, which includes a main chain for saving federated learning training data, and a side chain for recording the behavior data of each node at each round of training.

[0060] It is worth noting that the main chain and the side chain referred to in the present embodiment are in a master-slave relationship, that is, the side chain is an auxiliary chain serving the main chain. This asymmetric design is an innovative design in the present embodiment.

[0061] In a blockchain-based FL system, verifying a new block containing global model updates and reputation value updates is a time-consuming and resource-intensive process. Under the traditional single-chain architecture, the verification node needs to completely download and recalculate or verify all original local model updates (M_k) and original behavior data (ET_i, T_i) to confirm the correctness of the aggregation result (M') and reputation update (R') in the new block. This is inefficient when the model is large or the number of participants is large.

[0062] The double-chain solution proposed in this embodiment separates responsibilities: the main chain stores the calculation results (including but not limited to the final determined global model update M' and the set of client reputation value updates R'); and the side chain stores the original behavior data (including but not limited to the local model updates M_k submitted by the nodes, the estimated training duration ET_i and the actual training duration T_i). The model and reputation stored on the main chain are the results after consensus and are tamper-proof; the original behavior data on the side chain is also difficult to be tampered with by a single entity due to its distributed storage and consensus mechanism.

[0063] Therefore, the verification node does not need to re-download or process the huge original local model data, and only needs to access the necessary input in the lightweight side chain and then perform a relatively fast calculation verification, thereby reducing the calculation amount, network bandwidth and I / O overhead required for verification, and shortening the verification duration of a new block.

[0064] Further, with reference to Figure 2 The embodiment provides a federated learning method based on a double-chain blockchain, and the method comprises the following steps:

[0065] S10, initializing the local model and the reputation value of the initial node;

[0066] In this step, the task publisher broadcasts an FL task, which contains training requirements (target accuracy of the model, aggregation rules, etc.) and client requirements (local data volume, hardware resources, etc.). The nodes meeting the requirements become the initial nodes (also referred to as clients) of this FL task, and the local model and the reputation value of the initial nodes are initialized for subsequent steps,

[0067] S20, estimating the estimated training duration of each initial node, uploading the estimated training duration to the side chain, and screening target nodes from the initial nodes according to the reputation value and the estimated training duration by a smart contract in the double-chain blockchain;

[0068] In this step, the estimated training duration of each initial node is estimated.

[0069] In some optional embodiments, the estimation action can be performed by the initial node itself.

[0070] The obtained estimated training duration is uploaded to the side chain for recording and storage, and the estimated training duration and the reputation value of each initial node stored in the side chain are called by a smart contract in the double-chain blockchain, a participant selection algorithm is executed, and target nodes are selected from the initial nodes.

[0071] The participant selection mechanism is used to select a participant subset S from all client set C of the current FL task to participate in the current round of training. In this paper, a selection algorithm based on the current computing performance of the client is designed to make the selected participants have the shortest training duration in the current round.

[0072] Further, in some optional embodiments, the participant selection mechanism can include: eliminating the initial nodes with a reputation value less than a preset reputation threshold; and sorting the remaining initial nodes after elimination according to the estimated training duration from small to large, and selecting the top k remaining initial nodes as the target nodes.

[0073] For example, the participant (i.e., target node) subset S selected by the algorithm can be represented as:

[0074]

[0075] where k represents the number of specified participants in the current round (i.e., initial nodes), n represents the total number of all nodes in the current FL, and the function represents the maximum training duration in the node set.

[0076] However, there is a problem of the honesty of the nodes. Therefore, a reputation mechanism is designed to manage the election behavior of the nodes, and the participant subset S selected by the algorithm at this time is represented as:

[0077]

[0078] where the function represents the minimum reputation value in the client set, and r represents the specified reputation value threshold. The participants are selected based on the estimated duration and the reputation value to make the actual training duration in the current round the shortest, and the specific process is shown in Algorithm 1.

[0079] Let the target nodes k required in the current round, the initial nodes C participating in the election, the client estimated training duration set T, the client reputation value set R, the reputation threshold r, and the target node set of the current round of training be S:

[0080] (1) Initialize the participant set S;

[0081] (2) Traverse the clients in the set C, and obtain the reputation value of the current client from the set R. Delete the client with a reputation value less than the reputation threshold r from the set C;

[0082] (3) Rank the remaining clients in the set C according to their respective estimated training durations from small to large according to the estimated training duration in the set T;

[0083] (4) Cut the top k clients from the client ranking in the set C, and add them to the set S;

[0084] The reputation value of the client is updated according to the honesty of the client in each round of the election, and the reputation value of the client increases linearly when the client is honest, and the reputation value of the client decreases exponentially when the client is dishonest.

[0085] S30, performing a federated learning task action on the local model by the target node, obtaining a current reputation value of the target node after the target node performs the training action, selecting a target node with the highest current reputation value as a leader node, packaging the updated global model and the reputation value by the leader node to generate a new block, and broadcasting the new block to other target nodes for verification;

[0086] In this step, after selecting the target node as a participant in this round, the target node uses local data to perform the federated learning task action broadcast by the task publisher, and automatically broadcasts and records the obtained local training result in the side chain block, wherein the local training result includes an updated local model, an actual training duration and a current reputation value of the target node after updating.

[0087] Similarly, in this step, the current reputation value of the target node after performing the training action is obtained, and the target node with the highest current reputation value is selected as a leader node, and the leader node chains the updated global model and the reputation value to a newly generated new block and broadcasts it to other target nodes for verification.

[0088] Notably, in classical federated learning, a centralized parameter server is responsible for aggregating model updates, distributing global models and maintaining client state (such as reputation) reputation mechanisms can filter out malicious nodes, while in the framework of the present embodiment, the central server is abandoned, and the key responsibility is creatively assigned to the most trustworthy node selected in a decentralized manner, i.e. the leader node with the highest current reputation value, which can be considered as the most reliable node, ensuring that it will not tamper with the model or reputation data.

[0089] Further, considering the possibility of the leader trying to be malicious, the newly generated new block is verified by other target nodes, and if the malicious behavior is detected, the block will not be added to the main chain through consensus. In some optional embodiments, the step of verifying the new block by the other target nodes comprises:

[0090] requesting the other target nodes to submit an updated local model update set through the side chain; verifying the updated global model in the new block according to the local model update set; if the verification is passed, requesting the other target nodes to submit an estimated training duration and an actual training duration through the side chain, and performing a second verification on the reputation value in the new block according to the estimated training duration and the actual training duration, if the second verification is passed, it is determined that the verification of the new block is passed.

[0091] Exemplarily, let the global model update in the newly generated block be , the client reputation value update set be , and the verification result of the other target node j be :

[0092] ① Initialization ;

[0093] ② Request the local model submitted by the participant in this round of training from the side chain, and update the set ;

[0094] ③ Verify according to the updated local model, and directly output if the verification is wrong; ;

[0095] ④ Traverse the client reputation value , request the estimated training duration and the actual training duration of the client i from the side chain, verify according to and , and output if the verification is wrong; ;

[0096] ⑤ If and are both verified correctly, output the verification result ;

[0097] In addition, in some optional embodiments, about one-third of the total number of nodes are selected to store the main chain transaction data, and the remaining nodes store the side chain transaction data. In this design, the transaction storage of the double-chain architecture is essentially replaced by the side chain node to replace part of the main chain node, thereby effectively reducing the total overhead of all nodes in transaction storage,

[0098] S40, if the number of passed verifications of the other target node is greater than a preset threshold, the new block is chained to the main chain.

[0099] Finally, if the newly generated block passes the verification of more than a preset threshold of other target nodes, the block will be added to the main chain and accepted by the entire network.

[0100] In the technical scheme provided in the embodiment, on one hand, a blockchain architecture based on a main chain and a side chain is designed, and by separating verification and data storage, the verification efficiency of the blockchain for global model update and reputation value update results is effectively improved, and the total overhead of transaction storage is optimized; on the other hand, based on real-time available computing resources and a participant selection and management mechanism based on training behavior reputation, a batch of participants with good performance and honesty are selected, thereby effectively reducing the problems of low-performance devices prolonging training time and the existence of malicious and negative participants.

[0101] Second embodiment

[0102] Based on the first embodiment, in the embodiment, it is expected that the honesty of the target node (client) in each round of election is used to update the reputation value thereof, and when the target node is honest, the reputation value thereof increases linearly, and when the target node is dishonest, the reputation value thereof decreases exponentially, thereby ensuring that the node is reliable enough in each round of federated learning. Specifically, in S30, the target node adds a new action of updating the reputation value thereof after performing a training action on the local model.

[0103] Specifically, the step of updating the reputation value thereof includes:

[0104] S31, acquiring an actual training time length after the target node performs a training action on the local model;

[0105] S32, if the estimated training time length is greater than or equal to the actual training time length, dividing the reputation value of the target node by a preset decrease exponent to obtain a current reputation value of the target node;

[0106] S33, if the estimated training time length is less than the actual training time length, and a sum of the reputation value and a preset reputation value increase step is less than a preset reputation value upper limit, taking the sum of the reputation value and the preset reputation value increase step as the current reputation value of the target node;

[0107] S34, if the estimated training time length is less than the actual training time length, and the sum of the reputation value and the preset reputation value increase step is greater than or equal to the preset reputation value upper limit, taking the preset reputation value upper limit as the current reputation value of the target node.

[0108] Exemplarily, assuming that a current reputation value of the target node i is , an estimated training time length of the target node i is , an actual training time length is , a reputation value upper limit is MR, a reputation value increase step is t, and a decrease exponent is a, an updated reputation value of the target node i is :

[0109] (1) initialization ;

[0110] (2) If then

[0111] (3) ;

[0112] (4) else if then

[0113] (5) ;

[0114] (6) else

[0115] (7) ;

[0116] (8) end if

[0117] Third embodiment

[0118] Based on any of the above embodiments, in order to balance the performance of the blockchain in different stages of FL, a deep learning algorithm is used to dynamically adjust the block size to reduce the average delay of the blockchain in each stage of FL.

[0119] In this embodiment, the Kan network structure diagram as shown in Figure 3 is used to train the blockchain performance prediction model. Among them, the input layer is composed of two characteristics of transaction arrival rate and block size. These characteristics are mapped to the output layer after passing through multiple hidden layers, and the output layer represents the transaction delay of the blockchain.

[0120] When optimizing the parameters of the blockchain, the number of active clients in the next stage of FL is fed into the above trained KAN model together with different block sizes as the transaction arrival rate, and then the block size with the lowest predicted delay is used for adjustment. Specifically, before step S30, the following steps are further included:

[0121] S50, acquiring the current block size of each target node;

[0122] S60, adjusting the block size of the main chain and the secondary chain according to the total number of target nodes and the current block size.

[0123] Step S60 further includes:

[0124] S61, inputting the total number of target nodes and the current block size into the trained blockchain delay prediction model to obtain the minimum delay prediction value of the target node output by the blockchain delay prediction model;

[0125] S62, if the current delay of the target node is greater than the minimum delay prediction value, determining the current block size as the optimal block size of the target node;

[0126] S63, otherwise, adjusting the block size of the target node so that the current delay of the target node is greater than the minimum delay prediction value.

[0127] Exemplarily, assuming that the number of active target nodes in the next stage of the federation task is k, the selectable block size set , the trained blockchain delay prediction model is KAN, and the optimal block size :

[0128] (1) initialization , ;

[0129] (2) for in

[0130] (3) lat = KAN ( k, );

[0131] (4) if || then

[0132] (5) , ;

[0133] (6) end if

[0134] (7) end.

[0135] Fourth embodiment

[0136] As a verification embodiment, in this embodiment, the effectiveness of the participant selection algorithm based on the estimated training duration and the reputation mechanism, the performance of the main and auxiliary chain architecture in transaction storage optimization, and the performance of the block size optimization algorithm based on the DL method are evaluated.

[0137] In this embodiment, a workstation with a 64-bit Intel Core i7-8750H 2.20GHz CPU, a NVIDIA Geforce GTX 1050 GPU, 16 GB of memory and a Windows operating system is used for experiments. PyCharm is used as the integrated development environment, Python 3.8 is used as the programming language, and PyTorch 2.4.1 is used as the DL framework.

[0138] (1) Verification of participant selection mechanism

[0139] First, a standard convolutional neural network is used as the global model for the federated learning task, which has 6 convolutional layers (convolution kernel size of 3, channel numbers of 32, 32, 64, 64, 128, 128, each channel is activated by ReLU and batch normalized, and each two channels have a max pooling of 2), followed by 3 fully connected layers (the first two layers have 382 and 192 outputs activated by ReLU, and the last layer has 10 outputs activated by soft-max). The Fashion-MNIST dataset is used in this embodiment, which contains 60,000 training samples and 10,000 test samples.

[0140] Next, 20 clients are set up, and a total of 10 rounds of training are performed, with 10 participants selected for each round of training. Each client randomly selects a different size of data set as the training set for this round of training at each round of training, thereby obtaining the actual local training duration of the 20 clients (i.e., target nodes) irregularly changing in the 10 rounds of training, as shown in Figure 4 .

[0141] When all clients are honest, the clients use the actual local training duration to campaign for participants in this round. In this case, the actual training duration of each round is as shown in Figure 5 , compared with the excessive training duration of each round of the random selection algorithm, our participant selection algorithm can make the best selection of each round, so that the training duration of each round is the lowest.

[0142] In addition, 3 dishonest clients are set up, which will lie about the training duration to campaign to become participants in the round with a longer actual training duration. The initial reputation value of each client is set to 0.5, the reputation value growth step is set to 0.1, the reputation value decline exponent is set to 2, and the reputation value upper limit is set to 1. At this time, the actual training duration of each round is as shown in Figure 6 , and the reputation value change of the client is as shown in Figure 7 . The training duration of each round of our selection algorithm is close to that of the random selection algorithm without a reputation mechanism, but our selection algorithm can approach or achieve the best selection in most rounds with a reputation mechanism. As can be seen from the reputation value change of the client, the reputation of the honest client will steadily increase but not exceed the reputation value upper limit, and the reputation of the dishonest client will be greatly reduced in each dishonest training round, and the client will not be able to participate in training when the client reputation value is less than the reputation value threshold.

[0143] (2) Performance verification of the main and auxiliary chain architecture The storage file size of different transaction quantities on the node is used as the transaction storage overhead. The storage overhead of different nodes is as shown in Figure 8As shown, the storage overhead of the secondary chain node is only 50% or less of that of the primary chain node under different transaction quantities. Next, 20 clients are set as nodes of the single primary chain architecture, which is used as a control experiment, and the experimental results are as follows Figure 9 As shown, under different transaction quantities, the total storage overhead of the primary-secondary chain architecture designed by us is only about 66% of that of the single primary chain architecture.

[0144] (3) Performance verification of block size optimization algorithm

[0145] In this embodiment, the Blockchain Performance Dataset (BPD) is used to verify the performance of the block size optimization algorithm based on the DL method. The BPD contains four sub-datasets (BPD-1, BPD-2, BPD-3, BPD-4) with a total of 4320 samples, and each dataset contains four parameters: transaction arrival rate, block size, delay, and throughput.

[0146] KAN is used to train the blockchain performance prediction model. In the prediction task for blockchain transaction delay, two hidden layers are set (the output dimensions are 64 and 8 respectively), the Adam optimizer is used, the learning rate is set to 0.001, and the training data batch size is 32. MSE is used as the loss function for model training and the training rounds are set to 200 rounds. MAE (Mean Absolute Error), RMSE (Root Mean Square Error), and MAPE (Mean Absolute Percentage Error) are used as evaluation indicators, and the experimental results of 5-fold cross-validation are shown in Table 1:

[0147] Table 1. Delay prediction performance of different models on BPD

[0148]

[0149] Note: Wang's model is Wang Jishu, Zhu Chao, Miao Chen, et al. BPR: Blockchain-enabled efficient and secure parking reservation framework with block size dynamic adjustment method[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 24(3): 3555-3570.

[0150] It can be seen that the prediction error of our model is better than the existing work on the BPD dataset, which shows that the proposed blockchain performance prediction model can accurately predict the delay of the blockchain, thereby improving the effectiveness and feasibility of dynamically optimizing the block size.

[0151] Finally, 20 transaction arrival rates (10-200, step size 10) are set, and the trained delay prediction model is combined with the block size optimization algorithm to select the optimal block size for each transaction arrival rate. The actual blockchain delay corresponding to each pair of transaction arrival rate and optimized block size is obtained on the BPD dataset. Finally, four fixed block sizes (50, 100, 150, 200) are set as control experiments, and the experimental results are shown in Figure 10 As can be seen from the experimental results, under the same transaction arrival rate, the block size optimized by our algorithm can achieve the lowest delay.

[0152] As an implementation scheme, Figure 11 The figure shows the architecture of the hardware running environment of the computer system involved in the embodiment of the present application.

[0153] As Figure 11 shown, the computer system can include a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard). The optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as a magnetic disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0154] Those skilled in the art can understand that Figure 11 the computer system architecture shown in the figure does not constitute a limitation on the computer system, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0155] As Figure 11 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a computer program. The operating system is a program that manages and controls the hardware and software resources of the computer system, and the running of the computer program and other software or programs.

[0156] In Figure 11 In the computer system shown, the user interface 1003 is mainly used to connect the terminal and communicate data with the terminal; the network interface 1004 is mainly used to connect the background server and communicate data with the background server; and the processor 1001 can be used to call the computer program stored in the memory 1005.

[0157] In the embodiment, the computer system comprises a memory 1005, a processor 1001 and a computer program stored in the memory and executable on the processor, wherein:

[0158] When the processor 1001 calls the computer program stored in the memory 1005, the following operations are performed:

[0159] S10, initializing the local model and the reputation value of the initial node;

[0160] S20, estimating the estimated training duration of each initial node, and chaining the estimated training duration, wherein the smart contract in the double-chain blockchain selects target nodes from the initial nodes according to the reputation value and the estimated training duration;

[0161] S30, performing a federated learning task on the local model by the target node, obtaining the current reputation value of the target node after performing the training action, selecting the target node with the highest current reputation value as a leader node, and packaging the updated global model and reputation value by the leader node to generate a new block, and broadcasting the new block to other target nodes for verification;

[0162] S40, if the number of verification passes of the other target nodes is greater than a preset threshold, chaining the new block to the main chain.

[0163] When the processor 1001 calls the computer program stored in the memory 1005, the following operations are performed:

[0164] Eliminate the initial node with a reputation value less than a preset reputation threshold;

[0165] Sort the remaining initial nodes after elimination according to the estimated training duration from small to large, and select the first k remaining initial nodes as the target nodes.

[0166] When the processor 1001 calls the computer program stored in the memory 1005, the following operations are performed:

[0167] In the S30, the target node updates its reputation value after performing the training action on the local model.

[0168] The processor 1001 invokes the computer program stored in the memory 1005 to perform the following operations:

[0169] obtain an actual training duration after the target node performs a training action on the local model;

[0170] If the estimated training duration is greater than or equal to the actual training duration, divide the reputation value of the target node by a preset decrease index to obtain a current reputation value of the target node;

[0171] If the estimated training duration is less than the actual training duration, and a sum of the reputation value and a preset reputation value growth step is less than a preset reputation value upper limit, the sum of the reputation value and the preset reputation value growth step is taken as the current reputation value of the target node;

[0172] If the estimated training duration is less than the actual training duration, and the sum of the reputation value and the preset reputation value growth step is greater than or equal to the preset reputation value upper limit, the preset reputation value upper limit is taken as the current reputation value of the target node.

[0173] The processor 1001 invokes the computer program stored in the memory 1005 to perform the following operations:

[0174] request the other target nodes to submit the updated local model update set through the side chain;

[0175] perform a model aggregation algorithm in the smart contract according to the local model update set, and use a local model aggregation calculation result to perform comparison verification on the updated global model in the new block;

[0176] If the verification passes, request the other target nodes to submit the estimated training duration and the actual training duration through the side chain, perform a reputation value updating algorithm in the smart contract according to the estimated training duration and the actual training duration, and use a local reputation value updating calculation result to perform secondary verification on the reputation value in the new block;

[0177] If the secondary verification passes, it is determined that the verification of the new block passes.

[0178] The processor 1001 invokes the computer program stored in the memory 1005 to perform the following operations:

[0179] S50, obtain a current block size of each target node;

[0180] S60, adjust the block size of the main chain and the side chain according to the total number of target nodes and the current block size.

[0181] The processor 1001 executes the following operations when invoking a computer program stored in the memory 1005:

[0182] inputting the total number of the target nodes and the current block size into the trained blockchain delay prediction model to obtain a minimum delay prediction value of the target nodes output by the blockchain delay prediction model;

[0183] if the current delay of the target nodes is greater than the minimum delay prediction value, determining the current block size as the optimal block size of the target nodes;

[0184] otherwise, adjusting the block size of the target nodes so that the current delay of the target nodes is greater than the minimum delay prediction value.

[0185] In addition, with reference to Figure 12 the embodiment also provides a federated learning system, which comprises:

[0186] a network layer 100 comprising a task publisher for publishing a federated learning task and a node for executing the federated learning task;

[0187] a double-chain blockchain layer 200 comprising a main chain for saving federated learning training data and a side chain for recording behavior data of each node in each round of training;

[0188] an application layer 300 for executing the double-chain blockchain-based federated learning method as described above.

[0189] In addition, those skilled in the art can understand that all or part of the processes in the method for implementing the above embodiment can be completed by a computer program instructing related hardware. The computer program comprises program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the above-described embodiment.

[0190] Therefore, the application also provides a computer-readable storage medium storing a computer program, which is executed by a processor to implement each step of the double-chain blockchain-based federated learning method as described above.

[0191] The computer-readable storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, or various computer-readable storage media that can store program codes.

[0192] It should be noted that the storage medium provided by the embodiments of the present application is a storage medium used for implementing the method of the embodiments of the present application, and therefore, based on the method introduced in the embodiments of the present application, the specific structure and variations of the storage medium can be understood by those skilled in the art, and therefore, will not be described here. Any storage medium used by the method of the embodiments of the present application belongs to the scope of the present application.

[0193] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0195] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0196] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0197] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0198] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0199] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A federated learning method based on a dual-chain blockchain, characterized in that: The method is applied to a federated learning system including a dual-chain blockchain and multiple nodes, wherein the dual-chain blockchain includes a main chain for storing federated learning training data and a secondary chain for recording the behavior data of each node during each round of training. The method includes the following steps: S10, initialize the local model and reputation value of the initial node; S20, estimating an estimated training duration for each of the initial nodes, uploading the estimated training duration to the blockchain, and having the smart contract in the dual-chain blockchain select target nodes from each of the initial nodes based on the reputation value and the estimated training duration; S30, the target node performs a federated learning task action on the local model, obtains the current reputation value of the target node after performing the training action, selects the target node with the highest current reputation value as the leader node, and the leader node packages the updated global model and reputation value to generate a new block, and broadcasts the new block to other target nodes for verification; S40: If the number of verifications passed by the other target nodes is greater than a preset threshold, the new block is added to the main chain.

2. The federated learning method based on a dual-chain blockchain as claimed in claim 1, characterized in that: In S20, the step of selecting a target node from each of the initial nodes according to the reputation value and the estimated training duration includes: Eliminate the initial nodes whose reputation value is less than the preset reputation threshold; The remaining initial nodes after being eliminated are sorted from small to large according to the estimated training duration, and the first k remaining initial nodes are selected as the target nodes.

3. The federated learning method based on a dual-chain blockchain as claimed in claim 1, characterized in that: In S30, the target node updates its own reputation value after performing the training action on the local model.

4. The federated learning method based on a dual-chain blockchain as claimed in claim 3, characterized in that: The step of updating the reputation value of the user includes: Obtaining the actual training duration after the target node performs a training action on the local model; If the estimated training duration is greater than or equal to the actual training duration, dividing the reputation value of the target node by a preset decline index to obtain the current reputation value of the target node; If the estimated training duration is less than the actual training duration, and the sum of the reputation value and the preset reputation value growth step is less than the preset reputation value upper limit, the sum of the reputation value and the preset reputation value growth step is used as the current reputation value of the target node; If the estimated training duration is less than the actual training duration, and the sum of the reputation value and the preset reputation value growth step is greater than or equal to the preset reputation value upper limit, the preset reputation value upper limit is used as the current reputation value of the target node.

5. The federated learning method based on a dual-chain blockchain as claimed in claim 1, characterized in that: In S30, the step of verifying the new block by the other target nodes includes: Requesting the other target nodes to submit updated local model update sets through the secondary chain; Executing the model aggregation algorithm in the smart contract according to the local model update set, and using the local model aggregation calculation result to compare and verify the updated global model in the new block; If the verification is successful, the secondary chain requests the other target nodes to submit the estimated training duration and the actual training duration, executes the reputation value update algorithm in the smart contract based on the estimated training duration and the actual training duration, and uses the local reputation value update calculation result to perform a second verification on the reputation value in the new block; If the secondary verification is passed, it is determined that the verification of the new block is passed.

6. The federated learning method based on a dual-chain blockchain as claimed in claim 1, characterized in that: Before S30, the following steps are also included: S50, obtaining the current block size of each target node; S60: Adjust the block sizes of the main chain and the secondary chain according to the total number of the target nodes and the current block size.

7. The dual-chain blockchain-based federated learning method according to claim 6, wherein S60 includes: Input the total number of target nodes and the current block size into the trained blockchain delay prediction model, and obtain the minimum delay prediction value of the target node output by the blockchain delay prediction model; If the current delay of the target node is greater than the minimum delay prediction value, determining the current block size as the optimal block size of the target node; Otherwise, the block size of the target node is adjusted so that the current delay of the target node is greater than the minimum delay prediction value.

8. A federated learning system, characterized in that: The federated learning system includes: The network layer includes task publishers for publishing federated learning tasks and nodes for executing federated learning tasks; A dual-chain blockchain layer, including a main chain for storing federated learning training data and a secondary chain for recording the behavior data of each node during each round of training; The application layer is used to execute the federated learning method based on the dual-chain blockchain as described in any one of claims 1 to 7.

9. A computer system, characterized in that: The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the federated learning method based on a dual-chain blockchain as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the federated learning method based on a double-chain blockchain as described in any one of claims 1 to 7.

Images