A federated learning method, system and device based on double-layer blockchain

Abstract

The present invention provides a federated learning method, system and device based on a double-layer blockchain, which relates to the technical field of blockchain and the technical field of federated learning. The method mainly includes: constructing a double-layer blockchain in the system, dividing several sub-blockchain nodes and sub-blockchain key nodes into several federated learning groups through a classifier; sub-blockchain nodes perform local model training through a federated learning method based on local information; sub-blockchain key nodes collect training results in the sub-blockchain, and package the consensus training results into a new block and publish it to the sub-blockchain; main blockchain nodes perform global model training through a federated learning method; main blockchain key nodes collect training results in the main blockchain, and package the consensus training results into a new block and publish it to the main blockchain. This scheme can greatly improve the security of the federated learning framework, thereby obtaining an optimized learning model and accurate prediction data.

Description

Technical Field

[0001] The present invention relates to the fields of blockchain technology and federated learning technology, and in particular to a federated learning method, system and device based on a double-layer blockchain. Background Art

[0002] At present, machine learning is the main algorithm of embedded artificial intelligence systems and has gradually been promoted and applied in many fields, especially in the field of agricultural Internet of Things. Based on the information such as crop growth status, soil pH, temperature and precipitation detected by sensors, the machine learning system intelligently analyzes the above information and scientifically predicts the timing of sowing, irrigation, pest control, fertilization and harvesting, which will greatly promote the intelligent development and quality improvement of the agricultural field.

[0003] The prediction accuracy of a machine learning algorithm depends not only on the quality of the algorithm, but also on the size of the original data. In order to improve the prediction accuracy of a machine learning algorithm, it is often necessary to upload a large amount of information to a centralized computing center. However, this method not only increases the load of data transmission, but also makes it easy for privacy to be leaked. Federated learning, as a new collaborative machine learning framework, allows multiple participating nodes to use local data for model training and aggregate models through parameter servers, thereby achieving model prediction effects from multiple data sources.

[0004] However, in the federated learning framework, the gradient update process may still leak information about the participants' original training data. If the framework is directly introduced into practical applications, such as the agricultural Internet of Things, the basic data in the agricultural field will be at risk of being leaked. Moreover, the parameter server that the federated learning framework relies on for model aggregation is still centralized, which also increases the risk of data being maliciously tampered with and privacy leaked. In addition, the security risks brought about by the introduction of the federated learning framework into practical applications come not only from within the federated learning framework, but also from outside, such as model attacks, data attacks, reasoning attacks, backdoor attacks, connection attacks, and poisoning attacks. It can be seen that how to ensure privacy security has become a major problem that hinders the promotion and application of the federated learning framework. Summary of the invention

[0005] The purpose of the present invention is to provide a federated learning method, system and device based on a double-layer blockchain to solve at least one of the above-mentioned technical problems existing in the prior art.

[0006] In the first aspect, in order to solve the above technical problems, the present invention provides a federated learning method based on a double-layer blockchain, comprising the following steps:

[0007] Step 1: Build a blockchain in the system, including a sub-blockchain unit and a main blockchain unit:

[0008] The sub-blockchain unit includes a classifier, a sub-blockchain node and a sub-blockchain key node: the classifier divides a plurality of the sub-blockchain nodes and the sub-blockchain key nodes into a plurality of federated learning groups based on homogeneous data;

[0009] The main blockchain unit includes a main blockchain node and a main blockchain key node, and the main blockchain node is served by the sub-blockchain key node;

[0010] Step 2: The sub-blockchain nodes in the federated learning group perform local model training based on local information through a federated learning method;

[0011] Step 3: The key node of the sub-blockchain collects the training results in the sub-blockchain within a preset time interval, and packages the consensus-based training results into a new block and publishes it to the sub-blockchain;

[0012] Step 4: The main blockchain node in the federated learning group performs global model training through a federated learning method based on the training results in the sub-blockchain;

[0013] Step 5: The key nodes of the main blockchain collect the training results in the main blockchain within a preset time interval, and package the consensus training results in the training results into a new block and publish it to the main blockchain.

[0014] The blockchain belongs to the prior art and is a computer application model with the characteristics of distributed data storage, point-to-point data transmission, consensus mechanism, encryption algorithm, etc. The main body of the block is the packaged transaction, and the transaction is some actual data stored in the blockchain, such as payment data in commodity trading, detection data in agricultural Internet of Things, training results in machine learning, etc. The role of the block is consensus, that is, to record when and in what order the transaction becomes part of the blockchain. The blockchain can adopt a multi-layer structure. The blockchain at the lower level includes several nodes. The node can be a gateway named FLG. Among the several nodes, key nodes are selected to form a high-level blockchain, and so on. The node is used to collect the transaction and send it to nearby nodes, and at the same time achieve consensus between nodes. The key node is used to package the transaction into candidate blocks, and publish the new blocks mined in the candidate blocks based on existing rules to the blockchain at the layer.

[0015] In a possible embodiment, the system is an agricultural Internet of Things system.

[0016] Furthermore, the local information is agricultural information detected by sensors, such as crop growth status, soil pH, temperature and precipitation, etc.

[0017] In a possible embodiment, the sub-blockchain unit includes several private chains, and the main blockchain unit includes several alliance chains:

[0018] The private chain includes several sub-blockchain nodes, each of which is a gateway named FLG, which is used to perform federated learning training and send the training results to nearby FLGs, and achieve consensus among FLGs; a key FLG is selected from all FLGs of a private chain as the key node of this private chain, and the key node of the private chain is used to collect transactions in the private chain, package the transactions into candidate blocks, and publish the new blocks mined from the candidate blocks to the private chain;

[0019] The main blockchain unit builds a consortium chain through several key nodes of the private chain. In the consortium chain, the consortium chain nodes are used for federated learning training; several consortium chain nodes send and receive training results to each other, and then perform global model aggregation of federated learning; one node is selected from several nodes in the consortium chain as the key node of this consortium chain. The consortium chain key node is used to collect training results in the consortium chain, package the training results into candidate blocks, and publish new blocks mined from the candidate blocks to the consortium chain.

[0020] Furthermore, the classifier is used to divide the federated learning process of heterogeneous data into several federated learning processes of homogeneous data.

[0021] In a possible embodiment, the model may be a learning model such as a convolutional neural network model.

[0022] In a possible embodiment, the step 2 specifically includes:

[0023] Step 2.1, the private chain node performs local training based on the local information, and sends the training results to other private chain nodes in the private chain;

[0024] Step 2.2, after receiving the training results sent by other private chain nodes, the private chain node performs weighted calculation and updates the local training results;

[0025] Step 2.3: The model determines whether the local training result converges. If so, execute step 3; otherwise, execute step 2.

[0026] In a possible embodiment, step 4 includes the following steps:

[0027] Step 4.1: The alliance chain node sends the local training results to other alliance chain nodes in the alliance chain;

[0028] Step 4.2: After receiving the training results sent by other alliance chain nodes, the alliance chain node sends them to other private chain nodes in the private chain where it is located. After receiving the training results, the private chain node performs weighted calculations and updates the local training results.

[0029] Step 4.3: The model determines whether the global training results of the system have converged. If so, execute step 5; otherwise, execute step 4.

[0030] In a possible embodiment, the federated learning method based on the double-layer blockchain further includes privacy protection, and the privacy protection includes dynamic weighting and local differential privacy:

[0031] The dynamic weighting is used to prevent false training results from causing distortion of the global training results. The specific idea is that when the model starts iterative training in a federated learning manner, the participants Using a local test suite Verify another participant The training results sent are used as the weight value of the model training to evaluate the training quality. The higher the weight value, the better the training quality. In this solution, the participant is a sub-blockchain node or a main blockchain node.

[0032] Furthermore, the verification standard of the dynamic weighting is cross entropy, and the formula may be:

[0033] ;

[0034] in, represents the expected output of the test set; It indicates the expected result of model training. The smaller the cross entropy value is, the closer the probability distribution predicted by the model is to the actual result, that is, the better the training quality of the model is.

[0035] Furthermore, the weight value calculation formula may be:

[0036] ;

[0037] in, Indicates participants The cross entropy of represents the total number of participants;

[0038] Preferably, the weight value can also be updated regularly to dynamically evaluate the training quality of the participants. Specifically, after a round of weight value calculation, if the weight value of a participant drops abnormally, the participant will be regarded as a "suspect" and will be restricted from participating in federated learning until it receives the training results sent by other participants and its cross entropy returns to normal in the next round of weight value calculation. It can be seen that the weight value can reflect the model training quality of each FLG in the sub-blockchain unit, thereby reflecting the model training quality of each alliance chain in the main blockchain unit.

[0039] The local differential privacy includes an adaptive Laplace mechanism, which is used to achieve differential privacy and improve the degree of data privatization, which can greatly reduce the possibility of privacy leakage and the impact on the model.

[0040] Laplace mechanism, which belongs to the existing technology, specifically, assuming and is an adjacent data set with at most one different data, then for the random query function , its local sensitivity The formula can be defined as follows:

[0041] ;

[0042] It can be seen that the local sensitivity and Related, reflecting the random query function in The Laplace mechanism adds independent zero-mean Laplace noise to blur the query results and protect differential privacy. The specific addition process is shown in the formula:

[0043] ;

[0044] in, represents the Laplace noise operation; represents the location parameter of the Laplace distribution; represents the scale parameter of the Laplace distribution; represents the privacy budget.

[0045] Preferably, the adaptive Laplace mechanism can be expressed by the formula:

[0046] ;

[0047] in, Indicates participants The cross entropy of Indicates local sensitivity; represents the privacy budget; represents Laplace noise; represents the adaptive noise adjustment coefficient;

[0048] Visible, The smaller the value, the The larger the value, the better the privacy protection effect, but the noise interference will increase, resulting in a decrease in data availability. Suppress noise and improve data availability, thereby improving model quality while ensuring privacy.

[0049] In a possible embodiment, the federated learning method based on the two-layer blockchain also includes model aggregation, which is used to perform weighted fusion of global models in federated learning to replace the joint averaging process in the prior art. The joint averaging process cannot solve the problem of low quality of local model training results caused by the degradation of global model quality.

[0050] In a possible embodiment, the model aggregation method is an adaptive global model aggregation algorithm (AGMA) based on joint averaging, which is used to introduce the weight value of model training in the joint averaging process, thereby solving the problem of low quality of local model training results caused by the degradation of global model quality.

[0051] The specific formula of the adaptive global model aggregation algorithm can be expressed as:

[0052] ;

[0053] in, N Indicates the number of participants participating in the model aggregation; Indicates participants The weight value of is used to evaluate the quality of model training; Represents the training result of introducing the adaptive Laplace mechanism;

[0054] Furthermore, the stopping criterion of the AGMA is that the cross entropy after model aggregation is less than or equal to a preset threshold. , or the number of iterations reaches the preset threshold , the specific formula can be expressed as:

[0055] ;

[0056] In a possible embodiment, the federated learning method based on the double-layer blockchain further includes a double-layer consensus, wherein the double-layer consensus includes a sub-layer local consensus and a main-layer global consensus;

[0057] The sub-layer local consensus refers to the local consensus mechanism between the private chain nodes in the sub-blockchain unit;

[0058] Assumptions Representing a random private chain, the specific local consensus process is as follows:

[0059] Step a: After each FLG collects the local training results sent by other FLGs, it aggregates the model through AGMA to generate The local model is calculated, and the cross entropy of the local model is sent to Other FLGs in;

[0060] Step b: The key nodes in the collection collect the cross entropy results sent by each FLG, and pack the local training results and local model cross entropy results into candidate blocks;

[0061] Step c. The key nodes in the real-time statistics of the cross entropy results are The cross entropy of the local model cross entropy quantity is less than or equal to the preset threshold Or the number of iterations reaches the preset threshold In the case of , a block is mined from the candidate block and published to and with The original blocks in the blockchain are linked to form a blockchain, so that the local training results and the local model cross entropy results are permanently recorded in superior.

[0062] The main layer global consensus refers to the global consensus mechanism among the alliance chain nodes in the main blockchain unit;

[0063] The specific global consensus process is as follows:

[0064] Step A: Each private chain key node in the alliance chain broadcasts the global training result of the private chain in which it is located. The private chain key node receives the global training result broadcasted by other private chain key nodes and broadcasts it in the private chain in which it is located, so as to share the global training result in the entire system;

[0065] Step B: After receiving the global training results broadcast by other private chain key nodes, the FLG in each private chain performs model aggregation through AGMA to generate a global model, calculates the cross entropy of the global model, and sends the global model cross entropy result to other FLGs in the private chain;

[0066] Step C: Each private chain key node broadcasts the received global model cross entropy result in the alliance chain. After receiving the global model cross entropy result, the alliance chain key node packages the global training result and the global model cross entropy result into a candidate block;

[0067] Step D: The key nodes of the alliance chain count the cross entropy results of the global model in real time. The cross entropy of the global model cross entropy quantity is less than or equal to the preset threshold Or the number of iterations reaches the preset threshold In the case of a problem, a block is mined from the candidate block and published to the alliance chain, and linked to the original block in the alliance chain to form a blockchain, so that the global training results and the global model cross entropy results are permanently recorded on the alliance chain.

[0068] In the second aspect, based on the same inventive idea, the present invention also provides a federated learning system based on a double-layer blockchain, including a double-layer blockchain module, a double-layer consensus module, a privacy protection module and a model aggregation module:

[0069] The two-layer blockchain module includes a main blockchain unit and a sub-blockchain unit:

[0070] The sub-blockchain unit includes a classifier and several private chains: the private chain includes private chain nodes and private chain key nodes, and the private chain nodes perform local model training based on local information through a federated learning method; the private chain key nodes are used to collect training results in the private chain within a preset time interval, and package the training results that have passed consensus in the training results into a new block and publish it to the private chain; the classifier divides the several private chain nodes and the private chain key nodes into several federated learning groups based on homogeneous data;

[0071] The main blockchain unit includes a consortium chain, the consortium chain includes a consortium chain node and a consortium chain key node, the consortium chain node is served by the private chain key node, and is used for federated learning; the consortium chain key node is used to collect training results in the consortium chain within a preset time interval, and package the training results that have been agreed upon in the training results into a new block and publish it to the consortium chain;

[0072] The two-layer consensus module includes a sub-layer local consensus unit and a main-layer global consensus unit:

[0073] The sub-layer local consensus unit is used for a local consensus mechanism between private chain nodes in the sub-blockchain unit;

[0074] The main layer global consensus unit is used for the global consensus mechanism between the alliance chain nodes in the main blockchain unit;

[0075] The privacy protection module includes a dynamic weighting unit and a local differential privacy unit:

[0076] The dynamic weighting unit includes using cross entropy as a verification criterion to prevent false training results from causing distortion of global training results;

[0077] The local differential privacy unit includes an adaptive Laplace mechanism for achieving differential privacy;

[0078] The model aggregation module realizes weighted fusion of global models through adaptive global model aggregation calculation based on joint averaging.

[0079] In the third aspect, based on the same inventive idea, the present invention also provides a federated learning device based on a double-layer blockchain, including a processor, a memory, a bus and a network card, wherein the memory stores instructions and data that can be read by the processor; the processor is used to call the instructions and data in the memory to execute any of the above-described federated learning methods based on a double-layer blockchain; the bus connects the functional components to transmit information; and the network card is used to transmit information to the outside.

[0080] By adopting the above technical solution, the present invention has the following beneficial effects:

[0081] The federated learning method based on a double-layer blockchain provided by the present invention replaces the centralized aggregation server that is often present in the existing global model federated learning framework through blockchain, thereby avoiding data concentration and improving data security; while ensuring data availability, the level of privacy protection is further improved through adaptive differential privacy; the uniqueness and permanence of data records are achieved through a double-layer consensus method; by adopting this solution, federated learning can be performed more safely in large-scale, wide-area application systems, such as agricultural Internet of Things systems, to obtain more optimized learning models and more accurate prediction data. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0083] Figure 1 A flow chart of a federated learning method based on a double-layer blockchain provided in an embodiment of the present invention;

[0084] Figure 2 A diagram illustrating the blockchain structure provided by an embodiment of the present invention;

[0085] Figure 3 A diagram illustrating the principle of blockchain provided by an embodiment of the present invention;

[0086] Figure 4 A diagram of the training process under the MNIST data set provided by an embodiment of the present invention;

[0087] Figure 5 A diagram of the training process under the CIFAR-10 dataset provided by an embodiment of the present invention;

[0088] Figure 6 A comparison chart of training accuracy under a parameter condition provided by an embodiment of the present invention;

[0089] Figure 7 A comparison chart of training accuracy under another parameter condition provided by an embodiment of the present invention;

[0090] Figure 8 A diagram of a federated learning system based on a double-layer blockchain provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0091] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0092] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0093] The present invention is further explained below in conjunction with specific implementation modes.

[0094] like Figure 1 As shown, a federated learning method based on a double-layer blockchain provided by an embodiment of the present invention includes the following steps:

[0095] Step 1: Build a blockchain in the system, including a sub-blockchain unit and a main blockchain unit:

[0096] The sub-blockchain unit includes a classifier, a sub-blockchain node and a sub-blockchain key node: the classifier divides a plurality of the sub-blockchain nodes and the sub-blockchain key nodes into a plurality of federated learning groups based on homogeneous data;

[0097] The main blockchain unit includes a main blockchain node and a main blockchain key node, and the main blockchain node is served by the sub-blockchain key node;

[0098] Step 2: The sub-blockchain nodes in the federated learning group perform local model training based on local information through a federated learning method;

[0099] Step 3: The key node of the sub-blockchain collects the training results in the sub-blockchain within a preset time interval, and packages the consensus-based training results into a new block and publishes it to the sub-blockchain;

[0100] Step 4: The main blockchain node in the federated learning group performs global model training through a federated learning method based on the training results in the sub-blockchain;

[0101] Step 5: The key nodes of the main blockchain collect the training results in the main blockchain within a preset time interval, and package the consensus training results in the training results into a new block and publish it to the main blockchain.

[0102] Furthermore, if Figure 2 As shown, the system is an agricultural Internet of Things system.

[0103] Furthermore, the local information is agricultural information detected by sensors, such as crop growth status, soil pH, temperature and precipitation, etc.

[0104] Furthermore, the blockchain has a two-layer structure, namely a sub-blockchain layer and a main blockchain layer, the sub-blockchain layer includes sub-blockchain units, and the main blockchain layer includes main blockchain units.

[0105] Furthermore, if Figure 3 As shown, the sub-blockchain unit includes several private chains, and the main blockchain unit includes several alliance chains:

[0106] The private chain includes several sub-blockchain nodes, each of which is a gateway named FLG, which is used to perform federated learning training and send the training results to nearby FLGs, and achieve consensus among FLGs; a key FLG is selected from all FLGs of a private chain as the key node of this private chain, and the key node of the private chain is used to collect transactions in the private chain, package the transactions into candidate blocks, and publish the new blocks mined from the candidate blocks to the private chain;

[0107] The main blockchain unit builds a consortium chain through several key nodes of the private chain. In the consortium chain, the consortium chain nodes are used for federated learning training; several consortium chain nodes send and receive training results to each other, and then perform global model aggregation of federated learning; one node is selected from several nodes in the consortium chain as the key node of this consortium chain. The consortium chain key node is used to collect training results in the consortium chain, package the training results into candidate blocks, and publish new blocks mined from the candidate blocks to the consortium chain.

[0108] Furthermore, the classifier is used to divide the federated learning process of heterogeneous data into several federated learning processes of homogeneous data.

[0109] Furthermore, the model may be a learning model such as a convolutional neural network model.

[0110] Furthermore, the step 2 specifically includes:

[0111] Step 2.1, the private chain node performs local training based on the local information, and sends the training results to other private chain nodes in the private chain;

[0112] Step 2.2, after receiving the training results sent by other private chain nodes, the private chain node performs weighted calculation and updates the local training results;

[0113] Step 2.3: The model determines whether the local training result converges. If so, execute step 3; otherwise, execute step 2.

[0114] Furthermore, step 4 comprises the following steps:

[0115] Step 4.1: The alliance chain node sends the local training results to other alliance chain nodes in the alliance chain;

[0116] Step 4.2: After receiving the training results sent by other alliance chain nodes, the alliance chain node sends them to other private chain nodes in the private chain where it is located. After receiving the training results, the private chain node performs weighted calculations and updates the local training results.

[0117] Step 4.3: The model determines whether the global training results of the system have converged. If so, execute step 5; otherwise, execute step 4.

[0118] Furthermore, the federated learning method based on the double-layer blockchain also includes privacy protection, and the privacy protection includes dynamic weighting and local differential privacy:

[0119] The dynamic weighting is used to prevent false training results from causing distortion of the global training results. The specific idea is that when the model starts iterative training in a federated learning manner, the participants Using a local test suite Verify another participant The training results sent are used as the weight value of the model training to evaluate the training quality. The higher the weight value, the better the training quality. In this solution, the participant is a sub-blockchain node or a main blockchain node.

[0120] Furthermore, the verification standard of the dynamic weighting is cross entropy, and the formula may be:

[0121] ;

[0122] in, represents the expected output of the test set; It indicates the expected result of model training. The smaller the cross entropy value is, the closer the probability distribution predicted by the model is to the actual result, that is, the better the training quality of the model is.

[0123] Furthermore, the weight value calculation formula may be:

[0124] ;

[0125] in, Indicates participants The cross entropy of represents the total number of participants;

[0126] Preferably, the weight value can also be updated regularly to dynamically evaluate the training quality of the participants. Specifically, after a round of weight value calculation, if the weight value of a participant drops abnormally, the participant will be regarded as a "suspect" and will be restricted from participating in federated learning until it receives the training results sent by other participants and its cross entropy returns to normal in the next round of weight value calculation. It can be seen that the weight value can reflect the model training quality of each FLG in the sub-blockchain unit, thereby reflecting the model training quality of each alliance chain in the main blockchain unit.

[0127] The local differential privacy includes an adaptive Laplace mechanism, which is used to achieve differential privacy and improve the degree of data privatization, which can greatly reduce the possibility of privacy leakage and the impact on the model.

[0128] Laplace mechanism, which belongs to the existing technology, specifically, assuming and is an adjacent data set with at most one different data, then for the random query function , its local sensitivity The formula can be defined as follows:

[0129] ;

[0130] It can be seen that the local sensitivity and Related, reflecting the random query function in The Laplace mechanism adds independent zero-mean Laplace noise to blur the query results and protect differential privacy. The specific addition process is shown in the formula:

[0131] ;

[0132] in, represents the Laplace noise operation; represents the location parameter of the Laplace distribution; represents the scale parameter of the Laplace distribution; represents the privacy budget.

[0133] Preferably, the adaptive Laplace mechanism can be expressed by the formula:

[0134] ;

[0135] in, Indicates participants The cross entropy of Indicates local sensitivity; represents the privacy budget; represents Laplace noise; represents the adaptive noise adjustment coefficient;

[0136] Visible, The smaller the value, the The larger the value, the better the privacy protection effect, but the noise interference will increase, resulting in a decrease in data availability. Suppress noise and improve data availability, thereby improving model quality while ensuring privacy.

[0137] Furthermore, the federated learning method based on the two-layer blockchain also includes model aggregation, which is used to perform weighted fusion of global models in federated learning to replace the joint averaging process in the prior art. The joint averaging process cannot solve the problem of low quality of local model training results caused by the degradation of global model quality.

[0138] Furthermore, the model aggregation method is an adaptive global model aggregation algorithm (AGMA) based on joint averaging, which is used to introduce the weight value of model training in the joint averaging process, thereby solving the problem of low quality of local model training results caused by the degradation of global model quality.

[0139] The specific formula of the adaptive global model aggregation algorithm can be expressed as:

[0140] ;

[0141] in, N Indicates the number of participants participating in the model aggregation; Indicates participants The weight value of is used to evaluate the quality of model training; Represents the training result of introducing the adaptive Laplace mechanism;

[0142] Furthermore, the stopping criterion of the AGMA is that the cross entropy after model aggregation is less than or equal to a preset threshold. , or the number of iterations reaches the preset threshold , the specific formula can be expressed as:

[0143] ;

[0144] Furthermore, the federated learning method based on the double-layer blockchain also includes a double-layer consensus, and the double-layer consensus includes a sub-layer local consensus and a main-layer global consensus;

[0145] The sub-layer local consensus refers to the local consensus mechanism between the private chain nodes in the sub-blockchain unit;

[0146] Assumptions Representing a random private chain, the specific local consensus process is as follows:

[0147] Step a: After each FLG collects the local training results sent by other FLGs, it aggregates the model through AGMA to generate The local model is calculated, and the cross entropy of the local model is sent to Other FLGs in;

[0148] Step b: The key nodes in the collection collect the cross entropy results sent by each FLG, and pack the local training results and local model cross entropy results into candidate blocks;

[0149] Step c. The key nodes in the real-time statistics of the cross entropy results are The cross entropy of the local model cross entropy quantity is less than or equal to the preset threshold Or the number of iterations reaches the preset threshold In the case of , a block is mined from the candidate block and published to and with The original blocks in the blockchain are linked to form a blockchain, so that the local training results and the local model cross entropy results are permanently recorded in superior.

[0150] The main layer global consensus refers to the global consensus mechanism among the alliance chain nodes in the main blockchain unit;

[0151] The specific global consensus process is as follows:

[0152] Step A: Each private chain key node in the alliance chain broadcasts the global training result of the private chain in which it is located. The private chain key node receives the global training result broadcasted by other private chain key nodes and broadcasts it in the private chain in which it is located, so as to share the global training result in the entire system;

[0153] Step B: After receiving the global training results broadcast by other private chain key nodes, the FLG in each private chain performs model aggregation through AGMA to generate a global model, calculates the cross entropy of the global model, and sends the global model cross entropy result to other FLGs in the private chain;

[0154] Step C: Each private chain key node broadcasts the received global model cross entropy result in the alliance chain. After receiving the global model cross entropy result, the alliance chain key node packages the global training result and the global model cross entropy result into a candidate block;

[0155] Step D: The key nodes of the alliance chain count the cross entropy results of the global model in real time. The cross entropy of the global model cross entropy quantity is less than or equal to the preset threshold Or the number of iterations reaches the preset threshold In the case of a problem, a block is mined from the candidate block and published to the alliance chain, and linked to the original block in the alliance chain to form a blockchain, so that the global training results and the global model cross entropy results are permanently recorded on the alliance chain.

[0156] Embodiment 1:

[0157] Simulating horizontal federated learning in agricultural IoT systems through a two-layer blockchain-based federated learning approach;

[0158] The model of the task is a deep learning model composed of a multilayer perceptron and a convolutional neural network. The model is written in python3.6 and pytorch0.4.1. The model uses a stochastic gradient descent algorithm for iterative optimization. The model uses the MNIST and CIFAR-10 image datasets as training data, respectively. Each image dataset will be divided according to the number of working nodes.

[0159] In this example, Laplace noise is added to the intermediate parameters of the training process and a random function is called. To evaluate the impact of the privacy budget on the model accuracy, the data sensitivity Set to 1 to control the offset parameter Set to 1, privacy budget They are set to 0.1, 0.3, 1, and 5 respectively. The larger the privacy budget, the smaller the added noise value.

[0160] This example uses the MNIST and CIFAR-10 datasets to perform training in a single node to compare the loss changes during the training process. The training process under the MNIST dataset is as follows: Figure 4 As shown in the figure, as the number of test batches increases, the test loss decreases and eventually converges to 0.5.

[0161] The training process under the CIFAR-10 dataset is as follows Figure 5 As shown, the test loss converges to 2.0.

[0162] If the number of nodes participating in the training is set to 3, and each node aggregates 5 times locally, then the initial privacy budget The training accuracy comparison results when the settings are 0.1, 0.3, 1 and 5 are as follows Figure 6 When the privacy budget is 0.1, the impact of noise on training accuracy is obvious, and as the privacy budget increases, the training accuracy continues to improve.

[0163] If each node is set to aggregate locally 5 times, the learning rate = 1, then the training accuracy comparison when the number of nodes participating in training is set to 3, 6, 9, and 12 respectively is Figure 7 As shown in the figure, as the number of nodes participating in the training increases, the training accuracy continues to improve. If the model training results provided by the participating training nodes are of poor quality, the training accuracy will decrease, and when the global model aggregation reaches 100 times, the accuracy tends to converge. In particular, the average accuracy of the 12 participating training nodes is 96.8%.

[0164] It can be seen that the accuracy of the federated learning method based on the double-layer blockchain provided by the embodiment of the present invention increases with the increase in the number of data providers. Since the participating training nodes can perform local training in parallel, the accuracy of federated learning is improved, and the adaptive Laplace noise can achieve privacy protection and reduce the impact of agricultural IoT smart devices on model accuracy.

[0165] On the other hand, Figure 8 As shown, an embodiment of the present invention further provides a federated learning system based on a double-layer blockchain, including a double-layer blockchain module, a double-layer consensus module, a privacy protection module and a model aggregation module:

[0166] The two-layer blockchain module includes a main blockchain unit and a sub-blockchain unit:

[0167] The sub-blockchain unit includes a classifier and several private chains: the private chain includes private chain nodes and private chain key nodes, and the private chain nodes perform local model training based on local information through a federated learning method; the private chain key nodes are used to collect training results in the private chain within a preset time interval, and package the training results that have passed consensus in the training results into a new block and publish it to the private chain; the classifier divides the several private chain nodes and the private chain key nodes into several federated learning groups based on homogeneous data;

[0168] The main blockchain unit includes a consortium chain, the consortium chain includes a consortium chain node and a consortium chain key node, the consortium chain node is served by the private chain key node, and is used for federated learning; the consortium chain key node is used to collect training results in the consortium chain within a preset time interval, and package the training results that have been agreed upon in the training results into a new block and publish it to the consortium chain;

[0169] The two-layer consensus module includes a sub-layer local consensus unit and a main-layer global consensus unit:

[0170] The sub-layer local consensus unit is used for a local consensus mechanism between private chain nodes in the sub-blockchain unit;

[0171] The main layer global consensus unit is used for the global consensus mechanism between the alliance chain nodes in the main blockchain unit;

[0172] The privacy protection module includes a dynamic weighting unit and a local differential privacy unit:

[0173] The dynamic weighting unit includes using cross entropy as a verification criterion to prevent false training results from causing distortion of global training results;

[0174] The local differential privacy unit includes an adaptive Laplace mechanism for achieving differential privacy;

[0175] The model aggregation module realizes weighted fusion of global models through adaptive global model aggregation calculation based on joint averaging.

[0176] On the other hand, an embodiment of the present invention also provides a federated learning device based on a double-layer blockchain, including a processor, a memory, a bus and a network card, wherein the memory stores instructions and data that can be read by the processor; the processor is used to call the instructions and data in the memory to execute any of the double-layer blockchain-based federated learning methods described above; the bus connects the functional components to transmit information; and the network card is used to transmit information to the outside.

[0177] In another embodiment, the present solution can be implemented by a device, which may include a corresponding module for performing each or several steps in the above-mentioned embodiments. The module may be one or more hardware modules specially configured to perform the corresponding steps, or implemented by a processor configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0178] The processor performs the various methods and processes described above. For example, the method implementation in the present solution can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be executed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods by any other appropriate means (for example, by means of firmware).

[0179] The device can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus connects various circuits including one or more processors, memories, and / or hardware modules together. The bus can also connect various other circuits such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0180] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A federated learning method based on a double-layer blockchain, characterized in that: The steps include: Step 1: Build a two-layer blockchain in the agricultural IoT system, including a sub-blockchain unit and a main blockchain unit: The sub-blockchain unit includes a classifier, a sub-blockchain node and a sub-blockchain key node: the classifier divides a plurality of the sub-blockchain nodes and the sub-blockchain key nodes into a plurality of federated learning groups based on homogeneous data; The main blockchain unit includes a main blockchain node and a main blockchain key node, and the main blockchain node is served by the sub-blockchain key node; Step 2: The sub-blockchain nodes in the federated learning group perform local model training based on local information through a federated learning method; the local information is agricultural information detected by sensors, including crop growth status, soil pH, temperature and precipitation; Step 3: The key node of the sub-blockchain collects the training results in the sub-blockchain within a preset time interval, and packages the consensus-based training results into a new block and publishes it to the sub-blockchain; Step 4: The main blockchain node in the federated learning group performs global model training through a federated learning method based on the training results in the sub-blockchain; Step 5: The key node of the main blockchain collects the training results in the main blockchain within a preset time interval, and packages the consensus-based training results into a new block and publishes it to the main blockchain; The model is a deep learning model composed of a multi-layer perceptron and a convolutional neural network. The model is written in python 3.6 and pytorch 0.4.

1. The model is iteratively optimized using a stochastic gradient descent algorithm. The model uses the MNIST and CIFAR-10 image datasets as training data, respectively. Each image dataset will be divided according to the number of working nodes. The federated learning method includes privacy protection, and the privacy protection includes dynamic weighting and local differential privacy: The dynamic weighting is used to prevent false training results from causing distortion of global training results, and the verification standard of the dynamic weighting is cross entropy; The local differential privacy includes an adaptive Laplace mechanism; The federated learning method also includes a two-layer consensus, which includes a sub-layer local consensus and a main-layer global consensus: The sub-layer local consensus refers to the local consensus mechanism between the private chain nodes in the sub-blockchain unit; The main layer global consensus refers to the global consensus mechanism among the alliance chain nodes in the main blockchain unit.

2. The federated learning method based on a double-layer blockchain according to claim 1 is characterized in that: The sub-blockchain unit includes several private chains, and the main blockchain unit includes several alliance chains: The private chain includes a plurality of sub-blockchain nodes, and one of the sub-blockchain nodes of a private chain is selected as a key node of the private chain; The main blockchain unit builds a consortium chain through several key nodes of the private chain; and selects one from several nodes in the consortium chain as a key node of the consortium chain.

3. The federated learning method based on a double-layer blockchain according to claim 2 is characterized in that: The step 2 specifically includes: Step 2.1, the private chain node performs local training based on the local information, and sends the training results to other private chain nodes in the private chain; Step 2.2, after receiving the training results sent by other private chain nodes, the private chain node performs weighted calculation and updates the local training results; Step 2.3: The model determines whether the local training result converges. If so, execute step 3; otherwise, execute step 2.

4. The federated learning method based on a double-layer blockchain according to claim 2 is characterized in that: The step 4 comprises the following steps: Step 4.1: The alliance chain node sends the local training results to other alliance chain nodes in the alliance chain; Step 4.2: After receiving the training results sent by other alliance chain nodes, the alliance chain node sends them to other private chain nodes in the private chain where it is located. After receiving the training results, the private chain node performs weighted calculations and updates the local training results. Step 4.3: The model determines whether the global training results of the system have converged. If so, execute step 5; otherwise, execute step 4.

5. The federated learning method based on a double-layer blockchain according to claim 1, characterized in that: The federated learning method also includes model aggregation. The model aggregation method is to introduce the weight value of model training in the joint averaging process. The stopping standard of the model aggregation is that the cross entropy after model aggregation is less than or equal to a preset threshold or the number of iterations reaches a preset threshold.

6. A federated learning system based on a double-layer blockchain, characterized in that: Including the two-layer blockchain module, two-layer consensus module, privacy protection module and model aggregation module built in the agricultural Internet of Things system: The two-layer blockchain module includes a main blockchain unit and a sub-blockchain unit: The sub-blockchain unit includes a classifier and several private chains: the private chain includes private chain nodes and private chain key nodes, and the private chain nodes perform local model training based on local information through a federated learning method; the local information is agricultural information detected by sensors, including crop growth status, soil pH, temperature and precipitation; the private chain key nodes are used to collect training results in the private chain within a preset time interval, and package the training results that have passed consensus in the training results into a new block and publish it to the private chain; the classifier divides several private chain nodes and private chain key nodes into several federated learning groups based on homogeneous data; The main blockchain unit includes a consortium chain, the consortium chain includes a consortium chain node and a consortium chain key node, the consortium chain node is served by the private chain key node, and is used for federated learning; the consortium chain key node is used to collect training results in the consortium chain within a preset time interval, and package the training results that have been agreed upon in the training results into a new block and publish it to the consortium chain; The two-layer consensus module includes a sub-layer local consensus unit and a main-layer global consensus unit: The sub-layer local consensus unit is used for a local consensus mechanism between private chain nodes in the sub-blockchain unit; The main layer global consensus unit is used for the global consensus mechanism between the alliance chain nodes in the main blockchain unit; The privacy protection module includes a dynamic weighting unit and a local differential privacy unit: The dynamic weighting unit includes using cross entropy as a verification criterion to prevent false training results from causing distortion of global training results; The local differential privacy unit includes an adaptive Laplace mechanism for achieving differential privacy; The model aggregation module realizes weighted fusion of global models through adaptive global model aggregation calculation based on joint averaging; The model is a deep learning model composed of a multilayer perceptron and a convolutional neural network. The model is written in python3.6 and pytorch0.4.

1. The model uses a stochastic gradient descent algorithm for iterative optimization. The model uses MNIST and CIFAR-10 image datasets as training data, respectively. Each image dataset will be divided according to the number of working nodes.

7. A federated learning device based on a double-layer blockchain, characterized in that: It includes a processor, a memory, a bus and a network card, wherein the memory stores instructions and data that can be read by the processor; The processor is used to call the instructions and data in the memory to execute the federated learning method based on the double-layer blockchain as described in any one of claims 1 to 5; The bus connects the functional components to transmit information; The network card is used to transmit information externally.

Images