A Federated Learning System and Method Based on DAG Blockchain

Abstract

This invention relates to a federated learning system based on DAG blockchain, comprising a cloud service layer, an edge server layer, and a mobile device layer. It also relates to a federated learning method based on DAG blockchain. In this DAG blockchain-based federated learning, clients can participate in federated learning by actively publishing blocks that store machine learning models, thus shifting from passive to active participation in traditional federated learning. Clients and other nodes in the blockchain network store only a portion of the DAG ledger instead of the entire ledger, ensuring data reliability while reducing node storage resource consumption. The integration of DAG blockchain and the federated learning system creates a federated learning system that users can actively participate in, effectively solving the single point of failure problem. By having the mobile device layer and edge server layer store only a portion of the DAG ledger, the storage consumption of nodes in the DAG blockchain network is reduced.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology in a distributed environment, specifically relating to a federated learning system and method based on DAG blockchain. Background Technology

[0002] With the constant information collection and ever-increasing computing power of mobile devices such as smartphones, laptops, and tablets, the rapid development of edge intelligence has transformed many aspects of modern life, including business and services. To effectively utilize the data generated on these mobile devices to provide better services to users, artificial intelligence technologies such as machine learning have been widely used to achieve more intelligent applications. However, with the increase in personal data and the growing demand for data privacy protection, centralized machine learning methods are no longer suitable for model training based on mobile device data.

[0003] The emergence of federated learning has driven the development of edge intelligence. Federated learning is an emerging distributed machine learning paradigm that typically relies on multiple clients (such as smart mobile devices, IoT devices, or different organizations) and a central server to train machine learning models. Each time, the central server selects several clients as participants, aggregates the models uploaded by these clients into a global model, and sends it to the next selected clients to advance the training process. Federated learning maintains the decentralized nature of training data during training; the raw data held by each client does not leave its local storage medium, which largely guarantees client data privacy compared to centralized machine learning.

[0004] However, current federated learning systems still have some problems. First, clients need to trust the model aggregation results from a central server, which is impractical for federated learning systems exposed to public network environments. Second, if the central server is attacked, the entire federated learning system's reliance on its model aggregation function will lead to system crashes. Finally, mobile networks have high scalability; if the number of devices in the network reaches millions, the central server is likely to reach its performance bottleneck and be unable to perform model aggregation smoothly, thus causing delays to the entire federated learning system. Therefore, there is an urgent need for a decentralized federated learning solution that does not rely on a central server to solve the security and scalability issues.

[0005] Blockchain, as a distributed ledger technology, can provide an effective solution for decentralized federated learning. Blockchain has already demonstrated strong security and reliability in numerous digital cryptocurrency systems. A blockchain is a linked list where miners calculate hash values ​​to connect blocks under consensus mechanisms, such as proof-of-work. Blockchain can free federated learning from dependence on a central server, thus mitigating the risk of single points of failure. Furthermore, cryptographic methods such as digital signatures allow blockchain to permanently and immutably preserve historical data. Data updates on the blockchain can be traced transparently, placing the actions of every client participating in federated learning under public supervision and enhancing its robustness. Compared to traditional blockchains that maintain a main chain, blockchains based on the Directed Acyclic Graph (DAG) data structure represent a relatively novel paradigm. Current DAG-based blockchain projects include IOTA, ByteSnowball, and Hedera. In a DAG blockchain, blocks are no longer packaged and uploaded by miners. Instead, each node in a DAG blockchain can directly upload blocks to the network, and after verification, they are permanently recorded on the DAG ledger, thus consuming fewer resources.

[0006] Both traditional blockchains and DAG blockchains can be used to construct decentralized federated learning. Current research suggests two main differences: (1) Whether a global model emerges during the federated learning process. In federated learning based on traditional blockchains, the blockchain typically replaces the central server. The global model aggregation work, traditionally handled by the central server, is undertaken by miners and smart contracts. In federated learning based on DAG blockchains, each client aggregates a local model based on the unconfirmed blocks (tips) in the DAG ledger selected during participation, and trains the model accordingly. The probability of clients training on the same model is low. (2) The way clients participate in federated learning differs, which is related to the presence of a global model. In federated learning based on traditional blockchains, clients still participate by waiting to be selected. This participation method is limited by the need to compute a global model in federated learning. In federated learning based on DAG blockchains, there is no global model aggregation process. Therefore, the blockchain does not need to select some clients to participate in federated learning; clients can actively participate based on the DAG ledger status and their own needs.

[0007] Therefore, DAG blockchains offer advantages over traditional blockchains for deploying decentralized federated learning applications on mobile networks. First, DAG blockchains require fewer additional resources for client consensus, allowing clients to dedicate more resources to training machine learning models, which is beneficial for resource-constrained mobile devices. Second, in DAG-based federated learning, clients can actively participate in federated learning at any time, which is advantageous for mobile devices that cannot be online for extended periods. Finally, in traditional federated learning, unreliable clients can lead to insufficient data for a single global model aggregation; selected clients may go offline or fail to complete tasks in a timely manner during training, ultimately affecting model training performance. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a federated learning system and method based on DAG blockchain. In federated learning based on DAG blockchain, clients can participate in federated learning by actively publishing blocks that store machine learning models according to their own needs, realizing the transformation from passive participation to active participation in traditional federated learning. Clients and other nodes in the blockchain network store only a portion of the DAG ledger instead of storing the full DAG ledger, which reduces the storage resource consumption of nodes while ensuring data reliability.

[0009] The technical problem solved by this invention is achieved through the following technical solution:

[0010] A federated learning system based on DAG blockchain, characterized by comprising a cloud service layer, an edge server layer, and a mobile device layer;

[0011] Cloud service layer: Publishing federated learning tasks, selecting machine learning models, specifying training parameters, publishing training scripts, initializing model weights, allocating task incentives, creating the genesis block of the DAG ledger, formulating the tip selection algorithm, monitoring the federated learning process, and pricing and selling the global federated learning model.

[0012] Edge server layer: Stores and forwards message data from the cloud service layer to the mobile device layer, forwards DAG blocks published by devices in the mobile device layer, redundantly stores the DAG ledger, and handles data synchronization requests from devices in the mobile device layer;

[0013] Mobile device layer: Federated learning model training, DAG block publication, DAG ledger maintenance, and incentives obtained from the cloud service layer. The DAG blockchain maintained by the client in the mobile device layer is a public chain. The client participates in federated learning model training based on understanding the federated learning tasks published by the cloud service layer. According to the parameters and model specified in the genesis block, the model is trained on the local dataset. Each time, the local model is packaged into a DAG block and uploaded to the nearby edge server. When the client needs to synchronize data, it can choose between two modes: synchronization from neighboring clients and direct synchronization from the nearby edge server. The clients in the mobile device layer redundantly store the complete DAG ledger to prevent data loss between the edge server layer and the cloud service layer.

[0014] A federated learning method based on DAG blockchain, characterized in that: the steps of the learning method are as follows:

[0015] 1) The cloud service layer publishes federated learning tasks and genesis blocks containing necessary data to the edge server layer;

[0016] 2) The mobile device layer acts as a client to download several DAG blocks containing machine learning models from the edge server layer or other mobile devices, and calculates local models based on the downloaded models;

[0017] 3) After training the model on local data, the client packages the model data into DAG blocks;

[0018] 4) The client uploads the DAG block to a nearby edge server layer;

[0019] 5) The edge server layer broadcasts the received DAG blocks within the edge server layer network;

[0020] 6) The edge server layer uploads the DAG blocks to the cloud service layer;

[0021] 7) The cloud service layer monitors the federated learning process, i.e. the DAG ledger expansion process, based on the DAG blocks uploaded by the edge server layer, and selects an appropriate time to end the federated learning task. After the task is completed, the cloud service layer needs to allocate incentives to the clients that participated in the federated learning task based on the DAG ledger.

[0022] Furthermore, in step 4), after the edge server layer calculates the block storage value based on the DAG ledger state, it reaches a consensus on the redundant storage of low-value block data according to the consensus algorithm based on the Verifiable Random Function (VRF). The Verifiable Random Function takes the private key sk and the random seed seed as input and outputs a pseudo-random number rand and a proof of the pseudo-random number proof. Anyone can use the public key pk corresponding to the private key sk and the proof to verify the correctness of the pseudo-random number rand. The steps of the consensus algorithm are as follows:

[0023] S1. Based on the set random seed, the edge servers in the consortium chain use VRF to calculate and obtain a pseudo-random number rand and a proof. Then, the edge servers whose pseudo-random number rand meets the system's set threshold broadcast the pseudo-random number rand and the proof, thereby obtaining a consensus layer. The edge server layer leaves or joins the consensus group according to the size of the pseudo-random number rand, so that the number of edge server layers in the consensus group remains at k+m.

[0024] S2. Each edge server layer in the consensus group proposes an unprocessed low-value block proposal. The proposal with the fewest low-value blocks is selected. The threshold is used to prevent the DAG ledger from being full of high-value blocks due to network congestion or latency, which would adversely affect the consensus result. Then, the proposing edge server divides the low-value block into k original data blocks and encodes them into k+m encoded data blocks. It also proposes a data allocation scheme. Typically, one edge server in the consensus group stores one encoded data block. The script for encoding the original data is then sent to each edge server. Other edge servers selectively store the encoded data blocks according to the data allocation scheme and the encoding script. The final data allocation method depends on the performance of the edge server. If the edge server has strong computing power, the method described above is used. If the edge server has large bandwidth but general computing power, the proposing edge server can directly send the encoded data blocks to the edge servers in the group.

[0025] S3. After the edge server in the consensus group encodes the original data, it packages the result of step S2 into a consortium blockchain block and broadcasts it to the consortium blockchain network. The edge server layer that receives the consortium blockchain block can remove the block body data of the low-value block stored locally based on the original data Merkle tree in the block body, and mark the block as a processed low-value block.

[0026] The advantages and beneficial effects of this invention are as follows:

[0027] 1. This invention relates to a federated learning system and method based on DAG blockchain. In federated learning based on DAG blockchain, clients can participate in federated learning by actively publishing blocks that store machine learning models according to their own needs, realizing the transformation from passive participation to active participation in traditional federated learning. Clients and other nodes in the blockchain network store only a portion of the DAG ledger instead of storing the full DAG ledger, which reduces the storage resource consumption of nodes while ensuring data reliability.

[0028] 2. This invention integrates DAG blockchain with a federated learning system, realizing a federated learning system in which users can actively participate, and effectively solving the problem of single point of failure.

[0029] 3. This invention reduces the storage consumption of nodes in the DAG blockchain network by having the mobile device layer and edge server layer store only a portion of the DAG ledger.

[0030] 4. This invention designs a consortium blockchain and consensus algorithm to enable edge servers to reach a consensus on storing outdated and redundant DAG ledger data, thereby achieving system autonomy to a certain extent. Attached Figure Description

[0031] Figure 1 This is a system framework diagram of the present invention;

[0032] Figure 2 This is a schematic diagram illustrating client-side participation in federated learning according to the present invention.

[0033] Figure 3 This is a schematic diagram of the mobile device layer (client) storage optimization strategy of the present invention;

[0034] Figure 4 This is a diagram of the consortium blockchain data model of the present invention. Detailed Implementation

[0035] The present invention will be further described in detail below through specific embodiments. The following embodiments are merely descriptive and not limiting, and should not be used to limit the scope of protection of the present invention.

[0036] A federated learning system based on DAG blockchain, the system framework of this invention is as follows: Figure 1 As shown, the functions of the three roles are described in detail below:

[0037] Cloud service layer: Publishing federated learning tasks, selecting machine learning models, specifying training parameters, publishing training scripts, initializing model weights, allocating task incentives, creating the genesis block of the DAG ledger, formulating tip selection algorithms, monitoring the federated learning process, and pricing and selling the global federated learning model.

[0038] Edge server layer: Stores and forwards message data from the cloud service layer to the mobile device layer, forwards DAG blocks published by devices in the mobile device layer, redundantly stores the DAG ledger, and handles data synchronization requests from devices in the mobile device layer. The edge server layer primarily undertakes the task of storing the DAG ledger, serving the mobile device layer. By redundantly storing the DAG ledger on multiple edge servers, situations where devices in the mobile device layer cannot synchronize data from adjacent clients can be avoided, thus ensuring the availability of the DAG ledger data.

[0039] Mobile device layer: Federated learning model training, DAG block publication, DAG ledger maintenance, and incentives obtained from the cloud service layer. The DAG blockchain maintained by clients in the mobile device layer is a public blockchain. Clients participate in federated learning model training based on their understanding of the federated learning tasks published by the cloud service layer. According to the parameters and model specified in the genesis block, they train the model on their local dataset, packaging the local model into a DAG block and uploading it to a nearby edge server each time. When clients need to synchronize data, they can choose between synchronizing from neighboring clients and directly from a nearby edge server. Clients in the mobile device layer redundantly store the complete DAG ledger to prevent data loss between the edge server layer and the cloud service layer.

[0040] according to Figure 2 A federated learning method based on DAG blockchain. A complete federated learning task process includes the following seven steps, of which steps (2) to (6) need to be executed multiple times:

[0041] (1) The cloud service layer publishes federated learning tasks and genesis blocks containing necessary data to edge servers.

[0042] (2) The mobile device, as a client, downloads several DAG blocks containing machine learning models from the edge server or other mobile devices, and calculates local models based on the downloaded models.

[0043] (3) After training the model on local data, the client packages the model data into DAG blocks.

[0044] (4) The client uploads the DAG block to a nearby edge server.

[0045] (5) The edge server broadcasts the received DAG blocks within the edge server layer network.

[0046] (6) The edge server uploads the DAG block to the cloud server.

[0047] (7) The cloud service layer monitors the federated learning process, i.e. the DAG ledger expansion process, based on the DAG blocks uploaded by the edge server, and selects an appropriate time to end the federated learning task. After the task is completed, the cloud service layer needs to allocate incentives to the clients participating in the federated learning task based on the DAG ledger.

[0048] The node storage optimization strategy is divided into two parts: mobile device (client) storage optimization and edge server storage optimization. The storage optimization strategy for mobile devices is as follows: Figure 3As shown, the client only stores the blocks it uploads and the blocks directly connected to the uploaded blocks. From the perspective of the entire mobile device layer, the number of times a single block is stored is the sum of the block's in-degree and out-degree in the DAG ledger. Therefore, each block is stored on at least two clients, ensuring redundant storage of the complete DAG ledger at the mobile device layer. The edge server divides blocks into high-value and low-value blocks by calculating their storage value. A multi-replica storage strategy is used for high-value blocks to ensure data availability, while a redundancy storage strategy based on erasure coding is used for low-value blocks. The edge server executes a consensus algorithm based on a Verifiable Random Function (VRF), using the consortium blockchain as an index, to achieve consensus on the encoded low-value block data. The consortium blockchain data model is as follows: Figure 4 As shown.

[0049] After the edge server calculates the storage value of blocks based on the DAG ledger state, it needs to reach a consensus on the redundant storage of low-value block data according to the designed VRF-based consensus algorithm. VRF can take a private key sk and a random seed seed as input and output a pseudo-random number rand and a proof of the pseudo-random number proof. Anyone can use the public key pk corresponding to the private key sk and the proof to verify the correctness of the pseudo-random number rand. The consensus algorithm consists of the following three steps:

[0050] (1) According to the random seed set by the system, the edge servers in the consortium chain use VRF to calculate and obtain the pseudo-random number rand and the proof. Then, the edge servers whose pseudo-random number rand meets the threshold set by the system broadcast the pseudo-random number rand and the proof, thereby obtaining a consensus group. The edge servers leave or join the consensus group according to the size of the pseudo-random number rand, so that the number of edge servers in the consensus group remains at k+m.

[0051] (2) Each edge server in the consensus group proposes an unprocessed low-value block proposal. The proposal with the fewest low-value blocks is selected. This threshold is used to prevent network congestion or latency from causing the DAG ledger to contain only high-value blocks, negatively impacting the consensus result. The proposing edge server (if multiple edge servers submit the same proposal, the one with the largest pseudo-random number, rand, executes the proposal) divides the low-value block into k original data blocks, encodes them into k+m encoded data blocks, and proposes a data allocation scheme. Typically, one edge server within the consensus group stores one encoded data block. The script for encoding the original data is then sent to each edge server, and other edge servers selectively store the encoded data blocks according to the data allocation scheme and the encoding script. It should be noted that the final data allocation method depends on the edge server's performance. If the edge server has strong computing power, the method described above is used. If the edge server has large bandwidth but limited computing power, the proposing edge server can directly send the encoded data blocks to the edge servers within the group.

[0052] (3) The edge servers within the consensus group that encode the original data package the results from step (2) into a format such as... Figure 4 After a consortium blockchain block is generated, it is broadcast to the consortium blockchain network. Edge servers that receive the consortium blockchain block can remove the block body data of low-value blocks stored locally based on the original data Merkle tree in the block body, and mark the block as a processed low-value block.

[0053] Consortium blockchain block data model, such as Figure 4 As shown, besides the necessary block header information, the block body stores the storage Merkle tree and the original data Merkle tree. The transactions in the storage Merkle tree contain relevant storage information for the encoded data blocks, while the original data Merkle tree contains information about the original DAG block. The storage Merkle tree ensures that even if no more than m encoded data blocks are lost, the edge server can retrieve the encoded data from the accurate location and decode it to obtain the original data. The original data Merkle tree ensures that when the edge server receives a request to synchronize low-value blocks, it can quickly locate the block storage location and respond to the data synchronization request, in conjunction with the storage Merkle tree. The role of the consortium blockchain is to create an index for the encoded data blocks, which are stored off-chain on different edge servers.

[0054] Although embodiments and drawings of the present invention have been disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, variations and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.

Claims

1. A federated learning method based on DAG blockchain, characterized in that: The steps of the learning method are as follows: 1) The cloud service layer publishes federated learning tasks and genesis blocks containing necessary data to the edge server layer; 2) The mobile device layer acts as a client to download several DAG blocks containing machine learning models from the edge server layer or other mobile devices, and calculates local models based on the downloaded models; 3) After training the model on local data, the client packages the model data into DAG blocks; 4) The client uploads the DAG blocks to a nearby edge server layer; 5) The edge server layer broadcasts the received DAG blocks within the edge server layer network; 6) The edge server layer uploads the DAG blocks to the cloud service layer; 7) The cloud service layer monitors the federated learning process, i.e. the DAG ledger expansion process, based on the DAG blocks uploaded by the edge server layer, and selects an appropriate time to end the federated learning task. After the task is completed, the cloud service layer needs to allocate incentives to the clients that participated in the federated learning task based on the DAG ledger. In step 4), after the edge server layer calculates the block storage value based on the DAG ledger state, it reaches a consensus on the redundant storage of low-value block data according to the consensus algorithm based on the Verifiable Random Function (VRF). The Verifiable Random Function takes the private key sk and the random seed seed as input and outputs a pseudo-random number rand and a proof of the pseudo-random number proof. Anyone can use the public key pk corresponding to the private key sk and the proof to verify the correctness of the pseudo-random number rand. The steps of the consensus algorithm are as follows: S1. Based on the set random seed, the edge servers in the consortium chain use VRF to calculate and obtain a pseudo-random number rand and a proof. Then, the edge servers whose pseudo-random number rand meets the system's set threshold broadcast the pseudo-random number rand and the proof, thereby obtaining a consensus layer. The edge server layer leaves or joins the consensus group according to the size of the pseudo-random number rand, so that the number of edge server layers in the consensus group remains at k+m. S2. Each edge server layer in the consensus group proposes an unprocessed low-value block proposal. The proposal with the fewest low-value blocks is selected. The threshold is used to prevent the DAG ledger from being full of high-value blocks due to network congestion or latency, which would adversely affect the consensus result. Then, the proposing edge server divides the low-value block into k original data blocks and encodes them into k+m encoded data blocks. It also proposes a data allocation scheme. Typically, one edge server in the consensus group stores one encoded data block. The script for encoding the original data is then sent to each edge server. Other edge servers selectively store the encoded data blocks according to the data allocation scheme and the encoding script. The final data allocation method depends on the performance of the edge server. If the edge server has strong computing power, the method described above is used. If the edge server has large bandwidth but general computing power, the proposing edge server can directly send the encoded data blocks to the edge servers in the group. S3. After the edge server in the consensus group encodes the original data, it packages the result of step S2 into a consortium blockchain block and broadcasts it to the consortium blockchain network. The edge server layer that receives the consortium blockchain block can remove the block body data of the low-value block stored locally based on the original data Merkle tree in the block body, and mark the block as a processed low-value block.

2. A DAG-based federated learning system for performing the method as described in claim 1, comprising a cloud service layer, an edge server layer, and a mobile device layer; Cloud service layer: Publishing federated learning tasks, selecting machine learning models, specifying training parameters, publishing training scripts, initializing model weights, allocating task incentives, creating the genesis block of the DAG ledger, formulating the tip selection algorithm, monitoring the federated learning process, and pricing and selling the global federated learning model. Edge server layer: Stores and forwards message data from the cloud service layer to the mobile device layer, forwards DAG blocks published by devices in the mobile device layer, redundantly stores the DAG ledger, and handles data synchronization requests from devices in the mobile device layer; Mobile device layer: Federated learning model training, DAG block publication, DAG ledger maintenance, and incentives obtained from the cloud service layer. The DAG blockchain maintained by the client in the mobile device layer is a public chain. The client participates in federated learning model training based on understanding the federated learning tasks published by the cloud service layer. According to the parameters and model specified in the genesis block, the model is trained on the local dataset. Each time, the local model is packaged into a DAG block and uploaded to the nearby edge server. When the client needs to synchronize data, it can choose between two modes: synchronization from neighboring clients and direct synchronization from the nearby edge server. The clients in the mobile device layer redundantly store the complete DAG ledger to prevent data loss between the edge server layer and the cloud service layer.

Images