A blockchain-based personalized federated learning method and device
By performing personalized federated learning on the blockchain, separating common and personalized features to train the model, and using smart contracts for decentralized communication, the problems of poor model performance and security in traditional federated learning under heterogeneous datasets are solved, thereby improving model performance and enhancing system reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUN YAT SEN UNIV
- Filing Date
- 2023-04-18
- Publication Date
- 2026-05-15
AI Technical Summary
Traditional federated learning performs poorly in cases of severely heterogeneous datasets and suffers from security and scalability issues.
A blockchain-based personalized federated learning approach is adopted, which divides the client's local model into two parts: extracting common features and personalized features. The model is aggregated through blockchain consensus nodes, and decentralized communication is achieved using smart contracts.
In scenarios with highly heterogeneous datasets, this approach aims to improve model performance, reduce communication costs, enhance system reliability and scalability, and alleviate the storage and query pressure on the blockchain.
Smart Images
Figure CN116432777B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine learning technology, and in particular to a blockchain-based personalized federated learning method and apparatus. Background Technology
[0002] With the development of the digital society, massive amounts of data have been generated. Machine learning technology can be used to extract rich information from this data, and machine learning models trained on large amounts of data have been widely used in various application fields. However, with the increasing awareness of data security and the continuous improvement of privacy protection laws and regulations, data sharing between different users has become increasingly difficult, forming the phenomenon of "data silos," which restricts the further improvement of the capabilities of artificial intelligence models.
[0003] To address these issues, federated learning was proposed. In a federated learning system, the client uploads local model parameters or gradient information trained on local data to the server. Since the local model contains information about its local dataset, aggregation on the server side allows for joint training of the model using multiple datasets, thereby achieving privacy protection.
[0004] However, in practical applications, the distribution of datasets across different clients can vary significantly. This leads to substantial discrepancies in the trained local models, making it impossible to aggregate a global model that satisfies most clients during the aggregation phase. Furthermore, since traditional federated learning requires a central server to aggregate the global model, security and scalability issues also arise. Summary of the Invention
[0005] The purpose of this invention is to provide a blockchain-based personalized federated learning method and apparatus to solve the technical problems of poor model performance, security, and scalability in traditional federated learning when the datasets are severely heterogeneous.
[0006] The objective of this invention can be achieved through the following technical invention:
[0007] A blockchain-based personalized federated learning method includes:
[0008] Each client participating in federated learning downloads the global model for this round from the consensus node of the blockchain, uses the global model as the first local model, and each client trains a local model using the local training set. The local model includes the first local model, the second local model, and the local classification model.
[0009] The first local model is used to extract common features that are unrelated to the client distribution, the second local model is used to extract personalized features that are related to the client distribution, and the local classification model is trained based on the common features and the personalized features to classify the clients.
[0010] Each client uploads the trained first local model to the consensus node;
[0011] After each client uploads the trained first local model, the consensus node aggregates the first local models to obtain a new round of global model.
[0012] Repeat the above process until each client reaches a convergence state.
[0013] Optionally, each of the aforementioned clients trains a local model using a local training set, including:
[0014] Each client computes its target loss function using its local training set;
[0015] Each client updates its local model parameters using the target loss function, the local model parameters including first local model parameters, second local model parameters, and local classification model parameters;
[0016] The target loss function is constructed based on the local loss function of each client and the constraints between the first local model and the second local model.
[0017] Optionally, the constraints between the first local model and the second local model are:
[0018] Orthogonal constraints.
[0019] Optionally, the target loss function is:
[0020] L=λ*L orth +L CE ;
[0021] Among them, L orth For orthogonal constraint loss, L CE The loss is the local cross-entropy loss, where λ is a hyperparameter used to balance L. orth With L CE The weights between them.
[0022] Optionally, it also includes:
[0023] For clients newly joining federated learning, after downloading the global model for this round, the second local model is pre-trained using the orthogonal constraints.
[0024] Optionally, the local classification model is trained based on the common features and the individualized features, including:
[0025] The common features and the individual features are concatenated to obtain concatenated features, which are then input into the local classification model for training.
[0026] This invention also provides a blockchain-based personalized federated learning device, comprising:
[0027] The client download and training module is used by each client participating in federated learning to download the global model for this round from the consensus node of the blockchain, and use the global model as the first local model. Each client trains a local model using a local training set. The local model includes the first local model, the second local model, and the local classification model.
[0028] The first local model is used to extract common features that are unrelated to the client distribution, the second local model is used to extract personalized features that are related to the client distribution, and the local classification model is trained based on the common features and the personalized features to classify the clients.
[0029] The first local model upload module is used by each client to upload the trained first local model to the consensus node;
[0030] The first local model aggregation module is used to aggregate the first local models to obtain a new round of global model after each client has uploaded the trained first local model.
[0031] The repetitive learning module is used to repeatedly execute the above process until each client reaches a convergence state.
[0032] Optionally, the client download and training module is used by each client to train a local model using a local training set, including:
[0033] Each client computes its target loss function using its local training set;
[0034] Each client updates its local model parameters using the target loss function, the local model parameters including first local model parameters, second local model parameters, and local classification model parameters;
[0035] The target loss function is constructed based on the local loss function of each client and the constraints between the first local model and the second local model.
[0036] Optionally, the constraints between the first local model and the second local model are:
[0037] Orthogonal constraints.
[0038] Optionally, the local classification model is trained based on the common features and the individualized features, including:
[0039] The common features and the individual features are concatenated to obtain concatenated features, which are then input into the local classification model for training.
[0040] This invention provides a blockchain-based personalized federated learning method and apparatus. The method includes: each client participating in the federated learning downloads the global model for the current round from the consensus node of the blockchain, using the global model as a first local model; each client trains a local model using a local training set, the local model including a first local model, a second local model, and a local classification model; wherein, the first local model is used to extract common features unrelated to the client distribution, the second local model is used to extract personalized features related to the client distribution, and the local classification model is trained based on the common features and the personalized features to classify the clients; each client uploads the trained first local model to the consensus node; after all clients have uploaded their trained first local models, the consensus node aggregates the first local models to obtain a new round of the global model; the above process is repeated until each client reaches a convergence state.
[0041] In view of this, the beneficial effects of this invention are:
[0042] In this invention, each client downloads the current-round global model from the blockchain's consensus node and uses it as the first local model. Using a local training set, they train a local model including the first local model, a second local model, and a local classification model. The trained first local model is then uploaded to the consensus node, which aggregates all the first local models to obtain a new round of global model. Because this invention only requires extracting feature information independent of client distribution from the global model aggregation, it reduces the storage and query pressure on the blockchain and improves model performance in highly heterogeneous scenarios.
[0043] Compared to traditional decentralized federated learning systems, this invention utilizes blockchain to achieve decentralization. Clients only need to send their local model to blockchain nodes and download the global model using smart contracts. Therefore, direct communication between clients is unnecessary, reducing communication costs. Due to its distributed, immutable, and traceable characteristics, blockchain is well-suited for decentralized federated learning, enhancing the reliability and scalability of the entire system while further reducing communication costs. Attached Figure Description
[0044] Figure 1 This is a schematic flowchart of the method of the present invention;
[0045] Figure 2 This is a schematic diagram of the client-side training process in this invention;
[0046] Figure 3 This is a schematic diagram of the blockchain-based personalized federated learning method framework in this invention;
[0047] Figure 4 This is a schematic diagram of the structure of an embodiment of the device of the present invention. Detailed Implementation
[0048] Terminology Explanation:
[0049] Federated Learning: Federated learning is a distributed machine learning paradigm whose main purpose is to train machine learning models by pooling data from various parties in a privacy-preserving environment. In this process, the device holding the local dataset and participating in model training is called the client, while the device coordinating the entire training process is called the server. The training flow is as follows: the client uses its local dataset to train its local model, obtains gradient information, and uploads it to the server. The server receives the gradient information uploaded by all clients, aggregates it, and finally sends the aggregated gradient information back to the clients. The clients then use the received gradient information to perform gradient descent and update their local model parameters. Because this process does not involve the transmission of the client's original data, it achieves privacy protection.
[0050] Blockchain: A blockchain can be understood as a distributed ledger running on a public network, with decentralization as its core principle. In a blockchain system, multiple peer nodes are interconnected. When a node receives a transaction request, it broadcasts the transaction to other nodes, and the transaction is verified through a consensus mechanism. Verified transaction requests are recorded by all nodes. Therefore, blockchain systems possess strong reliability and stability; the system will not fail due to the failure of a single node, nor will it be damaged by attacks from a small number of malicious nodes.
[0051] Heterogeneity: Currently, heterogeneity is mainly divided into two types: data heterogeneity, which refers to the significant differences in the distribution of datasets possessed by each client in a federated learning system; and system heterogeneity, which refers to the differences in computing power, bandwidth, and other hardware devices possessed by each client. Both types of heterogeneity pose significant challenges to the traditional federated learning training process. Because the datasets possessed by each client vary considerably—for example, in image classification datasets, each client may have different image categories—the local models trained by different clients will differ significantly. If the server only aggregates a single global model, it will inevitably be unable to find a model suitable for all clients, thus reducing the convergence rate of the global model. System heterogeneity, on the other hand, will cause the model version uploaded by the client to lag behind the current version, resulting in a sharp decline in model performance.
[0052] Personalized Federated Learning: Traditional federated learning aggregates only a single global model from local models uploaded by clients on the server side, minimizing unnecessary computational overhead. However, due to the special case of data heterogeneity, aggregating only a single global model can be very ineffective; on some clients, the global model may perform worse than a model trained solely on local data. Therefore, to ensure that as many clients as possible benefit from federated learning, it is necessary to consider clients with significantly different local datasets separately—that is, to consider personalized federated learning, allowing each client to have both global knowledge and locally customized knowledge. Personalized federated learning can handle scenarios with particularly severe data heterogeneity.
[0053] This invention provides a blockchain-based personalized federated learning method and apparatus to address the technical problems of poor model performance in traditional federated learning when datasets are highly heterogeneous, as well as security and scalability issues.
[0054] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0056] Machine learning has achieved remarkable results in many fields such as image processing and text sentiment analysis, which is inseparable from the support of large-scale datasets. However, building large-scale datasets requires collecting and labeling a large amount of data, which often consumes significant human and financial resources. This makes training machine learning models on large-scale datasets extremely costly. Therefore, model training can only be performed on small-scale datasets. To address this challenge, one could try uploading labeled small datasets to a central server, thereby enabling machine learning model training on large datasets. However, with increasing awareness of data security and the continuous improvement of privacy protection laws and regulations, data sharing between different users has become increasingly difficult, ultimately leading to the data silo problem, where data use is limited to local access.
[0057] Federated learning is an excellent solution to the data silo problem. In a federated learning system, the client uploads local model parameters or gradient information trained on local data to the server. Because the local model contains information about the local dataset, aggregation on the server allows for joint training of the model using multiple datasets, thereby achieving privacy protection.
[0058] In practical applications, the distribution of datasets across different clients can vary significantly, leading to substantial differences in the trained local models. This makes it impossible to aggregate a global model that satisfies most clients during the aggregation phase. Therefore, it is necessary to research personalized federated learning mechanisms to train a personalized model for each client. Simultaneously, globally shared information should be introduced during local training to guide the process and improve client model performance under highly heterogeneous conditions. In real-world federated learning scenarios, clients may join the learning process midway. To enable these clients to train better models more quickly, appropriate mechanisms need to be designed to facilitate rapid convergence.
[0059] In traditional federated learning systems, a crucial role is played by the server, which coordinates the entire federated training process and aggregates the global model. However, due to the lack of trust and single point of failure of the central server—meaning that the entire federated learning system cannot continue to operate when the server fails or is attacked—security and scalability issues arise. Blockchain, as a decentralized technology, can solve these security and scalability problems associated with servers.
[0060] Traditional federated learning systems consist of a server and multiple clients, each with its own communication connection to the server, but no direct communication between the clients themselves. However, a single server is prone to single point of failure. Therefore, in-depth research into decentralized federated learning is necessary. Decentralized federated learning removes the server and, through appropriate communication protocol design, enables direct communication between clients, essentially building a peer-to-peer network. While this solves the single point of failure problem, it also introduces excessive communication overhead and results in low convergence efficiency.
[0061] In recent years, there has been some research on personalized federated learning. This research mainly focuses on finding clients with similar dataset distributions through models uploaded by clients, aggregating these clients, and thus achieving personalization.
[0062] A personalized federation scheme most similar to this invention is clustered federation. Its basic idea is to cluster the client models received by the server. Clients whose models are clustered into a single cluster often have similar dataset distributions. Thus, by aggregating models within each cluster, multiple global models can be obtained. Models from a particular cluster are then distributed to the clients corresponding to those models for further training.
[0063] Personalized federated learning has attracted attention from both industry and academia, and many excellent personalization mechanisms have emerged. However, these mechanisms primarily study the relationship between models and client distributions, suggesting that models with similar distributions correspond to clients with similar data distributions, and then cluster the models trained on clients with similar distributions. However, these inventions have drawbacks. Before training, certain hyperparameters in the clustering algorithm, such as the number of clusters, need to be specified, making them unsuitable for constantly changing external environments. Furthermore, a significant drawback of clustering methods is that even with similar client distributions, the trained model parameters can still differ considerably, especially in the early stages of training. Finally, maintaining multiple global models increases the server's memory and computational requirements. Therefore, this method is difficult to apply in real-world scenarios.
[0064] The purpose of this invention is to propose a novel personalized federated learning mechanism. The client-trained model is divided into two parts: one part extracts features unrelated to the client's dataset distribution, and the other part extracts personalized features related to the client's dataset distribution. These two parts of features are then fused for classifier training. When uploading a model, the client only uploads the model that extracts features unrelated to the client's dataset distribution, thus achieving the goal of personalized federated learning.
[0065] Furthermore, traditional federated learning requires a central service area to coordinate the training process, which raises security and scalability issues, necessitating the consideration of decentralized federated learning. Blockchain, with its distributed, immutable, and traceable characteristics, is well-suited for decentralized federated learning, enhancing the overall system's reliability and scalability while reducing communication costs.
[0066] Please see Figure 1 This invention provides an embodiment of a blockchain-based personalized federated learning method, comprising:
[0067] S100: Each client participating in federated learning downloads the global model for this round from the consensus node of the blockchain, uses the global model as the first local model, and each client trains a local model using the local training set. The local model includes the first local model, the second local model, and the local classification model.
[0068] The first local model is used to extract common features that are unrelated to the client distribution, the second local model is used to extract personalized features that are related to the client distribution, and the local classification model is trained based on the common features and the personalized features to classify the clients.
[0069] S200: Each client uploads the trained first local model to the consensus node;
[0070] S300: After each client uploads the trained first local model, the consensus node aggregates each first local model to obtain a new round of global model;
[0071] S400: Repeat the above process until each client reaches a convergence state.
[0072] In a decentralized environment, since there is no client to coordinate the entire federated training process, each client acts as a peer node. Using the common Gossip protocol to design the communication protocol between each client would significantly increase communication overhead, which is clearly unsuitable. Considering the decentralized nature of blockchain, a decentralized federated learning system can be implemented based on blockchain, thus reducing communication overhead even in a decentralized environment.
[0073] In this embodiment, before starting to train the local model, the local client first downloads the global model stored on the blockchain from the consensus node, and then trains the local model using the local training set to obtain a trained local model. The local model includes a first local model, a second local model, and a local classification model. The first local model is used to extract common features unrelated to the client distribution, the second local model is used to extract personalized features related to the client distribution, and the local classification model is trained based on the common and personalized features. The trained local classification model is then used to classify the clients. It is understood that the local model parameters include the parameters of the first local model, the parameters of the second local model, and the parameters of the local classification model.
[0074] In this embodiment, each client trains a local model using its local training set, including:
[0075] Each client uses its local training set to calculate its target loss function;
[0076] Each client updates its local model parameters using the target loss function. The local model parameters include the first local model parameters, the second local model parameters, and the local classification model parameters.
[0077] The target loss function is constructed based on the local loss function of each client and the constraints between the first local model and the second local model.
[0078] Specifically, this embodiment of the invention employs personalized federated learning to address the severe data heterogeneity problem. When training the local model on the client side, three models need to be trained: a first local model, a second local model, and a local classification model. The first local model is responsible for extracting common features independent of the client's distribution, and its parameters are denoted as follows: The second local model is responsible for extracting personalized features related to the client distribution. The corresponding parameters of the second local model are denoted as follows. The local classification model is used to classify clients, and the corresponding parameters of the local classification model are denoted as follows:
[0079] Please see Figure 2 When training their local models, each client downloads the global model for the current round from the blockchain node as its first local model. The local training set is then input into this first local model, which extracts common features independent of the client distribution. These common features can be denoted as... The local training set is input into a second local model, which extracts personalized features related to the client distribution. These personalized features can be denoted as...
[0080] To obtain the input features of the local classification model (classifier), common features can be used. With personalized features By splicing the pieces together, we can obtain the splicing features. [] represents the concatenation operation. The concatenated features are used as input features to the local classification model, and the local classification model is trained to obtain the trained local classification model.
[0081] The client is classified using a pre-trained local classification model to obtain the classification result. Let's assume the output of the local classification model is... To ensure the local model can correctly classify the data, cross-entropy is used to optimize the local model parameters; that is, the local loss function for client i is:
[0082] L CE =CE(p (i) ,Y (i) (1)
[0083] Among them, L CE Let Y represent the local loss function for client i, CE represent the cross-entropy loss function, and Y represent the local loss function for client i. (i) This represents the tag information of client i.
[0084] In this embodiment, to ensure that the first and second local models can extract the relevant feature information as much as possible, corresponding constraints need to be added. Considering that these two models should extract feature information from two different levels, the extracted feature information should be orthogonal. Therefore, applying orthogonal constraints is a reasonable choice. The specific form of the orthogonal constraints is as follows:
[0085]
[0086] Among them, L orth For orthogonal constraint loss, The matrix multiplication is represented by the following formula: B T This represents the transpose of matrix B, which is equivalent to changing the element in the i-th row of matrix B into the element in the i-th column. The square of the F-norm of x is expressed by the following formula: Where m and n represent the number of rows and columns of matrix x, respectively. ij Let x represent the element in the i-th row and j-th column of matrix x.
[0087] Therefore, the target loss function for the client is: L = λ * L orth +L CE , where L orth For orthogonal constraint loss, L CELet λ be the local cross-entropy loss, and λ be a hyperparameter used to balance the weights between the two losses. Based on this loss, gradient descent can be used to adjust the parameters. The specific update formula is as follows:
[0088]
[0089]
[0090]
[0091] Where η represents the learning rate, The loss function L(w; X) represents... (i) The partial derivative of X with respect to the local model parameter w (i) The dataset owned by the client.
[0092] In this system, the blockchain nodes act as servers in traditional federated learning systems. Due to the scalability of blockchain, when a blockchain node fails, the model information on that blockchain node has already been consensus-shared with other blockchain nodes. Therefore, when uploading the first local model or downloading the global model, the client only needs to communicate with other blockchain nodes.
[0093] In this embodiment, client i is responsible for training the local model, denoted as w, and its training objective is:
[0094]
[0095] Among them, X (i) Let w be the local training set owned by client i, and L be the target loss function to be optimized. For classification problems, the cross-entropy loss function is generally used. Equation (6) shows that it is necessary to find an optimal local model parameter w such that the loss function L can reach its minimum value, at which point the local model reaches the convergence state.
[0096] In order to obtain the minimum loss value of the target loss function L, this embodiment uses gradient descent to optimize formula (6), and the update formula of the local model parameters is shown in formula (7):
[0097]
[0098] In equation (7), η represents the learning rate. The target loss function L(w; X) represents the target loss function. (i) The partial derivative with respect to the local model parameter w.
[0099] After each client completes the training of its local model, it needs to upload the first local model. In this embodiment of the invention, a smart contract can be used to complete the upload of the first local model and the download of the global model.
[0100] Then, each client uploads its trained first local model to a blockchain node. The blockchain node packages the uploaded first local model into a block and uploads it to the blockchain. The blockchain node reaches a consensus with other nodes, so the first local model uploaded by the client is recognized by all blockchain nodes and can be used. It can be understood that all nodes on the blockchain are consensus nodes, and each client uploads its trained first local model to the consensus node of the blockchain.
[0101] Once all local clients have completed uploading their first local model, a blockchain node (consensus node) queries all the first local models uploaded in this round of federated learning, performs weighted aggregation, and obtains a new round of global model, which overwrites the previous round's global model.
[0102] In a blockchain-based personalized federated learning system, since the model extracted by client i for personalized features is not very useful to other clients, client i only needs to upload the first local model that extracts features independent of the client distribution when uploading its model. No need to upload a second local model
[0103] In this embodiment, each client downloads the global model from the blockchain node and extracts a feature model w that is independent of the client distribution. g (First local model) and a local distribution-related feature extraction model (second local model) are trained together using the local dataset. After training, the first local model is then used with a smart contract. After uploading to the blockchain node, the smart contract verifies that all clients have uploaded the first local model parameters and then performs an aggregation operation. The aggregation method is as follows:
[0104]
[0105] In equation (8), |D i | represents the size of the local training set for the i-th client. Finally, the architecture diagram of this invention is as follows: Figure 3 As shown.
[0106] For newly joined clients in federated learning, after downloading the global model for this round, the second local model can be pre-trained using orthogonal constraints. Since the features extracted by the client distribution-independent feature extraction model are orthogonal to the client-specific feature extraction model, newly joined clients in personalized federated learning can utilize this property to pre-train their personalized models after downloading the global model. This allows for rapid adjustments to the personalized models, thus this embodiment also benefits newly joined clients.
[0107] In this embodiment, to enable the federated learning system to adapt to scenarios with severe heterogeneity, a novel personalized federated learning scheme is proposed, allowing the system to maintain good performance even in data heterogeneous environments. This personalized federated learning scheme reduces the amount of data stored on the blockchain, thus not affecting query efficiency. This embodiment utilizes blockchain technology to implement a decentralized federated learning framework, solving problems such as untrusted servers and vulnerability to attacks that are common in traditional federated learning systems.
[0108] In this embodiment, each client downloads the current round's global model from the blockchain's consensus node and uses it as the first local model. Using the local training set, a local model including the first local model, a second local model, and a local classification model is trained. The trained first local model is then uploaded to the consensus node, which aggregates all the first local models to obtain a new round's global model. This invention reduces the storage and query pressure on the blockchain by only needing to extract feature information independent of client distribution from the global model aggregation, and can improve model performance in highly heterogeneous scenarios.
[0109] Compared to traditional decentralized federated learning systems, this embodiment utilizes blockchain for decentralization. Clients only need to send their local model to blockchain nodes and download the global model via smart contracts. Therefore, direct communication between clients is unnecessary, reducing communication costs. Blockchain, with its distributed, immutable, and traceable characteristics, is well-suited for decentralized federated learning, enhancing the overall system's reliability and scalability while further reducing communication costs.
[0110] Please see Figure 4 The present invention also provides embodiments of a blockchain-based personalized federated learning device, including:
[0111] The client download and training module 11 is used for each client participating in federated learning to download the global model of this round from the consensus node of the blockchain, and use the global model as the first local model. Each client uses the local training set to train the local model, which includes the first local model, the second local model and the local classification model.
[0112] The first local model is used to extract common features that are unrelated to the client distribution, the second local model is used to extract personalized features that are related to the client distribution, and the local classification model is trained based on the common features and the personalized features to classify the clients.
[0113] The first local model upload module 22 is used by each client to upload the trained first local model to the consensus node;
[0114] The first local model aggregation module 33 is used to aggregate the first local models to obtain a new round of global model after each client has uploaded the trained first local model.
[0115] The repeated learning module 44 is used to repeatedly execute the above process until each client reaches a convergence state.
[0116] The client download and training module is used by each client to train a local model using a local training set, including:
[0117] Preferably, each client uses its local training set to calculate its target loss function;
[0118] Each client updates its local model parameters using the target loss function, the local model parameters including first local model parameters, second local model parameters, and local classification model parameters;
[0119] The target loss function is constructed based on the local loss function of each client and the constraints between the first local model and the second local model.
[0120] Specifically, the constraint condition between the first local model and the second local model is an orthogonal constraint condition.
[0121] Preferably, the local classification model is trained based on the common features and the personalized features, including:
[0122] The common features and the individual features are concatenated to obtain concatenated features, which are then input into the local classification model for training.
[0123] This invention studies a novel personalized federated learning scheme that improves model performance in highly heterogeneous scenarios. Furthermore, considering that storing too much data on the blockchain can affect query efficiency, this invention only needs to extract client-distribution-independent feature models through global aggregation, which can alleviate the storage and query pressure on the blockchain.
[0124] Compared to traditional decentralized federated learning systems, this invention achieves decentralization through blockchain. Clients only need to use smart contracts to send their local models to blockchain nodes and download the global model. Therefore, clients do not need to communicate directly with each other, which reduces communication overhead.
[0125] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0126] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.
[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment of the invention, depending on actual needs.
[0128] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical invention of this invention, or the part that contributes to the prior art, or all or part of the technical invention, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0130] The above embodiments are only used to illustrate the technical invention of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical invention described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical invention to deviate from the spirit and scope of the technical invention of the embodiments of the present invention.
Claims
1. A blockchain-based personalized federated learning method, characterized in that, include: Each client participating in federated learning downloads the global model for this round from the consensus node of the blockchain, uses the global model as the first local model, and each client trains a local model using the local training set. The local model includes the first local model, the second local model, and the local classification model. Specifically, when training a local model, each client inputs the local training set into the first local model and uses the first local model to extract common features unrelated to the client distribution; the local training set is then input into the second local model, and the second local model extracts personalized features related to the client distribution; the local classification model is trained based on the common features and the personalized features to classify the clients. Each client uploads the trained first local model to the consensus node; After each client uploads the trained first local model, the consensus node aggregates the first local models to obtain a new round of global model. Repeat the above process until each client reaches a convergence state; The local classification model is trained based on the common features and the individual features, including: The common features and the individual features are concatenated to obtain concatenated features, which are then input into the local classification model for training.
2. The blockchain-based personalized federated learning method according to claim 1, characterized in that, Each of the aforementioned clients trains a local model using a local training set, including: Each client computes its target loss function using its local training set; Each client updates its local model parameters using the target loss function, the local model parameters including first local model parameters, second local model parameters, and local classification model parameters; The target loss function is constructed based on the local loss function of each client and the constraints between the first local model and the second local model.
3. The blockchain-based personalized federated learning method according to claim 2, characterized in that, The constraints between the first local model and the second local model are: Orthogonal constraints.
4. The blockchain-based personalized federated learning method according to claim 3, characterized in that, The target loss function is: ； in, For orthogonal constraint loss, For local cross-entropy loss, Hyperparameters are used to weigh trade-offs. and The weights between them.
5. The blockchain-based personalized federated learning method according to claim 3, characterized in that, Also includes: For clients newly joining federated learning, after downloading the global model for this round, the second local model is pre-trained using the orthogonal constraints.
6. A blockchain-based personalized federated learning device, characterized in that, include: The client download and training module is used by each client participating in federated learning to download the global model for this round from the consensus node of the blockchain, and use the global model as the first local model. Each client trains a local model using a local training set. The local model includes the first local model, the second local model, and the local classification model. Specifically, when training a local model, each client inputs the local training set into the first local model and uses the first local model to extract common features unrelated to the client distribution; the local training set is then input into the second local model, and the second local model extracts personalized features related to the client distribution; the local classification model is trained based on the common features and the personalized features to classify the clients. The first local model upload module is used by each client to upload the trained first local model to the consensus node; The first local model aggregation module is used to aggregate the first local models to obtain a new round of global model after each client has uploaded the trained first local model. The repetitive learning module is used to repeatedly execute the above process until each client reaches a convergence state. The local classification model is trained based on the common features and the individual features, including: The common features and the individual features are concatenated to obtain concatenated features, which are then input into the local classification model for training.
7. The blockchain-based personalized federated learning device according to claim 6, characterized in that, The client download and training module is used by each client to train a local model using a local training set, including: Each client computes its target loss function using its local training set; Each client updates its local model parameters using the target loss function, the local model parameters including first local model parameters, second local model parameters, and local classification model parameters; The target loss function is constructed based on the local loss function of each client and the constraints between the first local model and the second local model.
8. The blockchain-based personalized federated learning device according to claim 6, characterized in that, The constraints between the first local model and the second local model are: Orthogonal constraints.