Distributed node-level prediction method based on federated learning and storage medium

Abstract

The invention provides a distributed node-level prediction method based on federated learning. Each node performs independent training according to own local data and only retains training data locally; meanwhile, each node participates in aggregation calculation of a global model through a block chain network, and all model updating and prediction results are recorded through a block chain; after local training of each node is completed, prediction is carried out according to model parameters obtained through local training; the central server coordinates the training process to enable each client side to execute effective local training according to the time node state of the client side and the sub-network topology structure, prediction results from all the nodes are aggregated to obtain final global prediction, and the key problems of data privacy, inter-node heterogeneity, performance optimization and the like of federated learning are solved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a distributed node-level prediction method and storage medium based on federated learning. Background Art

[0002] Federated learning is a distributed machine learning method primarily used to protect user data privacy. It allows independent model training and sharing of model parameters on multiple devices without exchanging raw data, thus ensuring user data privacy. Its main application scenarios include: 1) Smart devices and mobile terminal applications: Machine learning model training and prediction on smart devices (such as smartphones and IoT devices). Since these devices typically have low computing power and data cannot be uploaded to the cloud or centralized servers, federated learning provides an effective solution. 2) Healthcare: In healthcare, patient privacy data is extremely sensitive, and data privacy protection requirements are very high. Federated learning enables cross-hospital and cross-institutional medical data sharing and modeling while protecting patient privacy. 3) Financial institutions such as banks and insurance companies hold large amounts of user data, but this data involves user privacy and cannot be directly shared. Federated learning enables joint modeling and data analysis across multiple institutions without exposing users' private information. 4) Industrial Internet and smart manufacturing: Data generated by industrial equipment and sensors is often decentralized, and data privacy and security are crucial. The framework of this invention can be used in the Industrial Internet of Things (IoT), supporting distributed node collaboration and intelligent prediction without the need for centralized data storage.

[0003] However, existing federated learning solutions still face the following challenges:

[0004] 1) Insufficient privacy protection. Although differential privacy and homomorphic encryption are widely used in privacy protection, how to improve model accuracy and training efficiency while maintaining privacy remains a difficult problem. 2) Data heterogeneity between nodes. In practical applications, the data distribution of different nodes may vary greatly, resulting in the inability of traditional federated learning methods to effectively solve the problem of heterogeneous data. Existing technologies generally lack effective collaborative training strategies to deal with this heterogeneity. 3) Low training efficiency. Traditional federated learning methods require synchronous training of each node, which leads to excessively long system training time, especially when the number of nodes is large. Existing technologies have failed to effectively optimize communication and computing efficiency. 4) Lack of decentralized aggregation mechanism. Most existing federated learning frameworks rely on central servers to aggregate models, which easily leads to the risk of single point failure and may lack sufficient credibility and transparency.

[0005] This application proposes a privacy-preserving distributed node-level prediction framework based on federated learning, aiming to address key issues such as data privacy, inter-node heterogeneity, and performance optimization. Summary of the Invention

[0006] This invention proposes a distributed node-level prediction method and storage medium for federated learning, which solves key issues such as data privacy, inter-node heterogeneity, and performance optimization in federated learning algorithms. The technical solution of the invention is achieved as follows:

[0007] The distributed node-level prediction method based on federated learning includes the following steps:

[0008] Each node performs independent training based on its own local data and retains the training data only locally; at the same time, each node participates in the aggregate calculation of the global model through the blockchain network, and all model updates and prediction results are recorded through the blockchain; after completing local training, each node makes predictions based on the model parameters obtained from its local training; the central server coordinates the training process so that each client performs effective local training based on its own time node status and sub-network topology, and then aggregates the prediction results from each node to obtain the final global prediction.

[0009] As a preferred technical solution, after completing local training, each node makes predictions based on the model parameters obtained from its local training. The input data of node i is x i , then the prediction result Calculated by the following formula:

[0010]

[0011] Among them, f((x i θ i )) is the prediction function of the node local model, θ i is the trained model parameter of node i.

[0012] As a preferred technical solution, the central server introduces a weighted average strategy to assign weights to each node based on its reliability, data volume, and computing resources. The weight of node i is ω i , then the global prediction result Calculated by the following formula:

[0013]

[0014] in, is the prediction result of node i, ω i is the weight of the node.

[0015] As a preferred technical solution, a decentralized aggregation mechanism based on blockchain technology is introduced in the global aggregation process. Each node not only calculates the prediction results locally, but also participates in the aggregation calculation of the global model through the blockchain network. All model updates and prediction results will be recorded through the blockchain to ensure the fairness and transparency of the system and avoid data tampering and inconsistency problems.

[0016] As an optimal technical solution, the training process introduces a collaborative training strategy, in which all nodes share the intermediate layer features and gradient information, and reduce the impact of data distribution differences through information collaboration and knowledge transfer; nodes i and j share the gradient information g of local training. i and g j , then the model updates between nodes can be optimized by exchanging gradients:

[0017]

[0018] Among them, α is the coefficient of collaborative training, which is used to balance the gradient contributions of different nodes.

[0019] As a preferred technical solution, each node performs data normalization locally during training. Normalization is performed using the following formula:

[0020]

[0021] Among them, μ i and σ i are the mean and standard deviation of the node i data set, x ij is the jth data point of node i. The standardized data can better fit the global model and reduce the differences between data.

[0022] As a preferred technical solution, each node dynamically selects the most suitable model structure based on its data volume, computing power and data characteristics. The selection strategy is as follows: for nodes with large data volume or strong computing resources, a complex neural network structure is adopted, while for nodes with limited computing power, a lightweight model is used.

[0023] As a preferred technical solution, during the training process, each node only uploads the incremental model parameters after local training to the central server, rather than the complete model parameters; the incremental parameters uploaded by node i are Δθ i , then the global model is updated as follows:

[0024]

[0025] As a preferred technical solution, each node can immediately upload the updated model parameters after local training without waiting for other nodes to complete training; during the model parameter aggregation process, homomorphic encryption technology encrypts the parameters, and the global model update is obtained through decryption.

[0026] A non-transitory storage medium is used to store a program for executing a distributed node-level prediction method based on federated learning.

[0027] Compared with the existing technology, this solution has the following beneficial effects:

[0028] (1) Enhanced privacy protection: By combining differential privacy and homomorphic encryption technology, data privacy is effectively protected while ensuring the accuracy and effectiveness of model training. Even if the data is intercepted during transmission, the data itself cannot be decrypted, thus avoiding the risk of data leakage.

[0029] (2) Efficient inter-node collaborative training. By introducing collaborative training strategies and adaptive model structure adjustment, this paper effectively solves the problem of data heterogeneity between nodes, improving model accuracy and training efficiency. At the same time, by sharing local training results, it compensates for data shortages between different nodes.

[0030] (3) Improve system training efficiency. Incremental learning and asynchronous update mechanisms significantly reduce the synchronization waiting time and communication burden between nodes, and increase the speed of model updates. This helps shorten training time and improve overall efficiency, especially for distributed systems with a large number of nodes.

[0031] (4) Decentralized aggregation mechanism: Blockchain technology is used for decentralized model aggregation, ensuring the transparency and credibility of the aggregation process. Blockchain technology not only enhances the security of the system, but also prevents the risk of single point failure and ensures the robustness of the system.

[0032] (5) High flexibility and applicability: The framework can adapt to the heterogeneity of different data sources and nodes and has wide applicability. Whether in smart devices, healthcare, or finance, it can provide efficient and privacy-protected distributed prediction solutions. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0034] The present invention provides a distributed node-level prediction method based on federated learning, comprising the following steps: each node performs independent training based on its own local data and retains the training data only locally; at the same time, each node participates in the aggregate calculation of the global model through the blockchain network, and all model updates and prediction results are recorded through the blockchain; after completing local training, each node performs prediction based on the model parameters obtained from its local training; the central server coordinates the training process so that each client performs effective local training based on its own time node status and subnetwork topology, and then aggregates the prediction results from each node to obtain the final global prediction.

[0035] Specifically,

[0036] 1. Implementation of the Federated Learning Model

[0037] 1.1 Local training on each node

[0038] In the present invention, each node (client) performs independent training based on its own local data, and the training data is only retained locally to ensure that data privacy is not leaked. Assume that the local data set of node i is Di = {(x i1 ,y i1 ),(x i2 ,y i2 ),…,(x in ,y in )}, where x ij is the input feature, y ij is a label. Each node minimizes the local loss function L i To optimize the model parameters θ i :

[0039]

[0040] During the training process, node iii updates the model parameters using the gradient descent method:

[0041]

[0042] Where η is the learning rate, is the gradient of the loss function.

[0043] 1.2 Differential Privacy Perturbation

[0044] To ensure the privacy of node data, after training, each node applies differential privacy technology to perform parameter perturbations before uploading model parameters. Differential privacy ensures that the training data of a single node is not leaked by adding noise. Specifically, the model parameters θ uploaded by the node are i Perturb with normally distributed noise:

[0045]

[0046] Among them, N(0,σ 2 ) is the normally distributed noise, and σ controls the intensity of the noise to achieve privacy protection.

[0047] 1.3 Homomorphic Encryption Protection

[0048] To further strengthen data protection, this paper uses homomorphic encryption technology. Even during data transmission, the model parameters remain encrypted, and the encrypted model parameters can be aggregated and calculated in the encrypted state. Assuming that the encryption function is E, the encryption parameters of node i are:

[0049]

[0050] Through this encryption method, even if the central server receives encryption parameters uploaded by multiple nodes, it cannot decrypt the data, thus ensuring privacy protection.

[0051] 2.1 Local Prediction per Node

[0052] After completing local training, each node makes predictions based on the model parameters obtained from its local training. Assume that the input data of node i is x i , then the prediction result Calculated by the following formula:

[0053]

[0054] Among them, f((x i θ i )) is the prediction function of the node local model, θ i is the trained model parameter of node i.

[0055] 2.2 Global Prediction and Weighted Aggregation

[0056] In the entire federated learning system, the prediction results from each node are aggregated to obtain the final global prediction. To this end, the present invention introduces a weighted average strategy to assign weights to each node based on its reliability, data volume, and computing resources. Let the weight of node i be ω i , then the global prediction result Calculated by the following formula:

[0057]

[0058] in, is the prediction result of node i, ω i is the weight of the node. The weight distribution mechanism takes into account the computing resources, data volume, and reliability of the node to ensure the accuracy of the aggregation results.

[0059] 2.3 Decentralized aggregation based on blockchain

[0060] To enhance the transparency and credibility of global aggregation, this paper introduces a decentralized aggregation mechanism based on blockchain technology. Under this mechanism, each node not only computes predictions locally but also participates in the global model aggregation calculations through the blockchain network. All model updates and predictions are recorded on the blockchain, ensuring system fairness and transparency while preventing data tampering and inconsistency.

[0061] 3. Solve data heterogeneity between nodes

[0062] 3.1 Collaborative Training Strategy

[0063] In federated learning, the data distribution of each node may be different, resulting in data heterogeneity. To address this challenge, this paper proposes a collaborative training strategy. Nodes share intermediate layer features and gradient information, and through information collaboration and knowledge transfer, reduce the impact of data distribution differences. Assume that node i and node j share the gradient information g of local training. i and g j , model updates between nodes can be optimized by exchanging gradients:

[0064]

[0065] Among them, α is the coefficient of collaborative training, which is used to balance the gradient contributions of different nodes.

[0066] 3.2 Local Data Standardization and Synchronization

[0067] To further alleviate data heterogeneity, the present invention proposes that each node perform data normalization locally. Normalization is performed using the following formula:

[0068]

[0069] Among them, μ i and σ i are the mean and standard deviation of the node i data set, x ij is the jth data point of node i. The standardized data can better fit the global model and reduce the differences between data.

[0070] 3.3 Adaptive model structure adjustment

[0071] To adapt to differences in data characteristics between nodes, this paper further proposes an adaptive model structure adjustment mechanism. Each node dynamically selects the most appropriate model structure based on its data volume, computing power, and data characteristics. For nodes with large data volumes or strong computing resources, a complex neural network structure is adopted, while for nodes with limited computing power, a lightweight model is used.

[0072] 4. Model Update and Privacy Protection

[0073] 4.1 Incremental Learning and Model Update

[0074] In order to reduce the burden of network communication, the present invention introduces an incremental learning mechanism. Each node only uploads the incremental model parameters after local training, rather than the complete model parameters. In this way, the amount of uploaded data is reduced and the global model can be updated efficiently. Assume that the incremental parameter uploaded by node i is Δθ i , then the global model is updated as follows:

[0075]

[0076] This incremental update method effectively reduces the communication overhead during training.

[0077] 4.2 Asynchronous Update Mechanism

[0078] To improve training efficiency, this paper uses an asynchronous update mechanism. This means that after each node completes local training, it can immediately upload the updated model parameters without waiting for other nodes to complete training. This asynchronous update mechanism significantly improves overall training speed and avoids the overall efficiency degradation of the system caused by delays in certain nodes.

[0079] 4.3 Homomorphic Encryption and Secure Aggregation

[0080] During the model parameter aggregation process, the present invention encrypts the parameters using homomorphic encryption technology to ensure that even if the data is intercepted during transmission, it cannot be decrypted. The model parameters of each node are encrypted and uploaded. During aggregation, the encrypted parameters are used for calculation to finally obtain the global model. The update of the global model is obtained by decryption:

[0081]

[0082] Model aggregation is performed in an encrypted state to avoid the risk of privacy leakage.

[0083] Compared with the prior art, this application has the following technical advancements:

[0084] 1) By combining differential privacy and homomorphic encryption technologies, data privacy is effectively protected while ensuring the accuracy and effectiveness of model training. Even if the data is intercepted during transmission, the data itself cannot be decrypted, thus avoiding the risk of data leakage.

[0085] 2) By introducing a collaborative training strategy and adaptive model structure adjustment, this paper effectively solves the problem of data heterogeneity between nodes, improving model accuracy and training efficiency. At the same time, by sharing local training results, it compensates for data shortages between different nodes.

[0086] 3) Incremental learning and asynchronous update mechanisms significantly reduce synchronization latency and communication burdens between nodes, speeding up model updates. This helps shorten training time and improve overall efficiency, especially for distributed systems with a large number of nodes.

[0087] 4) Using blockchain technology for decentralized model aggregation ensures transparency and credibility of the aggregation process. Blockchain technology not only enhances system security but also prevents the risk of single point failure, ensuring system robustness.

[0088] 5) The framework is adaptable to the heterogeneity of different data sources and nodes, and has broad applicability. Whether in smart devices, healthcare, or finance, it can provide efficient, privacy-preserving distributed prediction solutions.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed node-level prediction method based on federated learning, characterized by: The following steps are involved: Each node performs independent training based on its own local data and retains the training data locally. At the same time, each node participates in the aggregate calculation of the global model through the blockchain network. All model updates and prediction results are recorded on the blockchain. After completing local training, each node makes predictions based on the model parameters obtained from its local training. The central server coordinates the training process so that each client can perform effective local training based on its own time node status and sub-network topology, and then aggregate the prediction results from each node to obtain the final global prediction.

2. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: After completing local training, each node makes predictions based on the model parameters obtained from its local training. The input data of node i is x i , then the prediction result Calculated by the following formula: Among them, f((x i θ i )) is the prediction function of the node local model, θ i is the trained model parameter of node i.

3. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: The central server introduces a weighted average strategy to assign weights to each node based on its reliability, data volume, and computing resources. The weight of node i is ω i , then the global prediction result Calculated by the following formula: in, is the prediction result of node i, ω i is the weight of the node.

4. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: A decentralized aggregation mechanism based on blockchain technology is introduced in the global aggregation process. Each node not only calculates the prediction results locally, but also participates in the aggregation calculation of the global model through the blockchain network. All model updates and prediction results will be recorded through the blockchain to ensure the fairness and transparency of the system and avoid data tampering and inconsistency.

5. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: The training process introduces a collaborative training strategy, in which all nodes share intermediate layer features and gradient information. Through information collaboration and knowledge transfer, the impact of data distribution differences is reduced; nodes i and j share the gradient information g of local training. i and g j , then the model updates between nodes can be optimized by exchanging gradients: Among them, α is the coefficient of collaborative training, which is used to balance the gradient contributions of different nodes.

6. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: During the training process, each node performs data normalization locally. Normalization is performed using the following formula: Among them, μ i and σ i are the mean and standard deviation of the node i data set, x ij is the jth data point of node i. The standardized data can better fit the global model and reduce the differences between data.

7. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: Each node dynamically selects the most suitable model structure based on its data volume, computing power and data characteristics. The selection strategy is as follows: for nodes with large data volumes or strong computing resources, a complex neural network structure is adopted, while for nodes with limited computing power, a lightweight model is used.

8. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: During the training process, each node only uploads the incremental model parameters after local training to the central server, rather than the complete model parameters; the incremental parameters uploaded by node i are Δθ i , then the global model is updated as follows:

9. The distributed node-level prediction method based on federated learning according to claim 1, characterized in that: After local training, each node can immediately upload the updated model parameters without waiting for other nodes to complete training. During the model parameter aggregation process, homomorphic encryption technology encrypts the parameters, and the global model update is obtained through decryption.

10. A non-temporary storage medium, characterized in that: It is used to store a program for executing the distributed node-level prediction method based on federated learning as described in claims 1 to 9.

Images