Blockchain and knowledge distillation based heterogeneous federated learning system and method

Abstract

The application provides a heterogeneous federated learning system and method based on blockchain and knowledge distillation, and relates to the technical field of federated learning and blockchain cross. The method comprises the following steps: packaging and uploading task data of a federated learning task to IPFS, generating task metadata and broadcasting; obtaining the task data package and analyzing the same to obtain the CID of an initial global model; a client obtains the CID of the current global model, performs local minority class weighted knowledge distillation training, obtains local model update parameters, generates a transaction package, submits the same to a consensus node for verification, and then broadcasts; the consensus node performs two-stage robust aggregation based on abnormal elimination and weighted aggregation on the received transaction package, generates new round global model update parameters, uploads the same to IPFS, further generates a candidate block, verifies and signs the same by a consensus committee, writes the CID of the new round global model into a ledger, and completes the current round iteration. The application realizes decentralized and safe and reliable heterogeneous federated learning collaborative training.

Description

Technical Field

[0001] This invention relates to the field of federated learning and blockchain technology, and in particular to a heterogeneous federated learning system and method based on blockchain and knowledge distillation. Background Technology

[0002] Federated Learning (FL), a distributed, privacy-preserving machine learning paradigm, allows clients to collaboratively train a shared model without sharing their original private data. Traditional federated learning methods (such as FedAvg) typically rely on a central server to periodically collect and aggregate the model parameters trained by each client. To stabilize local training and overcome drastic drift in the parameter space, researchers have proposed using Knowledge Distillation (KD) to impose regularization constraints on the client-side local models. In this model, the global model acts as the teacher model, providing soft-label guidance to the clients, while the client-side local models act as student models, mimicking the teacher model's predictions on their local private datasets or specific unlabeled public datasets. This approach ensures both privacy and collaborative optimization across heterogeneous clients.

[0003] Meanwhile, due to the distributed and anonymous nature of federated learning, it is highly vulnerable to attacks from malicious Byzantine clients. Malicious actors can degrade the performance of the global model by tampering with model updates or poisoning data labels. To ensure the reliability of collaborative training, existing Byzantine robust federated learning frameworks mainly rely on anomaly detection algorithms based on statistical geometric features in the parameter space. Representative methods include: Krum (selecting updates based on the nearest majority neighbor according to Euclidean distance), Median (taking the median of each parameter dimension), and Trimmed Mean (excluding extreme values ​​before averaging in each dimension). These existing statistical optimization rules provide a certain degree of security resilience in traditional distributed environments.

[0004] However, in real-world deployment environments that are highly heterogeneous (non-independent and identically distributed, Non-IID) and prone to sophisticated and covert malicious attacks, the above methods suffer from the following serious drawbacks:

[0005] (1) Severe loss of minority class knowledge and deterioration of fairness: When there is a severe label distribution skew in the client's local data, some categories have extremely abundant samples (majority class), while others have extremely scarce samples (minority class). Traditional knowledge distillation algorithms treat all categories uniformly during distillation, causing the global teacher model to be dominated by majority class information. The client's local model will severely overfit the majority class and forget minority class features during training. This systematic bias is further amplified in parameter aggregation, causing the aggregated teacher model to send back more distorted knowledge to the client, forming a vicious cycle. This causes the system's generalization prediction ability for the minority class to drop sharply, greatly damaging the fairness of the system.

[0006] (2) Parameter space defenses struggle to effectively identify advanced malicious perturbations: In highly heterogeneous environments, honest clients exhibit significant statistical divergence in their normal model updates due to inherent differences in local data distribution. This naturally benign heterogeneous update is easily confused with malicious attack updates. Existing robust aggregation mechanisms (such as Krum and TrimmedMean) are entirely limited to distance measurement in low- or high-dimensional parameter geometric spaces, completely ignoring the semantic consistency of the model output layer. This allows malicious attackers to bypass pure geometric distance checks by injecting carefully constructed high-dimensional biased perturbations, hiding their attack behavior within the parameter space, and then spreading poisoned features to all honest clients through knowledge distillation, causing systemic collapse.

[0007] (3) Centralized architecture suffers from single point of failure and trust bottleneck: Traditional federated learning heavily relies on a centralized aggregation server to execute algorithms and detect anomalies. This not only introduces the potential security risk of single point of failure, but also easily causes performance and communication bottlenecks for the central node in large-scale client scenarios. In addition, centralized servers cannot provide a trust audit record with long-term traceability, transparency and immutability for the complete evolution of model updates and the anomaly detection process.

[0008] In summary, existing federated learning frameworks suffer from inherent problems when facing data heterogeneity and malicious attacks, including loss of minority class knowledge, failure of parameter space defense, and centralized trust bottlenecks. Particularly in the typical application scenario of cross-hospital collaborative assisted diagnosis, the differences in geographical location and specialty focus among hospitals lead to extreme skewed label distributions in medical image data. For example, there are numerous images of common diseases, but very few images of rare diseases (minority classes). Traditional federated learning, when processing such physically attributed image data, often suffers from severe degradation in the global model's ability to represent rare categories due to loss of minority class knowledge. Furthermore, the sensitivity of medical data demands extremely high security; if a medical node is attacked and uploads compromised model parameters, the reliability of the entire medical diagnostic system will be directly threatened. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention forces the model to focus on minority class features through a minority class weighted knowledge distillation mechanism, combines a two-stage robust parameter filter with direction verification and semantic consistency to intercept high-dimensional malicious perturbations, and seamlessly integrates into the distributed consensus layer of a blockchain. This invention proposes a heterogeneous federated learning system and method based on blockchain and knowledge distillation, aiming to overcome the deficiencies of existing federated learning technologies, such as low minority class generalization accuracy when label distribution is skewed, difficulty in resisting advanced covert Byzantine attacks due to single-dimensional parameter space review, and single point of failure and lack of trust auditing in centralized servers. The goal is to achieve decentralized, transparent, traceable, and highly reliable secure collaborative training of heterogeneous federated learning.

[0010] On the one hand, this invention proposes a heterogeneous federated learning system based on blockchain and knowledge distillation, the system comprising:

[0011] The task publisher initiates federated learning tasks, packages and uploads task data to the InterPlanetary File System (IPFS) and obtains the Content Addressing Hash (CID); based on the CID, it generates metadata for the federated learning task and broadcasts it to the blockchain network; the task data includes an unlabeled public dataset and an initial global model.

[0012] A blockchain network composed of several distributed consensus nodes is used to receive metadata and provide the client with the content addressing hash (CID) of the current global model; receive transaction packets submitted by the client, and the consensus nodes perform a two-stage aggregation based on anomaly removal and weighted aggregation on the transaction packets. The aggregation results are verified based on the unlabeled public dataset, generating a new round of global model update parameters and uploading them to IPFS. The content addressing hash (CID) of the new round of global model is obtained and written to the decentralized ledger.

[0013] Several clients are used to obtain the Content Addressing Hash (CID) of the current global model from the blockchain network, and based on the CID, train a local model using the minority class weighted knowledge distillation (MWD) algorithm to obtain the local model update parameters, generate a transaction package, and submit it to the blockchain network.

[0014] The task publisher, blockchain network, and client all interact with the InterPlanetary File System (IPFS) via a data communication link; the client also communicates with the blockchain network via a Remote Procedure Call (RPC) interface.

[0015] On the other hand, this invention proposes a heterogeneous federated learning method based on blockchain and knowledge distillation, which includes the following process:

[0016] The task publisher initiates a federated learning task, packages the task data and uploads it to the InterPlanetary File System (IPFS), and generates metadata for the federated learning task.

[0017] The metadata of the federated learning task is broadcast to each consensus node in the blockchain network. Each consensus node retrieves the task data packet from the InterPlanetary File System (IPFS) based on the metadata and parses it to obtain the content addressing hash (CID) of the initial global model.

[0018] Based on the content addressing hash (CID) of the initial global model, initialize the global iteration rounds and execute the following federated learning iteration process until the preset termination condition is met, thus completing the federated learning task:

[0019] For any client, obtain the content addressing hash (CID) of the current global model from the consensus node that the client is connected to, and perform local minority class weighted knowledge distillation training based on the content addressing hash (CID) to obtain the local model update parameters;

[0020] The client generates a transaction package based on the updated parameters of the local model, submits the transaction package to the consensus node connected to the client, and the consensus node verifies it. The verified transaction package is then broadcast to the blockchain network.

[0021] When any consensus node in the blockchain network receives a preset number of transaction packets, the consensus node performs a two-stage robust aggregation based on anomaly removal and weighted aggregation on the received transaction packets and verifies the aggregation result, generates a new round of global model update parameters and uploads them to the InterPlanetary File System (IPFS).

[0022] Obtain the Content Addressing Hash (CID) of the new global model from the InterPlanetary File System (IPFS) and generate candidate blocks;

[0023] Based on a verifiable random function of stake, a consensus committee is selected from the consensus nodes of the blockchain network. The consensus committee verifies and signs the candidate blocks. After verification, the content addressing hash (CID) of the new global model is written into the decentralized ledger, completing the current global iteration round.

[0024] Furthermore, the metadata includes: a unique task identifier (Task ID), a content addressing hash (CID) of the task data packet stored in IPFS, a model architecture description, training configuration parameters, and the digital signature of the task publisher.

[0025] Furthermore, each consensus node obtains task data packets from the InterPlanetary File System (IPFS) based on metadata, including:

[0026] For any consensus node, the consensus node receives and parses the metadata of the federated learning task to obtain the content addressing hash (CID) of the task data packet;

[0027] The consensus node downloads the task data packet from the InterPlanetary File System (IPFS) based on the content address hash CID of the task data packet.

[0028] Based on the task data package, maintain a local copy of the unlabeled public dataset on the consensus node;

[0029] By parsing the task data packet, the content addressing hash (CID) of the initial global model is obtained.

[0030] Furthermore, for any client, the content addressing hash (CID) of the current global model is obtained from the consensus node accessed by the client, and based on the content addressing hash CID, local minority class weighted knowledge distillation training is performed to obtain local model update parameters, including:

[0031] For any client Connecting the blockchain network with the client The nearest consensus node is used as the access node, and the content addressing hash (CID) of the current global model is obtained from that access node.

[0032] Based on the content addressing hash (CID) of the current global model, download the model parameters of the current global model from the InterPlanetary File System (IPFS), construct the current global model based on the downloaded model parameters, and use it as the teacher model;

[0033] Guided by the teacher model, initialize the local model and utilize the client. Using a local private dataset, we perform local minority class weighted knowledge distillation training on the local model to obtain the local model update parameters.

[0034] Furthermore, the local minority class weighted knowledge distillation training includes:

[0035] Set the local iteration rounds and use the local model as the student model;

[0036] According to the client The system retrieves the class labels of each sample in a local private dataset, counts the number of samples in each class, determines whether each class belongs to the majority or minority class based on the statistical results, and builds a client application based on the determination results. A collection of most classes; and clients A collection of minority classes;

[0037] During each local iteration, the following training steps are performed:

[0038] Input the local private dataset into the teacher model to obtain the teacher model's prediction distribution for different categories, and then base it on the client. majority class set By masking the prediction outputs for most categories, the corrected teacher probability distribution is obtained.

[0039] Input the local private dataset into the student model to obtain the student model's prediction distribution for different categories, and then base it on the client. majority class set By masking the prediction outputs of most categories and the prediction outputs of the categories to which the batch of training samples belong, the corrected student prediction distribution is obtained.

[0040] For any category in the minority class set, count the number of times that category is used by the client. The frequency in the local private dataset is used as the reciprocal of the frequency as the weight of the category;

[0041] Based on the weights of all classes in the minority class set, calculate the KL divergence between the modified teacher probability distribution and the modified student probability distribution, and construct the minority class weighted distillation loss.

[0042] The cross-entropy loss is calculated based on the student model's predicted distribution for different categories and the category labels of each sample in the training set.

[0043] We construct a local overall optimization objective function by weighted summation of minority class weighted distillation loss and cross-entropy loss;

[0044] With the goal of minimizing the local overall optimization objective function, the gradient is calculated and the student model is updated through backpropagation until the full preset local iteration rounds are completed, and the updated local model parameters are obtained.

[0045] Furthermore, the client generates a transaction package based on the local model update parameters, submits the transaction package to the consensus node accessed by the client for verification, and broadcasts the verified transaction package to the blockchain network, including:

[0046] For any client Using the client The digital private key is used to digitally sign the local model update parameters, the client's unique identifier, the local data size, and the current global iteration round index, and generate a transaction package; the transaction package includes: the local model update parameters, the client's unique identifier, the local data size, the current global iteration round index, and the digital signature generated by the client;

[0047] Client Upload the generated transaction package to the client. Access nodes;

[0048] Client Access node obtains client The public key is used to verify the authenticity of the transaction packet based on the elliptic curve digital signature algorithm and to conduct a compatibility and compliance review of the model structure.

[0049] The authenticity verification based on the elliptic curve digital signature algorithm is as follows: the transaction package is digitally signed and verified using the elliptic curve digital signature algorithm.

[0050] The compatibility and compliance review of the model structure is as follows: extract the local model update parameters from the transaction package, and verify whether the hierarchical structure, weight dimension and total number of parameters in the local model update parameters are consistent with the model architecture description in the metadata of the federated learning task.

[0051] If the authenticity verification and compatibility compliance review are passed, the access node will broadcast the transaction package to the blockchain network;

[0052] If either the authenticity verification or the compatibility compliance review fails, the access node will treat the transaction packet as an illegal transaction and immediately discard it; at the same time, the access node will record this illegal transaction event and trigger the reputation deduction mechanism.

[0053] Furthermore, the process of the consensus node performing a two-stage robust aggregation based on anomaly removal and weighted aggregation on the received transaction packets, verifying the aggregation results, generating a new round of global model update parameters, and uploading them to the InterPlanetary File System (IPFS) includes:

[0054] For the current consensus node, extract the local model update parameters of different clients from all transaction packets received by the current consensus node, and perform a two-stage anomaly removal based on directional filtering and semantic filtering.

[0055] The two-stage anomaly removal based on directional filtering and semantic filtering includes:

[0056] Direction filtering stage: The extracted local model update parameters are converted into gradient vectors for each client. All converted gradient vectors constitute the local update gradient set for the current global iteration round. Based on the local update gradient set for the current global iteration round, the global reference direction vector is calculated using the truncated mean method. The cosine similarity between the gradient vector of each client and the global reference direction vector is calculated. All clients with a cosine similarity greater than zero are retained to form the initial client set. For the clients not retained, a multiplicative exponential multipath penalty decay based on reputation score is performed.

[0057] The multiplicative exponential multipath punitive decay based on reputation score is as follows: calculate the product of the client's reputation score and the decay factor, which is the client's decayed reputation score; if the decayed reputation score is lower than the minimum reputation threshold, the client will be restricted from participating in the aggregation process in subsequent global iteration rounds.

[0058] Semantic filtering stage: Load the local models of each client in the initial screening client set and run them on the unlabeled public dataset to obtain the predicted probability distribution output by each local model; for any client in the initial screening client set, calculate the average JS divergence score between that client and other clients in the initial screening client set based on the predicted probability distribution output by each local model, forming a group score set; calculate the absolute median difference of the group score set and perform normal standardization correction to obtain the standardized absolute median difference; calculate the difference between the average JS divergence score and the median of the group score set, and retain clients whose difference is within a preset multiple of the standardized absolute median difference interval to form the filtered client set;

[0059] Adaptive multidimensional weighted aggregation is performed on each client in the filtered client set, and Nesterov acceleration gradient descent with embedded momentum is used to generate a new round of global model update parameters, which are then uploaded to the InterPlanetary File System (IPFS).

[0060] Furthermore, the adaptive multidimensional weighted aggregation of each client in the filtered client set, and the generation of a new round of global model update parameters using the Nesterov accelerated gradient descent method with embedded momentum, includes:

[0061] According to the client Local data scale Calculate the local data size The proportion of all clients in the total data size of the filtered client set is used as the data size weight. ;

[0062] For the client The average JS divergence score is subjected to exponential decay mapping and normalization to obtain the semantic similarity penalty weight. ;

[0063] Weighting based on data size Weights with semantic similarity penalty After weighted summation, multiply by the client's... The current reputation score is used to obtain the adaptive aggregation weight. ;

[0064] Based on adaptive aggregation weights The gradient vectors of each client in the filtered client set are weighted and aggregated to obtain the aggregated gradient. ;

[0065] Based on aggregation gradient Update the momentum term in the current global iteration round. ; the initial momentum term Let it be the zero vector. Then, the momentum term updated in the first round of global iteration is represented as: Starting from the second round of global iteration, updates will be performed according to the following recursive formula:

[0066] ;

[0067] in, Indicates the momentum coefficient; Indicates the first The momentum term updated in each round of global iteration; Indicates the aggregate learning rate;

[0068] Global model parameters for the current global iteration round The momentum term updated in the previous global iteration round and the momentum term updated in the global iteration rounds We perform a weighted summation to obtain the new round of global model update parameters.

[0069] Furthermore, the verifiable random function based on stake selects a consensus committee from the consensus nodes of the blockchain network. The consensus committee verifies and signs candidate blocks. Upon successful verification, the content addressing hash (CID) of the new global model is written into the decentralized ledger, including:

[0070] For any edge node in the blockchain network The edge nodes are consensus nodes in the blockchain network;

[0071] Using the block header hash of the latest block in the decentralized ledger as the random number seed, and utilizing edge nodes... VRF private key Cryptographic calculations are performed on the random number seed of the latest block to generate a... Unique random value uniformly distributed within the interval and the corresponding verifiable credentials;

[0072] If a unique random value If the inequality based on the lottery decision is satisfied, then the edge node... Candidates selected for the current global iteration round:

[0073] Edge nodes selected for the candidate committee Verifiable credentials are broadcast to the blockchain network, and then distributed across the blockchain network, excluding edge nodes. The remaining edge nodes, besides those, utilize the edge nodes public key In a zero-trust environment, a legitimacy verification is performed to confirm that the node's selection result is authentic and valid.

[0074] The consensus committee is composed of all the edge nodes that have passed the verification. Each edge node in the consensus committee independently verifies the candidate block. After the verification is passed, each committee member digitally signs the candidate block.

[0075] Once a candidate block has collected valid digital signatures from all edge nodes in the consensus committee within the specified timeframe, the candidate block will be recorded in the decentralized ledger.

[0076] The beneficial effects of adopting the above technical solution are as follows:

[0077] (1) Solving the problems of local knowledge forgetting and global bias amplification under non-independent and identically distributed unbalanced samples: This invention proposes a minority class weighted distillation design with majority class Logits masking and inverse frequency normalization. This design accurately opens the constraint channel of asymmetric classes in knowledge distillation. By completely cutting off the absolute dominance of majority class information in the knowledge flow, the student model is forced to specialize in receiving and strengthening the dark knowledge of the minority class. At the same time, the true class masking is introduced asymmetrically at the student end to provide pure negative supervision ability, which perfectly solves the problems of local knowledge forgetting and global bias amplification under non-independent and identically distributed unbalanced samples.

[0078] (2) Significantly eliminate class injustice and ensure the stability of minority class prediction: This invention proposes a two-stage robust filtering protocol that combines parameter space orientation test and output space semantic consistency. This protocol completely breaks the limitations of traditional protection that relies solely on high-dimensional geometric space parameter distance review. It cascades a lightweight and fast cosine similarity reputation dynamic decay initial screening (to solve the problem of large-scale parameter anomalies) with a semantic filter based on the predicted output JS divergence and standardized absolute median deviation (MAD) constraints of public dataset (to solve the problem of high-dimensional hidden poisoning perturbations). This not only prevents honest clients from being mistakenly killed by traditional methods due to natural heterogeneous updates, but also builds an absolute blockade against advanced hidden Byzantine parameter poisoning attacks. Through proactive skewed supervision via the Minority-Class Weighted Distillation (MWD) mechanism, this invention fundamentally compensates for the severe sample skew caused by the local physical environment of each client, effectively prevents the global model from forgetting a very small number of marginal feature categories, significantly raises and stabilizes the generalization accuracy limit of minority classes, and significantly improves the class fairness of collaborative deep learning.

[0079] (3) Completely eliminates the risk of single point of failure and provides full-process transparent traceability and auditing: This invention integrates the traceability, distribution, anomaly detection, and confirmation processes of model updates with the immutable blockchain state machine through a decentralized security control collaborative architecture led by a blockchain and VRF dynamic committee. Combined with a VRF-based multi-way random hidden node election mechanism to form a dynamic review committee, it eliminates excessive reliance on a single centralized server, removes single points of failure, and achieves complete transparency in auditing. This decentralized architecture design successfully removes the privileges of centralized aggregators, greatly reduces network bottlenecks in large-scale networking, and ensures that the evolution of all models and reputation decay scoring processes are verifiable, signed, and immutable, providing a system-level trust guarantee that meets stringent security compliance and financial / medical-grade requirements.

[0080] (4) Extremely high generalization prediction accuracy and top-notch defense robustness: In the conventional scenario where there is a high degree of heterogeneity and no malicious attacks, the generalization accuracy of the present invention is 1.04 to 1.96 times that of the mainstream classical and cutting-edge baseline methods; while in the advanced Byzantine adversarial environment where there are as many as 40% malicious attackers and the methods are extremely vicious, the prediction accuracy of the present invention is 1.14 to 3.35 times that of the baseline methods, demonstrating a dominant level of security resilience. Attached Figure Description

[0081] Figure 1 This is a structural diagram of a heterogeneous federated learning system based on blockchain and knowledge distillation in this embodiment;

[0082] Figure 2 This is a flowchart of a heterogeneous federated learning system based on blockchain and knowledge distillation in this embodiment. Detailed Implementation

[0083] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0084] Example 1:

[0085] This embodiment presents a heterogeneous federated learning system based on blockchain and knowledge distillation, such as... Figure 1 As shown, the system includes: a task publisher, a blockchain network, and several clients; the blockchain network consists of several distributed consensus nodes.

[0086] In this embodiment, the task publisher acts as the initiator of the federated learning task, and the client (Worker) acts as the training terminal participating in the federated learning.

[0087] The task publisher is used to initiate federated learning tasks, package and upload task data to the InterPlanetary File System (IPFS) and obtain the Content Addressing Hash (CID); based on the CID, it generates metadata for the federated learning task and broadcasts it to the blockchain network; the task data includes an unlabeled public dataset and an initial global model.

[0088] The blockchain network is used to receive metadata and provide the client with the content addressing hash (CID) of the current global model; receive transaction packets submitted by the client, and have consensus nodes perform a two-stage robust aggregation based on anomaly removal and weighted aggregation on the transaction packets, verify the aggregation results based on an unlabeled public dataset, generate a new round of global model update parameters and upload them to IPFS, obtain the content addressing hash (CID) of the new round of global model and write it into the decentralized ledger.

[0089] In this embodiment, the blockchain network is used to maintain a decentralized ledger and store model metadata. The decentralized ledger records the content addressing hash (CID) of the current global model. This model metadata includes, but is not limited to, the content addressing hash of the global model in IPFS for each round, model version identifiers, and aggregate signature records. Each consensus node in the blockchain network has the ability to execute verification scripts, such as digital signature verification and model structure compliance checks. A consensus committee is elected based on the consensus algorithm, responsible for executing a two-phase robust aggregation algorithm (ByzAgg) based on anomaly removal and weighted aggregation. Consensus nodes communicate with each other via a P2P (peer-to-peer) protocol to broadcast transaction packets, transmit messages during the consensus process, and synchronize the final blocks.

[0090] It should be noted that consensus nodes in a blockchain network play different roles at different stages of participation, such as... Figure 1 As shown, in the two-phase robust aggregation process based on anomaly removal and weighted aggregation, consensus nodes act as aggregators, aggregating and generating candidate blocks. Each candidate block is a data unit to be verified, packaged with the new round of global model update parameters and related transaction data. During the verification phase, consensus nodes act as verifiers, forming a committee responsible for coordinating and ultimately confirming the entire aggregation and verification process. By verifying candidate blocks, they identify and reject candidate blocks contaminated by attackers, jointly maintaining the normal operation of the blockchain network. Furthermore, attackers may also exist in the system, potentially launching malicious attacks on the federated learning process, attempting to include malicious model update parameters in the candidate blocks generated by the aggregators. Verifiers verify the correctness of candidate blocks, identifying and rejecting candidate blocks contaminated by attacks, ensuring the credibility of the global model updates uploaded to the chain.

[0091] The client is used to obtain the Content Addressing Hash (CID) of the current global model from the blockchain network, and based on the CID, to train the local model using the Minority Class Weighted Knowledge Distillation (MWD) algorithm to obtain the local model update parameters, generate a transaction package, and submit it to the blockchain network.

[0092] In this embodiment, the client has a local private dataset and is equipped with a local computing unit for performing local model training using the minority class weighted knowledge distillation (MWD) algorithm. Figure 1 As shown, the client, as a participant, performs federated training, obtains the content addressing hash (CID) of the current global model from the blockchain network, performs local training, and then packages the updated parameters of the local model and provides them to the blockchain network after training is complete.

[0093] The task publisher, blockchain network, and client all interact with the InterPlanetary File System (IPFS) via a data communication link; the client communicates with the blockchain network through a Remote Procedure Call (RPC) interface.

[0094] In this embodiment, the task publisher, blockchain network, and client all interact with the InterPlanetary File System (IPFS) via HTTP API or content routing addressing to store large files such as model parameters in IPFS, while only the content-addressed hash (CID) circulates on the blockchain. The client connects to the nearest blockchain node via RPC (Remote Procedure Call) interface, obtains the latest global model hash value from the node, and submits a transaction package containing local model update parameters and a digital signature to the blockchain network after training is complete.

[0095] In this embodiment, the data transfer relationship during the training process is as follows:

[0096] Downlink direction: The client first obtains the latest recorded content addressing hash (CID) of the global model from its access node, then locates and downloads the complete global model parameters for the current round from IPFS based on the content addressing hash, and finally uses it as a guiding signal for local minority class knowledge distillation MWD training.

[0097] Upstream direction: After the client completes local iteration, it uploads the model update (gradient or parameter difference), which enters the local waiting pool after being verified by the consensus node.

[0098] Aggregation Feedback: Nodes in the consensus committee extract model updates from multiple clients from their local pending pools, perform ByzAgg filtering and weighted aggregation, generate a new round of global models and upload them to IPFS, and at the same time obtain new content addressing hashes and record them in the decentralized ledger, completing a closed loop.

[0099] Example 2:

[0100] This embodiment presents a heterogeneous federated learning method based on blockchain and knowledge distillation, implemented using the heterogeneous federated learning system based on blockchain and knowledge distillation described in Embodiment 1. Figure 2 As shown, the method includes the following steps:

[0101] The task publisher initiates a federated learning task, packages the task data and uploads it to the InterPlanetary File System (IPFS), and generates metadata for the federated learning task.

[0102] In this embodiment, the task publisher initiates a federated learning task, which packages and uploads task data, including an unlabeled public dataset and an initial global model, to the InterPlanetary File System (IPFS).

[0103] The task data includes: an unlabeled public dataset and an initial global model.

[0104] In this embodiment, an unlabeled public dataset is constructed using standardized medical feature vectors provided by a publicly available medical image database. A neural network classifier is used as the global model, and the output layer of the neural network classifier employs the Softmax function to map the model output to the probability distribution of each category.

[0105] The metadata includes: Task ID (unique identifier), Content Addressing Hash (CID) of the task data packet stored in IPFS, Model Architecture Description (including network layers and parameter dimensions), Training Configuration Parameters (including learning rate, target round number, and participation threshold), and the digital signature of the task publisher.

[0106] The metadata of the federated learning task is broadcast to each consensus node in the blockchain network. Each consensus node retrieves the task data packet from the InterPlanetary File System (IPFS) based on the metadata and parses it to obtain the content addressing hash (CID) of the initial global model.

[0107] In this embodiment, the metadata of the federated learning task is broadcast in a decentralized blockchain network. The metadata is a descriptive index of the task data packet. This embodiment uses an off-chain storage method for data and an on-chain storage method for hashes / indexes to construct the system.

[0108] Each consensus node retrieves and parses task data packets from the InterPlanetary File System (IPFS) based on metadata to obtain the content addressing hash (CID) of the initial global model, including:

[0109] For any consensus node, the consensus node receives and parses the metadata of the federated learning task to obtain the content addressing hash (CID) of the task data packet.

[0110] The consensus node downloads the task data packet from the InterPlanetary File System (IPFS) by addressing the hash CID based on the content of the task data packet.

[0111] Based on the task data package, maintain a local copy of the unlabeled public dataset on the consensus node.

[0112] By parsing the task data packet, the content addressing hash (CID) of the initial global model is obtained.

[0113] In this embodiment, since the task metadata broadcast by the task publisher is packaged in the form of transactions within the blocks, the consensus node obtains the content addressing hash (CID) of the task data packet by retrieving it from the transaction information (or decentralized ledger) of the blockchain network. It then locates and downloads the specific unlabeled public dataset and initial global model in IPFS. Based on the task data packet, a local copy of the unlabeled public dataset is maintained on the consensus node for subsequent verification.

[0114] Initialize the global iteration rounds based on the content addressing hash (CID) of the initial global model. The federated learning process is then executed in the following iterative manner until the preset termination conditions are met, thus completing the federated learning task.

[0115] For any client, obtain the content addressing hash (CID) of the current global model from the consensus node that the client connects to, and perform local minority class weighted knowledge distillation training based on the content addressing hash (CID) to obtain the local model update parameters.

[0116] In this embodiment, the client participating in the training downloads the current global teacher model from the nearest consensus node via IPFS content hash and performs local regularized collaborative training with minority class weighted regularity constraints on its local private dataset.

[0117] For any client, the content addressing hash (CID) of the current global model is obtained from the consensus node accessed by the client. Based on this CID, local minority class weighted knowledge distillation training is performed to obtain local model update parameters, including:

[0118] For any client Connecting the blockchain network with the client The nearest consensus node is used as the access node, and the content addressing hash (CID) of the current global model is obtained from the access node.

[0119] Based on the content addressing hash (CID) of the current global model, download the model parameters of the current global model from the InterPlanetary File System (IPFS), construct the current global model based on the downloaded model parameters, and use it as the teacher model.

[0120] Guided by the teacher model, initialize the local model and utilize the client. Using a local private dataset, we perform local minority class weighted knowledge distillation training on the local model to obtain the local model update parameters.

[0121] The local minority class weighted knowledge distillation training includes:

[0122] Set the local iteration rounds and use the local model as the student model.

[0123] According to the client The class labels of each sample in the local private dataset are determined, and the number of samples in each class is counted. If any class Number of samples satisfy Then determine the category. Belongs to the majority class and builds the client. majority class set Conversely, the category is determined. It belongs to a minority class and builds a client. minority class set ;in For the client The total number of samples in the local private dataset; For the client The total number of different categories in a local private dataset.

[0124] In this embodiment, each sample in the local private dataset consists of CT or MRI image data or pathological slide features containing human tissue anatomical structural features, and a category label corresponding to the image data or pathological slide features.

[0125] During each local iteration, the following training steps are performed:

[0126] Input the local private dataset into the teacher model to obtain the teacher model's prediction distribution for different categories, and then base it on the client. majority class set By masking the prediction outputs for most categories, we obtain the corrected teacher probability distribution.

[0127] To eliminate the absolute dominance of the majority class and correct the inherent bias of the teacher model, this embodiment employs an alignment masking strategy for teacher and student distributions. Specifically, during the distillation process, the outputs corresponding to the majority class in the teacher model are directly masked, constructing a modified teacher probability distribution that focuses on the minority class, expressed as:

[0128] ;

[0129] in, This indicates that for the input sample In the modified teacher probability distribution, it belongs to category The probability of; Indicates category index, The value ranges from 1 to Positive integers; This indicates the total number of categories involved in the federated learning task. The value of is a positive integer greater than or equal to 2; The teacher model is categorized. The output logit is the raw predicted score before the Softmax function in the output layer of the teacher model. This represents the distillation temperature coefficient, used to control the smoothness of the probability distribution. Its value is a real number greater than 0, and in this embodiment, it is preferably... .

[0130] Input the local private dataset into the student model to obtain the student model's prediction distribution for different categories, and then base it on the client. majority class set By masking the prediction outputs of most categories and the prediction outputs of the categories to which the batch of training samples belong, the corrected student prediction distribution is obtained.

[0131] In this embodiment, to eliminate the conflict between this masking operation and the forced increase in the probability of the target true class by the conventional cross-entropy loss, the target true class (i.e., the class to which the input sample belongs) is also masked in the prediction distribution of the student model, and the remaining majority class non-target logistic values ​​(Logits) are removed. To provide negative supervision information that facilitates discrimination, the corrected student prediction distribution is obtained as follows:

[0132] ;

[0133] in, This indicates that for the input sample In the corrected student prediction distribution, it belongs to category The probability, with a range of . ; The student model represents the category. The output logit is the original predicted score before the Softmax function in the output layer of the student model, and it takes the value of a real number.

[0134] For any category in the minority class set, count the number of times that category is used by the client. The frequency of the data in the local private dataset is used as the reciprocal of the frequency as the weight of the category.

[0135] Based on the weights of all classes in the minority class set, calculate the KL divergence between the modified teacher probability distribution and the modified student probability distribution, and construct the minority class weighted distillation loss.

[0136] This embodiment employs inverse frequency inverse dynamic weighting constraints to calculate the minority class weighted distillation loss. Specifically, it uses empirical frequencies to assign inverse amplification weights to the minority class that are inversely proportional to their occurrence probability, thus weighting the class... The frequency is denoted as , Indicates the client The local private dataset belongs to the category Number of samples; assigning categories The weight is A minority-class weighted distillation loss is constructed by calculating the KL divergence between the corrected teacher probability distribution and the corrected student probability distribution. This loss guides the model to correct class bias and is expressed as:

[0137] ;

[0138] in, This indicates the calculation of the modified teacher probability distribution. With modified student probability distribution The KL divergence between the two is used as the minority class weighted distillation loss.

[0139] The cross-entropy loss is calculated based on the student model's predicted distribution for different categories and the category labels of each sample in the training set.

[0140] We construct a local overall optimization objective function by weighted summation of minority class weighted distillation loss and cross-entropy loss.

[0141] With the goal of minimizing the local overall optimization objective function, the gradient is calculated and the student model is updated through backpropagation until the full preset local iteration rounds are completed, and the updated local model parameters are obtained.

[0142] In this embodiment, the client uses the calculated minority class weighted distillation loss as a regularization constraint to construct the local overall optimization objective function. During the local model parameter update process, the client employs stochastic gradient descent (SGD) or its variants to minimize the local overall optimization objective function. To achieve the objective function, the gradient is calculated via backpropagation, and the parameters of the local student model are iteratively updated. The local overall optimization objective function is then applied. Represented as:

[0143] ;

[0144] in, The cross-entropy loss is based on the local real class labels and is used to ensure the enhancement of basic classification ability. The sufficient learning of the majority class is independently guaranteed by the cross-entropy, while the distillation part is completely biased towards the preservation of minority class features. This represents the predicted distribution of the student model for different categories; Indicates input sample Corresponding real category labels The one-hot encoded vector; in the one-hot encoded vector, the value at the index corresponding to the true class is 1, and the value at the other indices is 0; This represents the distillation weight hyperparameter, used to adjust the contribution weights between the cross-entropy loss and the minority class weighted distillation loss. Its value is a real number greater than 0. In this embodiment, This is used to balance the intensity of local supervised learning and global knowledge transfer.

[0145] The client generates a transaction package based on the updated parameters of the local model, submits the transaction package to the consensus node connected to the client, and the consensus node verifies it. The verified transaction package is then broadcast to the blockchain network.

[0146] In this embodiment, the client uses its own digital private key to digitally sign the generated local model update parameters to prevent forgery, and submits the signed transaction package to the blockchain network. The receiving end first verifies the authenticity of the digital signature and performs a compatibility and compliance review of the model structure. Once approved, the compliant transaction is broadcast and disseminated throughout the network.

[0147] The client generates a transaction package based on the local model update parameters, submits the transaction package to the consensus node it has accessed, and the consensus node verifies it. The verified transaction package is then broadcast to the blockchain network, including:

[0148] For any client Using the client The digital private key is used to digitally sign the local model update parameters, the client's unique identifier, the size of the local private dataset, and the current global iteration round index, and generate a transaction package; the transaction package includes: local model update parameters, client's unique identifier, local data size, current global iteration round index, and the digital signature generated by the client.

[0149] In this embodiment, in order to achieve consensus verification of the blockchain and precise aggregation of federated learning, the transaction package must include the following five components: local model update parameters, such as the flattened gradient vector. Or weight update difference A unique client identifier (Client ID) is used to record contribution levels and update reputation scores. Local data scale That is, the size of the local private dataset, which is used as the data size weight for subsequent weighted aggregation. The necessary basis; the current global iteration round index, used to prevent replay attacks and ensure the timeliness of updates; the digital signature is an encrypted signature of the above content using the client's private key, used to ensure the non-repudiation and integrity of the data.

[0150] Client Upload the generated transaction package to the client. Access nodes.

[0151] Client Access node obtains client The public key is used to verify the authenticity of transaction packets based on the elliptic curve digital signature algorithm and to conduct a compatibility and compliance review of the model structure.

[0152] The authenticity verification based on the elliptic curve digital signature algorithm is performed by digitally signing the transaction package using the elliptic curve digital signature algorithm.

[0153] The compatibility and compliance review of the model structure is as follows: extract the local model update parameters from the transaction package, and verify whether the hierarchical structure, weight dimension and total number of parameters in the local model update parameters are consistent with the model architecture description in the metadata of the federated learning task.

[0154] If the transaction passes the authenticity verification and compatibility compliance review, the access node will broadcast the transaction package to the blockchain network.

[0155] If either the authenticity verification or the compatibility compliance review fails, the access node will treat the transaction packet as an illegal transaction and immediately discard it; at the same time, the access node will record this illegal transaction event and trigger the reputation deduction mechanism.

[0156] In this embodiment, the access node utilizes a client. The public key is used to perform authenticity verification and model structure compatibility compliance checks on transaction packets based on the Elliptic Curve Digital Signature Algorithm (ECDSA). If the authenticity verification and compatibility compliance checks pass, the source of the updated data is ensured to be trustworthy and has aggregation compatibility, and the access node broadcasts the transaction packet to the blockchain network via the P2P protocol. If the authenticity verification and compatibility compliance checks fail, the access node considers the transaction packet an illegal transaction and immediately discards it, refusing to forward it to the P2P network. Simultaneously, the access node records this illegal transaction event and triggers a reputation deduction mechanism, directly impacting the client. Historical dynamics reputation score Perform multiplicative exponential multipath punitive decay.

[0157] This embodiment ensures that blockchain bandwidth resources are not occupied by illegal payloads and initially filters out obvious malicious disturbances by strongly intercepting and punishing non-compliant transactions at the network entry point. Specifically, for non-compliant transactions that fail verification (digital signature error or model structure mismatch), a processing strategy of "immediate discard, rejection of broadcast, and credit downgrade" should be adopted. Direct discard means that the receiving node will not store the transaction packet in the pending transaction pool, nor will it forward it to other nodes in the P2P network, thereby cutting off the propagation of illegal data at the source. Rejection response means that an error code is returned to the sending client, indicating the specific reason for the verification failure (invalid signature or structure mismatch). Reputation penalty means that a reputation assessment mechanism is triggered, directly affecting the client's historical dynamic reputation score. Implement punitive deductions. This effectively deters malicious attackers from launching Denial-of-Service (DoS) attacks by sending large numbers of spam transactions.

[0158] When any consensus node in the blockchain network receives a preset number of transaction packets, the consensus node performs a two-stage robust aggregation based on anomaly removal and weighted aggregation on the received transaction packets, verifies the aggregation result, generates a new round of global model update parameters, and uploads them to the InterPlanetary File System (IPFS).

[0159] In this embodiment, each consensus node in the decentralized blockchain network receives transaction packets propagated across the entire network via a P2P protocol. Each consensus node has a local processing pool for storing the transaction packets received by that node. When any consensus node collects a preset number of compliant local updates in its local processing pool (i.e., when the number of client updates in a consensus node's local processing pool reaches a preset participation threshold, such as 5% client participation in federated learning, which means 10 updates out of 200 clients), the current round's proposing node (i.e., the aggregator) triggers subsequent algorithms, including a two-stage anomaly removal protocol composed of directional filtering and semantic filtering, and weighted aggregation. The number refers to the total number of verified transaction packets stored in the node's local processing pool after network-wide P2P broadcast synchronization, corresponding to different client IDs.

[0160] The process involves the consensus node performing a two-stage robust aggregation based on anomaly removal and weighted aggregation on the received transaction packets, verifying the aggregation results, generating new global model update parameters, and uploading them to the InterPlanetary File System (IPFS).

[0161] For the current consensus node, extract the local model update parameters of different clients from all transaction packets received by the current consensus node, and perform a two-stage anomaly removal based on directional filtering and semantic filtering.

[0162] The two-stage anomaly removal based on directional filtering and semantic filtering includes:

[0163] Direction filtering stage: The extracted local model update parameters are converted into gradient vectors for each client. All converted gradient vectors constitute the local update gradient set for the current global iteration round. Based on the local update gradient set for the current global iteration round, the global reference direction vector is calculated using the truncated mean method. The cosine similarity between the gradient vector of each client and the global reference direction vector is calculated. All clients with a cosine similarity greater than zero are retained to form the initial client set. For the clients not retained, a multiplicative exponential multipath penalty decay based on reputation score is performed.

[0164] In this embodiment, the two-stage anomaly removal protocol, which combines directional filtering and semantic filtering, includes the following two stages: The first stage (coarse-grained directional filtering) is as follows: After collecting a sufficient number of transaction packets, the consensus node first extracts the local model update parameters of each client from the transaction packets and converts them into flattened gradient vectors. , Containing multiple real-valued elements, the vector dimension of which is consistent with the total parameter dimension of the global model, thus constructing the local update gradient set for this iteration. For the local update gradient set under the current global iteration, the element-wise trimmed mean is calculated as the global reference direction vector for system noise immunity, expressed as:

[0165] ;

[0166] in, This is the global reference direction vector; This is a truncated mean function.

[0167] Then, the cosine similarity between the gradient vector of each client and the global reference direction vector is calculated:

[0168] ;

[0169] in, For the client The cosine similarity between the gradient vector and the global reference direction vector, with values ​​ranging from 1 to 2. This is used to quantify the angular proximity between the client's update direction and the group's robust reference direction; only positive alignment consistency is displayed. The client.

[0170] At the same time, each consensus node maintains the historical dynamic reputation score of each client. If the client If the direction filtering fails in the current global iteration round, then for the client... The reputation score is subject to multiplicative exponential multi-way punitive decay. Clients whose reputation scores fall below the minimum threshold after decay will be directly rejected in the initial screening, thereby filtering out malicious updates that show significant deviations.

[0171] This embodiment designs a dynamic reputation feedback strategy to address Byzantine attacks, namely, a multiplicative exponential multi-path punitive attenuation based on reputation scores. The specific logic is based on the formula... The multiplicative nature of reputation means that the update of reputation value is based on the current reputation score. This is achieved by multiplying by a decay factor. Exponential decay is characterized by a decay factor containing an exponential term. This means that as the number of failures or parameter configurations change, the reputation score will decrease exponentially. The preset gradual decay target; This represents the maximum number of consecutive failures that can be tolerated.

[0172] The multiplicative exponential multipath punitive decay based on reputation score is as follows: The product of the client's reputation score and the decay factor is calculated as the client's decayed reputation score; if the decayed reputation score is lower than the minimum reputation threshold, the client will be restricted from participating in the aggregation process in subsequent global iteration rounds. Specifically, when the client... The update failed to pass the direction filtering stage (i.e. When the reputation score is reduced, a decay operation is performed. The new reputation score is equal to the original reputation score multiplied by the decay factor. If the diminished reputation score falls below the set minimum reputation threshold... If the client is excluded, the client will be permanently or for a certain period of time excluded from the aggregation process. The minimum reputation threshold... The value ranges from 0.1 to 0.4, and is preferably 0.2 in this embodiment. The setting of the minimum reputation threshold is based on balancing the robustness and inclusiveness of the system: on the one hand, by setting a certain lower limit, hard removal of nodes with consistently abnormal performance is achieved to prevent the cumulative effect of malicious disturbances; on the other hand, by retaining a low minimum fault tolerance space, honest nodes are prevented from being wrongly banned due to occasional update deviations caused by extremely heterogeneous local data, thereby maintaining the diversity of participants in federated learning while ensuring security.

[0173] Semantic filtering stage: Load the local models of each client in the initial screening client set and run them on the unlabeled public dataset to obtain the predicted probability distribution output by each local model; for any client in the initial screening client set, calculate the average JS divergence score between that client and other clients in the initial screening client set based on the predicted probability distribution output by each local model to form a group score set; calculate the absolute median difference of the group score set and perform normal standardization correction to obtain the standardized absolute median difference; calculate the difference between the average JS divergence score and the median of the group score set, and retain clients whose difference is within a preset multiple of the standardized absolute median difference interval to form the filtered client set.

[0174] In this embodiment, to intercept highly deceptive high-dimensional covert attacks in terms of orientation and parametric geometric distance, the second stage (fine-grained semantic output filtering) involves loading and running the local models of each client that passed the initial screening in the first stage on the same unlabeled public dataset, and extracting the predicted probability distribution of their outputs. The JS divergence (Jensen-Shannon Divergence, JS), which possesses symmetry and well-defined upper and lower bounds, is used to rigorously evaluate the semantic inconsistencies at the output level of each model. Specifically, for any client in the initial screening client set, the average JS divergence score between it and other clients in the initial screening client set is calculated. ;in, Indicates client The Jensen-Shannon divergence (JS divergence) between the predicted probability distribution generated on the unlabeled public dataset and the predicted probability distribution output by other clients in the initial screening client set is used to quantify the clients. The degree of output offset in semantic logic, its value range is within between. The closer the value is to 0, the more likely the client's discrimination logic is to converge with the group; the closer the value is to 1, the more likely the client's prediction results have significant semantic anomalies, which may be due to malicious perturbation or severe non-independent identically distributed noise.

[0175] To accurately establish safe boundaries amidst the distribution divergence caused by strong Non-IID, this embodiment introduces a robust statistical discrimination criterion based on Median Absolute Deviation (MAD), calculating the absolute median of the group score set, expressed as:

[0176] ;

[0177] in, The absolute median difference of the group's score set; This indicates taking the median.

[0178] Then, normal standardization correction was applied to the absolute median to obtain:

[0179] ;

[0180] in, The Standardized Median Absolute Deviation (SMD) is a statistic that is comparable to the standard deviation under a standard normal distribution, obtained by scaling the dispersion of the original JS divergence score. It is used to construct confidence intervals for anomaly detection. This represents the value of the inverse function (quantile function) of the cumulative distribution function of the standard normal distribution at 0.75. As a scaling constant, approximately 0.67449, its physical meaning represents the distance between the median and the upper quartile in the standard normal distribution. In statistics, this constant is used to normalize the MAD (Multiple Quantitation Difference). It becomes an unbiased estimator of the standard deviation under a normal distribution.

[0181] An update is only officially accepted when the deviation between the client's average JS divergence score and the overall median is strictly controlled within a preset multiple of the standardized absolute median difference range, thus completely eliminating outlier malicious pollution updates at the semantic level.

[0182] In this embodiment, the preset multiplier ranges from 1.5 to 3.0, preferably set to 2.0. This multiplier is set based on establishing a statistically based outlier detection boundary: by selecting a standardized MAD interval of 2.0, the system can mathematically cover approximately 95% of the normal update distribution range. This setting is compatible with natural semantic biases arising from the high heterogeneity of local data (Non-IID), ensuring the effectiveness of honest node contributions; it also precisely truncates hidden malicious perturbations at the distribution edge through a defined confidence upper limit, thereby maximizing the system's immunity to contaminated data while ensuring model training convergence speed.

[0183] Adaptive multidimensional weighted aggregation is performed on each client in the filtered client set, and Nesterov acceleration gradient descent with embedded momentum is used to generate a new round of global model update parameters, which are then uploaded to the InterPlanetary File System (IPFS).

[0184] In this embodiment, for the finally filtered clients, the weights of their local data size, output semantic similarity penalty, and historical reputation score are comprehensively considered to form the final adaptive aggregation weight coefficients. The model parameters are then weighted and aggregated to obtain the aggregation gradient. The next-generation global model is calculated using Nesterov accelerated gradient descent with embedded momentum, and uploaded to the IPFS system to generate a new content hash address. This new content hash address serves as the unique retrieval credential for the global model in IPFS during the current iteration round and is written into the candidate block payload to be generated. This payload guides the client in retrieving the updated global model at the start of the next iteration round.

[0185] The process of adaptively weighting and aggregating each client in the selected client set, and generating new global model update parameters using Nesterov accelerated gradient descent with embedded momentum, includes:

[0186] According to the client Local data scale Calculate the local data size The proportion of all clients in the total data size of the filtered client set is used as the data size weight. .

[0187] For the client The average JS divergence score is subjected to exponential decay mapping and normalization to obtain the semantic similarity penalty weight. .

[0188] Weighting based on data size Weights with semantic similarity penalty After weighted summation, multiply by the client's... The current reputation score is used to obtain the adaptive aggregation weight. .

[0189] In this embodiment, adaptive aggregation weights The calculation method is as follows:

[0190] Data size weight Based on the amount of data used for training locally on the client, calculate its proportion in the total amount of data from all selected clients, expressed as:

[0191] ;

[0192] in, For the filtered client set; For the client The size of the local data; the larger the data volume, the higher the basic weight.

[0193] Semantic similarity penalty weight The semantic similarity penalty weight is used to measure the quality of client updates. Its value is based on the client's average JS divergence score, which is then subjected to exponential decay mapping and normalization. The core idea of ​​this design is that the larger the divergence, the further the client's update deviates from the group's semantics, and the faster the corresponding weight decays, thus effectively suppressing heterogeneous noise. The specific calculation formula is as follows:

[0194] ;

[0195] in, This is a sensitivity parameter used to control the penalty for updates with high bias, and ; For the client The average JS divergence score.

[0196] Based on data size weight and semantic similarity penalty weight Adjusting historical reputation: using balancing factors Weights based on data size and semantic similarity penalty weight Perform a weighted sum and multiply the weighted sum by the client's current reputation score to obtain the adaptive aggregation weight. , represented as:

[0197] ;

[0198] Adaptive aggregate weights This comprehensively reflects the client's physical data contribution, logical semantic consistency, and long-term credit level, ensuring the fairness and anti-interference ability of the aggregation process.

[0199] Based on adaptive aggregation weights The gradient vectors of each client in the filtered client set are weighted and aggregated to obtain the aggregated gradient. .

[0200] In federated learning, applying Nesterov accelerated gradient descent with embedded momentum to the global model update of the server (or consensus node) can effectively alleviate the client drift problem caused by data heterogeneity (Non-IID) and improve the convergence stability of the global model.

[0201] Specifically, the method for generating a new round of global model update parameters using the Nesterov accelerated gradient descent method with embedded momentum is as follows:

[0202] Based on aggregation gradient Update the momentum term in the current global iteration round. ; the initial momentum term Let it be the zero vector, then in the first round of global iteration (i.e. The updated momentum term is represented as follows (at time) Starting from the second round of global iteration ( If ), then update according to the following recursive formula:

[0203] ;

[0204] in, Indicates the first The momentum term updated in each round of global iteration, i.e., the momentum buffer vector; This represents the momentum coefficient, used to control the influence of historical gradients on the current update. Its value typically ranges from [value missing]. In this embodiment, the value is preferably 0.9. Indicates the first The momentum term updated in each round of global iteration; This represents the aggregate learning rate, which is a real number greater than 0. In this embodiment, it is preferably 1.0.

[0205] Global model parameters for the current global iteration round The momentum term updated in the previous global iteration round and the momentum term updated in the global iteration rounds We perform a weighted summation to obtain the new round of global model update parameters.

[0206] In this embodiment, the global model parameters of the current global iteration round are used. The momentum term updated in the previous global iteration round and the momentum term updated in the global iteration rounds The new global model is calculated according to the following formula:

[0207] ;

[0208] Obtain the Content Addressing Hash (CID) of the new round of global model from the InterPlanetary File System (IPFS) and generate candidate blocks.

[0209] Based on a verifiable random function of stake, a consensus committee is selected from the consensus nodes of the blockchain network. The consensus committee verifies and signs the candidate blocks. After verification, the content addressing hash (CID) of the new global model is written into the decentralized ledger, completing the current global iteration round.

[0210] To achieve efficient and attack-resistant block consensus in a decentralized edge environment, this invention utilizes a stake-based verifiable random function (VRF) to select a subset of edge nodes to form a consensus committee in a private, non-interactive, and Sybil-resistant manner, which is responsible for verifying and recording subsequent candidate blocks.

[0211] It should be noted that, in this embodiment, an edge node refers to a physical entity deployed at the network edge (such as a base station, roadside unit, or edge server) and possessing computing and storage capabilities. These edge nodes collectively constitute a decentralized blockchain network and logically function as blockchain consensus nodes.

[0212] The verifiable random function based on stake selects a consensus committee from the consensus nodes of the blockchain network. The consensus committee verifies and signs candidate blocks. Upon successful verification, the content addressing hash (CID) of the new global model is written into the decentralized ledger, including:

[0213] For any edge node in the blockchain network The edge node is a consensus node in the blockchain network.

[0214] Using the block header hash of the latest block in the decentralized ledger as the random number seed, and utilizing edge nodes... VRF private key Cryptographic calculations are performed on the random number seed of the latest block to generate a... Unique random value uniformly distributed within the interval And the corresponding verifiable credentials.

[0215] In this embodiment, the decentralized ledger consists of blocks linked sequentially in chronological order. The block header of each block contains information such as the hash value, timestamp, and random number of the previous block. The block header hash value can be used as a publicly verifiable random seed.

[0216] Each edge node independently runs a self-selection algorithm, secretly checking whether it has been selected to enter the consensus committee for the current round, without knowing the results of others. Based on a verifiable random function (VRF), the edge node uses its VRF private key. For use as a seed for random numbers Cryptographic calculations are performed on the block header hash value of the latest block in the current blockchain ledger to generate a... Unique random value uniformly distributed within the interval And the corresponding verifiable credentials, represented as:

[0217] ;

[0218] in, Represents edge nodes Using the private key to generate a pseudo-random hash value via the VRF algorithm, It possesses uniqueness and unpredictability, and is typically standardized and mapped to real numbers in subsequent steps. This serves as a random score for determining whether an edge node is selected for the consensus committee. This represents a verifiable mathematical proof corresponding to the pseudo-random hash value. Allows other edge nodes in the blockchain network to access the blockchain without knowing the edge node's private key. Under the premise of using its public key for verification To verify the authenticity and validity of the calculation process, thereby preventing nodes from forging lottery results; As a public input seed for VRF computation, its value is the block header hash of the latest block in the blockchain network, ensuring the synchronization and fairness of the lottery logic of all nodes in the network. Indicates edge nodes private key For secret parameters, with It is a verifiable random function operation with common parameters that ensures that, for a given input, its output is deterministic and cannot be chosen by the node itself.

[0219] If a unique random value If the following equity-based lottery determination inequality is satisfied, then the edge node... Candidates selected for the current global iteration round:

[0220] ;

[0221] in, For edge nodes The value of local tokens currently held or converted from historical contributions; The total equity of all edge nodes in the blockchain network; The expected basic size of the consensus committee; To adjust the hyperparameters for the redundancy and activity of the committee, The range of values ​​is In this embodiment, 3 is preferred.

[0222] In this embodiment, in order to ensure that a sufficient number (i.e., at least) are generated in each round of drawing... The system identifies (a number of) legitimate members and uses hyperparameters to... Adjust the overall expected number of selected nodes. Since the selection of each edge node independently follows a Bernoulli distribution, the total number of selected nodes... The mathematical expectation is Combining the constraints of the Chernov bound on independent Bernoulli trials, the probability of failing to assemble a sufficient committee decreases exponentially with the parameters. Under the system's default configuration, the probability of failing to form a committee is less than [missing value]. .

[0223] Edge nodes selected for the candidate committee Verifiable credentials are broadcast to the blockchain network, and then distributed across the blockchain network, excluding edge nodes. The remaining edge nodes, besides those, utilize the edge nodes public key In a zero-trust environment, a legitimacy verification is performed to confirm that the node's selection result is genuine and valid.

[0224] In this embodiment, it is broadcast to the entire network. and Other nodes can use the node's public key. Complete the legitimacy verification in a zero-trust environment.

[0225] In this embodiment, the core advantage of the autonomous lottery algorithm lies in the fact that the probability of an edge node being selected for the committee is directly proportional to its actual stake in the network. Edge node probability of winning the lottery Strictly speaking:

[0226] ;

[0227] in, Represents edge nodes The theoretical probability of being selected as a consensus committee candidate in an autonomous lottery algorithm.

[0228] This probabilistic design, deeply tied to stake, can mathematically eradicate Sybil attacks, a common problem in distributed edge networks. Even if a malicious entity creates a large number of fake node credentials through cloning, virtualization, or other means, the probability of a malicious node being selected for the authoritative committee and controlling the consensus process will remain tightly compressed within an extremely low security threshold because attackers cannot create or control the vast majority of substantial physical stakes across the entire network in a short period of time.

[0229] The consensus committee is composed of all edge nodes that have passed the verification. Each edge node in the consensus committee independently verifies the candidate blocks, including verifying the validity of the CID, the legality of the aggregation result, and the correctness of the block hash.

[0230] After verification, each committee member digitally signs the candidate block.

[0231] Once a candidate block receives valid digital signatures from all edge nodes in the formula committee within the specified timeframe, the candidate block is recorded in the decentralized ledger, and the new round of reliable global model officially takes effect across the entire network.

[0232] Once a candidate block has collected valid digital signatures from all edge nodes in the consensus committee within the specified timeframe, the candidate block will be recorded in the decentralized ledger.

[0233] After the new round of reliable global model officially takes effect across the entire network, the system enters the next round of global iteration. Participating clients retrieve the latest global teacher model from IPFS by searching the hash address of the content recorded in the latest record on the blockchain and start a new round of local minority class weighted knowledge distillation training, thereby achieving continuous evolution of knowledge.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the present invention.

Claims

1. A heterogeneous federated learning system based on blockchain and knowledge distillation, characterized in that: The system includes: The task publisher initiates federated learning tasks, packages and uploads task data to the InterPlanetary File System (IPFS) and obtains the Content Addressing Hash (CID); based on the CID, it generates metadata for the federated learning task and broadcasts it to the blockchain network; the task data includes an unlabeled public dataset and an initial global model. A blockchain network composed of several distributed consensus nodes is used to receive metadata and provide the client with the content addressing hash (CID) of the current global model; receive transaction packets submitted by the client, and the consensus nodes perform a two-stage aggregation based on anomaly removal and weighted aggregation on the transaction packets. The aggregation results are verified based on the unlabeled public dataset, generating a new round of global model update parameters and uploading them to IPFS. The content addressing hash (CID) of the new round of global model is obtained and written to the decentralized ledger. Several clients are used to obtain the Content Addressing Hash (CID) of the current global model from the blockchain network, and based on the CID, train a local model using the minority class weighted knowledge distillation (MWD) algorithm to obtain the local model update parameters, generate a transaction package, and submit it to the blockchain network. The task publisher, blockchain network, and client all interact with the InterPlanetary File System (IPFS) via a data communication link; the client also communicates with the blockchain network via a Remote Procedure Call (RPC) interface.

2. A heterogeneous federated learning method based on blockchain and knowledge distillation, implemented using the heterogeneous federated learning system based on blockchain and knowledge distillation as described in claim 1, characterized in that... This method includes the following steps: The task publisher initiates a federated learning task, packages the task data and uploads it to the InterPlanetary File System (IPFS), and generates metadata for the federated learning task. The metadata of the federated learning task is broadcast to each consensus node in the blockchain network. Each consensus node retrieves the task data packet from the InterPlanetary File System (IPFS) based on the metadata and parses it to obtain the content addressing hash (CID) of the initial global model. Based on the content addressing hash (CID) of the initial global model, initialize the global iteration rounds and execute the following federated learning iteration process until the preset termination condition is met, thus completing the federated learning task: For any client, obtain the content addressing hash (CID) of the current global model from the consensus node that the client is connected to, and perform local minority class weighted knowledge distillation training based on the content addressing hash (CID) to obtain the local model update parameters; The client generates a transaction package based on the updated parameters of the local model, submits the transaction package to the consensus node connected to the client, and the consensus node verifies it. The verified transaction package is then broadcast to the blockchain network. When any consensus node in the blockchain network receives a preset number of transaction packets, the consensus node performs a two-stage robust aggregation based on anomaly removal and weighted aggregation on the received transaction packets and verifies the aggregation result, generates a new round of global model update parameters and uploads them to the InterPlanetary File System (IPFS). Obtain the Content Addressing Hash (CID) of the new global model from the InterPlanetary File System (IPFS) and generate candidate blocks; Based on a verifiable random function of stake, a consensus committee is selected from the consensus nodes of the blockchain network. The consensus committee verifies and signs the candidate blocks. After verification, the content addressing hash (CID) of the new global model is written into the decentralized ledger, completing the current global iteration round.

3. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 2, characterized in that, The metadata includes: Task ID (unique identifier), Content Address Hash (CID) of the task data packet stored in IPFS, Model Architecture Description, Training Configuration Parameters, and the digital signature of the task publisher.

4. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 3, characterized in that, Each consensus node obtains task data packets from the InterPlanetary File System (IPFS) based on metadata, including: For any consensus node, the consensus node receives and parses the metadata of the federated learning task to obtain the content addressing hash (CID) of the task data packet; The consensus node downloads the task data packet from the InterPlanetary File System (IPFS) based on the content address hash CID of the task data packet. Based on the task data package, maintain a local copy of the unlabeled public dataset on the consensus node; By parsing the task data packet, the content addressing hash (CID) of the initial global model is obtained.

5. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 4, characterized in that, For any client, the content addressing hash (CID) of the current global model is obtained from the consensus node accessed by the client. Based on this CID, local minority class weighted knowledge distillation training is performed to obtain local model update parameters, including: For any client Connecting the blockchain network with the client The nearest consensus node is used as the access node, and the content addressing hash (CID) of the current global model is obtained from that access node. Based on the content addressing hash (CID) of the current global model, download the model parameters of the current global model from the InterPlanetary File System (IPFS), construct the current global model based on the downloaded model parameters, and use it as the teacher model; Guided by the teacher model, initialize the local model and utilize the client. Using a local private dataset, we perform local minority class weighted knowledge distillation training on the local model to obtain the updated parameters of the local model.

6. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 5, characterized in that, The local minority class weighted knowledge distillation training includes: Set the local iteration rounds and use the local model as the student model; According to the client The system retrieves the class labels of each sample in a local private dataset, counts the number of samples in each class, determines whether each class belongs to the majority or minority class based on the statistical results, and builds a client application based on the determination results. A collection of most classes; and clients A collection of minority classes; During each local iteration, the following training steps are performed: Input the local private dataset into the teacher model to obtain the teacher model's prediction distribution for different categories, and then base it on the client. majority class set By masking the prediction outputs for most categories, the corrected teacher probability distribution is obtained. Input the local private dataset into the student model to obtain the student model's prediction distribution for different categories, and then base it on the client. majority class set By masking the prediction outputs of most categories and the prediction outputs of the corresponding batch of training samples, the corrected student prediction distribution is obtained. For any category in the minority class set, count the number of times that category is used by the client. The frequency in the local private dataset is used as the reciprocal of the frequency as the weight of the category; Based on the weights of all classes in the minority class set, calculate the KL divergence between the modified teacher probability distribution and the modified student probability distribution, and construct the minority class weighted distillation loss. The cross-entropy loss is calculated based on the student model's predicted distribution for different categories and the category labels of each sample in the training set. We construct a local overall optimization objective function by weighted summation of minority class weighted distillation loss and cross-entropy loss; With the goal of minimizing the local overall optimization objective function, the gradient is calculated and the student model is updated through backpropagation until the full preset local iteration rounds are completed, and the updated local model parameters are obtained.

7. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 6, characterized in that, The client generates a transaction package based on the local model update parameters, submits the transaction package to the consensus node it has accessed, and the consensus node verifies it. The verified transaction package is then broadcast to the blockchain network, including: For any client Using the client The digital private key is used to digitally sign the local model update parameters, the client's unique identifier, the local data size, and the current global iteration round index, and generate a transaction package; the transaction package includes: the local model update parameters, the client's unique identifier, the local data size, the current global iteration round index, and the digital signature generated by the client; Client Upload the generated transaction package to the client. Access nodes; Client Access node obtains client The public key is used to verify the authenticity of the transaction packet based on the elliptic curve digital signature algorithm and to conduct a compatibility and compliance review of the model structure. The authenticity verification based on the elliptic curve digital signature algorithm is as follows: the transaction package is digitally signed and verified using the elliptic curve digital signature algorithm. The compatibility and compliance review of the model structure is as follows: extract the local model update parameters from the transaction package, and verify whether the hierarchical structure, weight dimension and total number of parameters in the local model update parameters are consistent with the model architecture description in the metadata of the federated learning task. If the transaction passes the authenticity verification and compatibility compliance review, the access node will broadcast the transaction package to the blockchain network. If either the authenticity verification or the compatibility compliance review fails, the access node will treat the transaction packet as an illegal transaction and immediately discard it; at the same time, the access node will record this illegal transaction event and trigger the reputation deduction mechanism.

8. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 7, characterized in that, The process involves the consensus node performing a two-stage robust aggregation based on anomaly removal and weighted aggregation on the received transaction packets, verifying the aggregation results, generating new round of global model update parameters, and uploading them to the InterPlanetary File System (IPFS). This includes: For the current consensus node, extract the local model update parameters of different clients from all transaction packets received by the current consensus node, and perform a two-stage anomaly removal based on directional filtering and semantic filtering. The two-stage anomaly removal based on directional filtering and semantic filtering includes: Direction filtering stage: The extracted local model update parameters are converted into gradient vectors for each client. All converted gradient vectors constitute the local update gradient set for the current global iteration round. Based on the local update gradient set for the current global iteration round, the global reference direction vector is calculated using the truncated mean method. The cosine similarity between the gradient vector of each client and the global reference direction vector is calculated. All clients with a cosine similarity greater than zero are retained to form the initial client set. For the clients not retained, a multiplicative exponential multipath penalty decay based on reputation score is performed. The multiplicative exponential multipath punitive decay based on reputation score is as follows: calculate the product of the client's reputation score and the decay factor, which is the client's decayed reputation score; if the decayed reputation score is lower than the minimum reputation threshold, the client will be restricted from participating in the aggregation process in subsequent global iteration rounds. Semantic filtering stage: Load the local models of each client in the initial screening client set and run them on the unlabeled public dataset to obtain the predicted probability distribution output by each local model; for any client in the initial screening client set, calculate the average JS divergence score between that client and other clients in the initial screening client set based on the predicted probability distribution output by each local model, forming a group score set; calculate the absolute median difference of the group score set and perform normal standardization correction to obtain the standardized absolute median difference; calculate the difference between the average JS divergence score and the median of the group score set, and retain clients whose difference is within a preset multiple of the standardized absolute median difference interval to form the filtered client set; Adaptive multidimensional weighted aggregation is performed on each client in the filtered client set, and Nesterov acceleration gradient descent with embedded momentum is used to generate a new round of global model update parameters, which are then uploaded to the InterPlanetary File System (IPFS).

9. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 8, characterized in that, The process of adaptively weighting and aggregating each client in the selected client set, and generating new global model update parameters using Nesterov accelerated gradient descent with embedded momentum, includes: According to the client Local data scale Calculate the local data size The proportion of all clients in the total data size of the filtered client set is used as the data size weight. ; For the client The average JS divergence score is subjected to exponential decay mapping and normalization to obtain the semantic similarity penalty weight. ; Weighting based on data size Weights with semantic similarity penalty After weighted summation, multiply by the client's... The current reputation score is used to obtain the adaptive aggregation weight. ; Based on adaptive aggregation weights The gradient vectors of each client in the filtered client set are weighted and aggregated to obtain the aggregated gradient. ; Based on aggregation gradient Update the momentum term in the current global iteration round. ; the initial momentum term Let it be the zero vector. Then, the momentum term updated in the first round of global iteration is represented as: Starting from the second round of global iteration, updates will be performed according to the following recursive formula: ； in, Indicates the momentum coefficient; Indicates the first The momentum term updated in each round of global iteration; Indicates the aggregate learning rate; Global model parameters for the current global iteration round The momentum term updated in the previous global iteration round and the momentum term updated in the global iteration rounds We perform a weighted summation to obtain the new round of global model update parameters.

10. The heterogeneous federated learning method based on blockchain and knowledge distillation according to claim 9, characterized in that, The verifiable random function based on stake selects a consensus committee from the consensus nodes of the blockchain network. The consensus committee verifies and signs candidate blocks. Upon successful verification, the content addressing hash (CID) of the new global model is written into the decentralized ledger, including: For any edge node in the blockchain network The edge nodes are consensus nodes in the blockchain network; Using the block header hash of the latest block in the decentralized ledger as the random number seed, and utilizing edge nodes... VRF private key Cryptographic calculations are performed on the random number seed of the latest block to generate a... Unique random value uniformly distributed within the interval and the corresponding verifiable credentials; If a unique random value If the inequality based on the lottery decision is satisfied, then the edge node... Candidates selected for the current global iteration round: Edge nodes selected for the candidate committee Verifiable credentials are broadcast to the blockchain network, and then distributed across the blockchain network, excluding edge nodes. The remaining edge nodes, besides those, utilize the edge nodes public key In a zero-trust environment, a legitimacy verification is performed to confirm that the node's selection result is authentic and valid. The consensus committee is composed of all the edge nodes that have passed the verification. Each edge node in the consensus committee independently verifies the candidate block. After the verification is passed, each committee member digitally signs the candidate block. Once a candidate block has collected valid digital signatures from all edge nodes in the consensus committee within the specified timeframe, the candidate block will be recorded in the decentralized ledger.

Images