A method for sharing flight operation data based on federated knowledge distillation

By introducing blockchain technology and master node clustering into federated knowledge distillation, the problems of single point of failure and poisoning attacks on servers are solved, and efficient and secure sharing of flight operation data and model training are achieved.

CN117312442BActive Publication Date: 2026-11-17CIVIL AVIATION UNIV OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311158371.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-08
Publication Date
2026-11-17
Estimated Expiration
2043-09-08

AI Technical Summary

Technical Problem

Existing federated knowledge distillation algorithms are vulnerable to single points of failure on servers and poisoning attacks, resulting in low data security and privacy protection performance.

Method used

The system uses blockchain technology to build servers and aviation equipment as clients. Flight operation data is shared through a federated knowledge distillation method, including uploading public data to the blockchain by the clients, training and testing neural network models, using master nodes to cluster and aggregate soft tag sets, removing abnormal soft tags, and performing multiple rounds of iterative knowledge distillation.

Benefits of technology

It improves data security and privacy protection in data sharing, avoids single points of failure and poisoning attacks on servers, and enhances model training efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117312442B_ABST
    Figure CN117312442B_ABST
Patent Text Reader

Abstract

This invention discloses a flight operation data sharing method based on federated knowledge distillation. At least some clients first upload public data to the blockchain. Then, each client obtains the public data, performs pre-training, and tests the public data, uploading the test results as soft tag sets to the blockchain. Next, a master node is selected from among the clients. The master node clusters the soft tag sets uploaded by each client, removes abnormal soft tag sets, and aggregates the remaining soft tag sets to obtain an aggregated soft tag set. Each client obtains the public data and aggregated soft tag set from the blockchain to perform knowledge distillation. Finally, business operations are executed through various neural network models trained over multiple rounds. This invention uses generated data as public data, improving data security for data sharing. By filtering soft tags through clustering and eliminating abnormal soft tags, it avoids single points of failure and poisoning attacks on servers, improving data security and privacy protection performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method for sharing flight operation data based on federated knowledge distillation. Background Technology

[0002] Generally, the interconnection of data in the Aviation Internet of Things (AIoT) can create significant value for all data-sharing participants. With the continuous expansion of AIoT and smart civil aviation, the importance of data security and privacy protection is also constantly increasing. Currently, aircraft, radar, satellites, and ground monitoring equipment generate massive amounts of data daily. This data has immense value in AIoT, and utilizing it can improve the efficiency of civil aviation production and operation units such as airlines, air traffic control, and airports. Taking airlines as an example, AIoT technology can help airlines improve aircraft flight management and engineering management, enhance passenger baggage handling experience, promote air cargo informatization, and improve operational management levels. However, traditional centralized data storage and mining models are insufficient to meet the growing needs for data security and privacy protection.

[0003] Federated machine learning (Federated Learning) is a distributed machine learning technique proposed by McMahan in 2017. Also known as federated learning, it enables data value extraction while protecting data privacy. Its main idea is that the model is visible but the data is not; the server aggregates the model parameters trained by the clients to obtain a better machine learning model. Federated learning technology has been widely used in privacy protection and data sharing in fields such as healthcare, finance, and connected vehicles. However, in federated learning, each client uses the same model. Using the same model structure between clients with significantly different computing and communication capabilities leads to extremely high computation and communication latency. Furthermore, the transmission of model parameters or gradients between clients and the server in federated learning algorithms poses data security risks such as model inversion attacks and gradient backpropagation.

[0004] Federated knowledge distillation is a novel federated learning paradigm. Federated knowledge distillation clients learn from each other's predicted soft labels on a public dataset through model learning, allowing clients to use different model structures. Furthermore, because federated knowledge distillation transmits predicted soft labels from public data rather than model parameters, it effectively prevents attacks such as model inversion and gradient backpropagation. In addition, federated knowledge distillation offers higher communication efficiency compared to traditional federated learning. The theory and applications of federated knowledge distillation have been extensively studied. Taya et al. proposed a decentralized consensus-based multi-hop federated knowledge distillation algorithm (CMFD) and analyzed its convergence in function space. Kumar et al. proposed a federated knowledge distillation algorithm in a MediSecFed adversary environment and analyzed its performance in improving communication efficiency, model heterogeneity, and attack resistance. Ni et al. proposed a Fed2Codl bidirectional federated knowledge distillation algorithm, where the client and server update their respective model parameters through a knowledge distillation loss function, and conducted convergence analysis under the assumption of convex functions. Wang et al. proposed a heterogeneous federated distillation algorithm that dynamically adjusts the parameter T of the soft label cross-entropy and uses a "brainstorming" mechanism for bidirectional federated knowledge distillation. Gong et al. proposed a privacy-preserving integrated attention federated knowledge distillation algorithm and performed a theoretical analysis of the performance boundary (convergence) of the global model. Liu et al. proposed a federated knowledge distillation algorithm that transmits both model updates and predicted soft labels to improve algorithm accuracy and performed convergence analysis. Wu et al. proposed a federated knowledge distillation algorithm in which the client contains teacher and student models, and uses the SVD algorithm to decompose the parameters of the student model uploaded by the client to reduce communication volume.

[0005] However, the aforementioned data sharing methods based on federated knowledge distillation algorithms are all vulnerable to single points of failure on servers and poisoning attacks, resulting in low data security and privacy protection performance. Summary of the Invention

[0006] Therefore, it is necessary to provide a flight operation data sharing method based on federated knowledge distillation to address the aforementioned technical issues.

[0007] The following technical solution is adopted in this specification:

[0008] This specification provides a method for sharing flight operation data based on federated knowledge distillation, including:

[0009] Servers and / or aviation equipment that are added to the pre-built blockchain will act as clients, and at least some clients will upload public data to the blockchain; wherein, the public data includes at least one of the client's flight operation data and generated data obtained by the client based on real flight operation data through a generative model, and the public data includes flight operation data records and real labels;

[0010] Each client obtains public data from the blockchain, trains its neural network model based on the data records and real labels of the public data, tests the public data using the trained neural network model, and uploads the test results as a soft label set to the blockchain.

[0011] Based on the amount of public data uploaded by each client, a master node is selected from each client. The soft tag sets uploaded by each client are clustered by the master node, and soft tag sets that are different from the soft tag sets of the master node are removed. The remaining soft tag sets are then aggregated in each public data dimension to obtain an aggregated soft tag set, which is then uploaded to the blockchain.

[0012] Each client obtains public data and aggregated soft tag sets from the blockchain, trains its neural network model based on the data records of the public data and the corresponding soft tags, and trains its neural network model based on local flight operation data to complete a single round of federated knowledge distillation.

[0013] The neural network models of each client are trained through multiple rounds of iterative federated knowledge distillation, thereby enabling the sharing of flight operation data based on federated knowledge distillation.

[0014] Optionally, the client generates data based on real flight operation data according to the following objective function:

[0015] min G max D V(D,G)=E x~pdata(x) [logD(x|y)]-+E z~pdata(z) [log(1-G(z|y))]

[0016] x gen =G(w,y)

[0017] Where G is the generator, D is the discriminator, x is the actual flight operation data, y is the label of the generated data, z is the generated data, w is the generator parameter, and E is the generator parameter. x~pdata(x) [] represents the discriminant loss function, x gen To generate data.

[0018] Optionally, each client trains its neural network model based on public data records and real labels according to the following formula:

[0019]

[0020] Among them, L CE Let η be the cross-entropy loss function, η be the learning rate, and D be the cross-entropy loss function pubFor public datasets of federated knowledge distillation, w i For local model parameters on the client, ← indicates update.

[0021] Alternatively, the trained neural network model can be tested on public data according to the following formula:

[0022]

[0023] in, Let F() be the set of soft tags uploaded by client i, and let F() be the neural network model of the client.

[0024] Optionally, the master node can cluster the soft tag sets uploaded by each client according to the following formula:

[0025]

[0026] Among them, u k is the cluster center, and m is the cluster category.

[0027] Optionally, the remaining soft tag set can be aggregated for each common data dimension according to the following formula to obtain the aggregated soft tag set:

[0028]

[0029] in, K represents the aggregated soft tag set, and K is the number of remaining soft tag sets participating in the aggregation.

[0030] Optionally, the neural network model for each client is trained based on the data records of the public data and the corresponding soft labels according to the following formula:

[0031]

[0032] Where τ is the learning rate, L KL Let KL divergence loss function be used.

[0033] Optionally, the neural network model for each client can be trained based on the following formula using local flight operation data:

[0034]

[0035] Among them, D i This is a local dataset for the client.

[0036] Optionally, a master node can be selected from each client based on the amount of public data uploaded by each client, specifically including:

[0037] The overall score of the server nodes contained in the blockchain is initialized using the following formula:

[0038]

[0039] Among them, S initial This is the initial comprehensive score for the consensus nodes. The number of public data uploaded by the client, where α is the public data scoring weight;

[0040] Based on the comprehensive scores of each client, each server node is divided into a consensus node set and a candidate node set;

[0041] The master node is selected from the consensus node set using a roulette wheel algorithm.

[0042] Optionally, at least some clients upload public data to the blockchain and simultaneously reach consensus on the uploaded public data within the blockchain; the master node uploads the aggregated soft tag set to the blockchain and simultaneously reaches consensus on the aggregated soft tag set within the blockchain. During the consensus process, the master node reduces the comprehensive score of server nodes corresponding to soft tag sets that are different from the master node's soft tag set during the clustering process, and increases the comprehensive score of server nodes corresponding to soft tag sets that are of the same type as the master node's soft tag set.

[0043] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for sharing flight operation data based on federated knowledge distillation.

[0044] This specification provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method for sharing flight operation data based on federated knowledge distillation.

[0045] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0046] At least some clients first upload public data to the blockchain. This public data includes at least one of the client's flight operation data and generated data obtained by the client based on real flight operation data through a generative model. Then, each client retrieves the public data from the blockchain for pre-training and tests the public data, uploading the test results as soft-label sets to the blockchain. Next, a master node is selected from among the clients. The master node clusters the soft-label sets uploaded by each client, removing soft-label sets that are not in the same category as the master node's set. The remaining soft-label sets are then aggregated along each public data dimension to obtain an aggregated soft-label set, which is uploaded to the blockchain. Each client retrieves the public data and aggregated soft-label sets from the blockchain to perform knowledge distillation. Finally, through multiple rounds of iterative federated knowledge distillation, the neural network models of each client are trained, and the trained neural network models are used to execute business operations.

[0047] This invention treats the generated data as public data, improving the data security of data sharing. By using clustering to filter soft tags, abnormal soft tags are eliminated, thereby avoiding single points of failure and poisoning attacks on the server and improving data security and privacy protection performance. Attached Figure Description

[0048] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0049] Figure 1 This document provides a schematic diagram of a practical Byzantine fault-tolerant algorithm process.

[0050] Figure 2 This specification provides a flowchart illustrating a flight operation data sharing method based on federated knowledge distillation.

[0051] Figure 3 A schematic diagram of a blockchain-based federated knowledge distillation system provided for this specification;

[0052] Figure 4 This is a schematic diagram of a block structure provided in this specification;

[0053] Figure 5 This document provides a schematic diagram of a consensus process.

[0054] Figure 6 A schematic diagram illustrating a flight operation data sharing method based on federated knowledge distillation, provided for this specification;

[0055] Figure 7a This document provides a schematic diagram illustrating the test results of client data being independently and identically distributed on the FEMNIST dataset under different proportions of malicious clients.

[0056] Figure 7b This document provides a schematic diagram illustrating the test results of client data being non-independently and identically distributed on the FEMNIST dataset under different proportions of malicious clients.

[0057] Figure 7c This document provides a schematic diagram illustrating the test results of client data being independently and identically distributed on the CIFAR100 dataset under different proportions of malicious clients.

[0058] Figure 7d This diagram illustrates the test results of client data being non-independently and identically distributed on the CIFAR100 dataset under different proportions of malicious clients, as provided in this specification.

[0059] Figure 8a This document provides a comparative diagram showing the test results of two types of client data independently and identically distributed on the FEMNIST dataset and the test results of training and testing on a local dataset under different proportions of malicious clients.

[0060] Figure 8b This diagram illustrates a comparison of test results on the FEMNIST dataset and the local dataset under different proportions of malicious clients, where the client data is non-independent and identically distributed, with test results from separate training and testing on the local dataset.

[0061] Figure 8c This document provides a comparative diagram showing the test results of two types of client data independently and identically distributed on the CIFAR100 dataset and the test results of training on a local dataset alone, under different proportions of malicious clients.

[0062] Figure 8d This diagram illustrates a comparison between test results of different client data with different malicious client ratios on the CIFAR100 dataset and test results of training on a local dataset alone, where the data are not independently identically distributed under different malicious client ratios.

[0063] Figure 9 This document provides a schematic diagram illustrating consensus latency and communication overhead.

[0064] Figure 10 This specification provides a schematic diagram of a computer device for implementing a flight operation data sharing method based on federated knowledge distillation. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0066] Currently, data sharing methods based on federated knowledge distillation algorithms all rely on a central service. Blockchain, proposed by Nakamoto in 2008, is a distributed ledger technology. Blockchain possesses characteristics such as decentralization, immutability, traceability, and distribution, and has been widely applied in record keeping and data sharing in logistics supply chains, power systems, and government departments. Blockchain can effectively solve problems such as single points of failure and record keeping in federated learning, ensuring data security. Decentralized, traceable, and highly secure federated learning data sharing methods based on blockchain have also become a research hotspot in recent years. Li et al. proposed a decentralized federated learning algorithm based on committee consensus and verified client node updates, improving the model's anti-attack performance. Liu et al. proposed a decentralized federated learning algorithm based on 5G communication networks, writing federated learning into blockchain smart contracts to aggregate model parameters while ensuring data security. Fang Chen et al. proposed a decentralized federated learning algorithm based on the DPoS consensus algorithm and introduced adaptive differential privacy technology, improving privacy protection performance while enhancing data security. Qi et al. proposed a blockchain-based decentralized federated learning algorithm, which verifies client model updates, allowing only verified nodes to participate in aggregation, thus improving model training accuracy. Qin et al. proposed a reputation-based decentralized federated learning algorithm, storing client node performance on the blockchain and selecting nodes to participate in updates based on client reputation values. Song Yubo et al. proposed an APBFT consensus algorithm based on client model training performance, achieving decentralized federated learning and improving the attack resistance and training accuracy of the federated learning model. However, these blockchain-based federated learning algorithms all rely on the same client model, making them unsuitable for heterogeneous model patterns, and the consensus efficiency, data security, and privacy protection performance of these blockchain methods still need improvement.

[0067] The core idea of ​​federated learning is that the client trains the model locally, uploading only parameters to the server and not the data itself, thus achieving data isolation and protecting user privacy. The server aggregates the client parameters and then calculates the optimal global model parameters. Its objective function is as follows:

[0068]

[0069] Where, p i L represents the weight of client i. i The loss function representing client i, D i represents the local sample of client i, and N is the number of clients participating in the training.

[0070] Knowledge distillation was proposed by Hinton in 2015. Its purpose is to transfer knowledge learned from one or more complex models (as the teacher model) to another lightweight model (as the student model), while keeping the model as lightweight as possible without sacrificing performance. In knowledge distillation, the teacher model is trained on all the data, while the student model is trained on a small subset of the data. A well-trained student model learns not only the labels of the data (called hard labels) but also the predicted distribution of this subset of data output by the teacher model (called soft labels). This model is generated using the following formula:

[0071]

[0072] Where M is the number of label categories, z j The value corresponding to the j-th category in the output of the model's last layer is represented by T, which is called temperature. This smooths the probability distribution of the output. The presence of soft labels significantly improves the accuracy of student model training, meaning that soft labels convey the knowledge learned during the training of the teacher model.

[0073] Blockchain, as a decentralized and tamper-proof distributed ledger, can securely and verifiably construct a ledger in a trustless environment. In a blockchain, all participating nodes can verify and forward transactions, and a consensus algorithm maintains a consistent ledger across the entire network. Each block in the ledger records a series of transactions and the hash value of the previous block, thus connecting the current block to the previous one. Based on the degree of decentralization, blockchains are classified into public blockchains, private blockchains, and consortium blockchains.

[0074] Consensus algorithms are the core of blockchain technology. Practical Byzantine Fault Tolerance (PBFT), as a state machine replication algorithm, is a highly efficient blockchain consensus algorithm. Figure 1 As shown, under normal circumstances, the master node sorts all requests from clients, and then all correct nodes reach consensus by running a three-phase protocol, including pre-preparation, preparation, and confirmation. The master node may act maliciously or time out due to delays, triggering a view rotation mechanism to replace it with a new master node. The PBFT algorithm has 1 / 3 of the Byzantine fault tolerance. In the pre-preparation phase, the number of communications in the consensus network is (N-1). In the preparation phase, the number of communications in the consensus network is (N-1)·(N-1). In the commit phase, the number of communications in the consensus network is N·(N-1). Therefore, the traditional PBFT consensus algorithm has 2N·(N-1) communications in one consensus process. Figure 1 As shown.

[0075] Figure 1This document provides a schematic diagram of a practical Byzantine fault-tolerant algorithm. Figure 1 In this diagram, C represents a client node, and 0 to 3 nodes represent consensus nodes. 0 is the master node, 1 and 2 are replica nodes, and 3 is a faulty node. The master node is calculated using the following formula:

[0076] P = v mod |R|

[0077] Where P is the view number, v is the replica number, and |R| is the number of replica node sets.

[0078] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0079] Figure 2 This is a flowchart illustrating a flight operation data sharing method based on federated knowledge distillation, as described in this specification. The method includes the following steps:

[0080] S101: Servers and / or aviation equipment that are added to the pre-built blockchain will act as clients, with at least some clients uploading public data to the blockchain.

[0081] In typical aviation IoT applications, flight operation data can be shared between servers of relevant service providers and aviation IoT devices, thereby increasing the available data for each party's neural network models. This invention proposes a blockchain-based Federated Knowledge Distillation algorithm (BFKD) to achieve flight operation data sharing based on federated knowledge distillation.

[0082] For example, suppose a BFKD system consists of M servers and a set of K aviation IoT devices (M+K=N), such as Figure 3 As shown. Figure 3 This document provides a schematic diagram of a blockchain-based federated knowledge distillation system. Figure 3 As can be seen, multiple aviation IoT servers and devices have joined a pre-built blockchain. Therefore, each server and aviation IoT device can be considered a client in the federated knowledge distillation process, each possessing its own local dataset D. i ,i=1,2,…,N.

[0083] For each client in the blockchain, at least some clients can upload public data to the blockchain. This public data includes at least one of the client's flight operation data and generated data obtained by the client through a generative model based on real flight operation data. The public data contains flight operation data records and real-world tags.

[0084] The flight operation data of the client is the client's own local data. This local data includes both sample data used to train the model and annotations of each sample data, i.e., real labels.

[0085] To protect the data privacy of each client, in one or more embodiments of this specification, each client can upload generated data obtained from local data through a generative model. Here, a Conditional Generative Adversarial Network (CGAN) is used to obtain labeled generated data, and CGAN can integrate the label information y into the generator G and the discriminator D.

[0086] Specifically, in one or more embodiments of this specification, the client can generate data based on real flight operation data through the following objective function:

[0087] min G max D V(D,G)=E x~pdata(x) [logD(x|y)]-+E z~pdata(z) [log(1-G(z|y))]

[0088] x gen =G(w,y)

[0089] Where G is the generator and D is the discriminator.

[0090] The server mentioned in this manual can be a server set up on a business platform, or a device such as a desktop computer or laptop computer that can execute the solution in this manual.

[0091] S102: Each client obtains public data from the blockchain, trains its neural network model based on the data records and real labels of the public data, tests the public data using the trained neural network model, and uploads the test results as a soft label set to the blockchain.

[0092] After at least some of the clients upload public data to the blockchain, each client can obtain public data from the blockchain and use data sharing to obtain more data to train its local model.

[0093] The local model mentioned here refers to the neural network model used by the client to execute business operations locally. The specific type of neural network model can be determined as needed, and this specification does not impose any restrictions on it. Furthermore, different clients can hold different neural network models. The specific business operations performed by the model can also be determined as needed, and this specification does not impose any restrictions on it. For example, it could be a neural network model for flight delay prediction, or a neural network model for flight user classification, etc.

[0094] Of course, different clients can perform the same business through different neural network models. For example, they can use different neural network models to predict flight delays based on flight operation data.

[0095] At least some clients upload public data to the blockchain while simultaneously reaching a consensus on the uploaded public data within the blockchain. After consensus confirmation, public data can be accumulated based on the amount of public data. Once a batch of data is obtained, in one or more embodiments of this specification, each client can retrieve the public data from the blockchain and pre-train its neural network model based on the data records and real labels of the public data according to the following formula:

[0096]

[0097] Among them, L CE Let η be the cross-entropy loss function, η be the learning rate, and D be the cross-entropy loss function pub For public datasets of federated knowledge distillation, w i For local model parameters on the client, ← indicates update.

[0098] After pre-training is completed, in one or more embodiments of this specification, each client can use its local model to test the public data based on the following formula:

[0099]

[0100] in, Let F() be the set of soft tags uploaded by client i, and let F() be the neural network model of the client.

[0101] Each client can upload its test results as a soft tag set to the blockchain. Continuing with the flight delay prediction example, assuming that for each data record in the public data set, the client's neural network model can obtain a 1×2 vector containing the probability of flight delay and the probability of no delay for that data record, this prediction vector is the test result mentioned above, i.e., the soft tag. The combination of soft tags corresponding to each data record constitutes the soft tag set. This soft tag set reflects the knowledge possessed by the corresponding client's neural network model. After uploading and processing each soft tag set to the blockchain, each client can perform knowledge distillation through the soft tag set, thereby further improving model training efficiency.

[0102] S103: Select a master node from each client based on the amount of public data uploaded by each client. Cluster the soft tag sets uploaded by each client through the master node, remove soft tag sets that are different from the soft tag sets of the master node, and aggregate the remaining soft tag sets in each public data dimension to obtain an aggregated soft tag set. Upload the aggregated soft tag set to the blockchain.

[0103] When a client uploads a set of soft tags to the blockchain, the blockchain can confirm and package the data for on-chain recording through a consensus mechanism. Therefore, it is necessary to first select a master node from among the clients based on the amount of public data uploaded by each client. For example, the probability of each client being selected as a master node can be set to a positive correlation with the amount of public data uploaded by each client. There are already relatively mature technologies for selecting master nodes, and this specification does not restrict the specific method used.

[0104] Of course, in one or more embodiments of this specification, for each server node in the blockchain, a master node can be selected based on the amount of public data uploaded by each server node, thereby incentivizing each server node to upload public data. In one or more embodiments of this specification, the comprehensive score of the server nodes included in the blockchain can be initialized using the following formula:

[0105]

[0106] Among them, S initial This is the initial comprehensive score for the consensus nodes. α represents the amount of public data uploaded by the client, and α represents the public data scoring weight.

[0107] Then, based on the comprehensive scores of each client, each server node can be divided into a consensus node set and a candidate node set, and a master node can be selected from the consensus node set using a roulette wheel algorithm. As can be seen from the above, a corresponding public data score is assigned to each client based on the amount of public data uploaded, and clients with more public data scores have a higher probability of being selected as the master node.

[0108] After the master node is determined, in order to resist single points of failure and poisoning attacks, in one or more embodiments of this specification, the master node can cluster the soft tag sets uploaded by each client according to the following formula:

[0109]

[0110] Among them, u k is the cluster center, and m is the cluster category.

[0111] After clustering is completed, the master node can remove soft tag sets that are different from the master node's soft tag set. It can be understood that the soft tag set uploaded by the master node is assumed to be a normal soft tag set, while soft tag sets that are different from the master node's soft tag set are abnormal soft tag sets, which may have single points of failure or poisoning attacks. Therefore, soft tag sets of different types are removed to further ensure data security.

[0112] Subsequently, in one or more embodiments of this specification, the master node may aggregate the remaining soft tag set in each common data dimension according to the following formula to obtain an aggregated soft tag set:

[0113]

[0114] in, Let K be the number of remaining soft tag sets participating in the aggregation. As can be seen from the above formula, the aggregation here uses a summation and averaging method. Continuing with the aforementioned flight delay prediction example, assuming the public data includes 100 data records, each client's neural network model will obtain a soft tag set consisting of 100 1×2 prediction vectors. Here, during aggregation, the summation and averaging are performed along the dimension of each prediction vector.

[0115] S104: Each client obtains public data and aggregated soft tag sets from the blockchain, trains its neural network model based on the data records of the public data and the corresponding soft tags, and trains its neural network model based on local flight operation data, thus completing a single round of federated knowledge distillation.

[0116] S105: Train the neural network models of each client through multiple rounds of iterative federated knowledge distillation to complete the sharing of flight operation data based on federated knowledge distillation.

[0117] After filtering and aggregating the soft tag sets uploaded by each client as described above, each client can obtain public data and aggregated soft tag sets from the blockchain for knowledge distillation. That is, based on the data records of the public data and the corresponding soft tags, the neural network model of each client is trained according to the following formula:

[0118]

[0119] Where τ is the learning rate, L KL Let KL divergence loss function be used.

[0120] After knowledge distillation is completed, the client can further train the neural network model of each client based on the following formula using local flight operation data, thereby completing one round of federated knowledge distillation:

[0121]

[0122] Among them, D i This is a local dataset for the client.

[0123] Finally, through multiple rounds of iterative federated knowledge distillation, the neural network models of each client are trained using data sharing, thereby achieving the sharing of flight operation data based on federated knowledge distillation. Business operations can then be executed through the trained neural network models. Utilizing data sharing based on federated knowledge distillation can significantly improve the training efficiency of the models.

[0124] based on Figure 2 The flight operation data sharing method based on federated knowledge distillation, as shown, involves at least some clients uploading public data to the blockchain. This public data includes at least one of the clients' flight operation data and generated data obtained by the clients using generative models based on real flight operation data. Then, each client retrieves the public data from the blockchain for pre-training and tests the public data, uploading the test results as soft-label sets to the blockchain. Next, a master node is selected from among the clients. The master node clusters the soft-label sets uploaded by each client, removing soft-label sets that are not in the same category as the master node's set. The remaining soft-label sets are then aggregated along each public data dimension to obtain an aggregated soft-label set, which is uploaded to the blockchain. Each client retrieves the public data and aggregated soft-label sets from the blockchain to perform knowledge distillation. Finally, through multiple rounds of iterative federated knowledge distillation, the neural network models of each client are trained, and the trained neural network models are used to execute business operations.

[0125] This invention treats the generated data as public data, improving the data security of data sharing. By using clustering to filter soft tags, abnormal soft tags are eliminated, thereby avoiding single points of failure and poisoning attacks on the server and improving data security and privacy protection performance.

[0126] When applying the flight operation data sharing method based on federated knowledge distillation provided in this specification, it is not necessary to... Figure 2 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this manual does not impose any restrictions on it.

[0127] Furthermore, in one or more embodiments of this specification, traditional federated knowledge distillation algorithms lack a data storage mechanism, and centralized data storage methods are susceptible to single points of failure. Blockchain possesses characteristics such as decentralization, traceability, and immutability. Therefore, this invention utilizes blockchain for data storage of federated knowledge distillation and related information. Due to varying storage capabilities of distributed devices and the slow consensus speed of blockchain when dealing with large amounts of data, this specification employs both on-chain and off-chain data storage methods. The client soft tag set is stored off-chain on a cloud server or IPFS network, while the soft tag hash value is stored in the Merkle tree of the block body, such as... Figure 4 As shown. Figure 4 This is a schematic diagram of a block structure in this specification.

[0128] Furthermore, in one or more embodiments of this specification, at least some clients upload public data to the blockchain while simultaneously reaching a consensus on the uploaded public data within the blockchain. After consensus confirmation, the blockchain adds the new block content to the chain. Additionally, the master node uploads the aggregated soft tag set to the blockchain while simultaneously reaching a consensus on the aggregated soft tag set within the blockchain. During the consensus process, the master node reduces the overall score of server nodes corresponding to soft tag sets that are different from the master node's soft tag set during clustering, and increases the overall score of server nodes corresponding to soft tag sets that are of the same type as the master node's soft tag set. Of course, to prevent the master node from having excessively high scores, which would lead to an excessively high probability of it being continuously elected as the master node and thus weakening the degree of consensus decentralization, the master node's score may not be updated.

[0129] In one or more embodiments of this specification, the client may employ the FSGPBFT (Federated Node Behavior-Based Group Consensus) algorithm proposed in this specification during consensus processing. Specifically, the consensus process for reaching consensus on aggregated soft tag sets is as follows: Figure 5 As shown. Figure 5 This is a schematic diagram of a consensus process provided in this specification. Figure 5 As can be seen, during the consensus process, the master node initiates consensus on the current round of messages and broadcasts the new block using pre-prepared messages, i.e. <PRE-PREPARE,H B ,D(M,sk p )>. Among them, H B It is the block height, D(M,sk) p The block is defined by the master node's private key sk. p The consensus node set receives the new block and verifies the digital signatures of the transactions and the new block to confirm the validity of the new block's digital signature. Additionally, it recalculates the soft-label filtering and aggregation results to confirm the master node's calculations are correct. After verification, each consensus node stores the new block locally.

[0130] After pre-processing message verification, the consensus node will prepare the message. <PREPARE,H B ,D(M,sk m The block is sent to other consensus nodes. The master node and all consensus nodes receive the preparation message and confirm that more than 2 / 3 of the consensus nodes have verified and agreed to the new block.

[0131] After reaching an agreement on the new block, the consensus node broadcasts a response message to the master node. <RESPONSE,H B ,D(M,sk m If the master node receives confirmation messages from more than 2 / 3 of the consensus nodes, then the consensus is confirmed.

[0132] Once consensus is reached, the new block takes effect, and the master node sends a response message to all nodes. <REPLY,H B ,D(M,sk p This indicates that a new block has been stored in the blockchain, preparing it for the next round of consensus processing.

[0133] In decentralized blockchain systems, when each edge server is unaware of malicious intent, the steps of preparation, commit, and response are necessary to reach consensus. The consensus protocol ensures the correctness of the global model aggregation through authentication and verification of computation results on the edge servers.

[0134] In addition, this specification also provides a schematic diagram of a flight operation data sharing method based on federated knowledge distillation. For example... Figure 6 As shown, after the aviation IoT client completes local model training and soft tag prediction, it uploads the soft tag prediction results to the master node. The master node uses a smart contract to filter and aggregate soft tags. Then, the consensus master node broadcasts the locally acquired and processed information to the consensus node set for confirmation. The information confirmed by the consensus nodes is stored in the blockchain. Finally, each client can download public data and soft tags from the blockchain to perform knowledge distillation, leveraging data sharing to improve the efficiency and accuracy of model training.

[0135] Specifically, each client can train a local conditional generative adversarial network (GAN) model using local data, generate public data using the generator model, and upload the public data to the blockchain. The blockchain can then reach a consensus and confirm the received public data. Afterward, each client can retrieve the public data from the blockchain, perform pre-training and testing to obtain a soft tag set. Subsequently, a master node can be selected, and the master node will cluster the soft tag sets to remove abnormal soft tag sets and aggregate normal soft tag sets. The aggregated soft tag set is then uploaded to the blockchain and consensus is initiated. Clients can download the public data and the aggregated soft tag set to perform knowledge distillation, thereby achieving flight operation data sharing based on federated knowledge distillation.

[0136] In addition, this specification provides a performance analysis of the flight operation data sharing method based on federated knowledge distillation proposed in this specification, including complexity analysis and convergence analysis.

[0137] The complexity analysis includes federated knowledge distillation communication overhead analysis and consensus communication overhead analysis. Regarding the federated knowledge distillation communication overhead analysis, this invention analyzes the complexity of the BFKD method from the perspective of computational and communication costs. The number of communication rounds is represented by R, and the average data size per client is represented by D. Therefore, the model is directly learned in a federated manner (parameter set denoted as θ). i The computational cost is O(RD|θ).i The communication overhead is O(R|θ). i |). In BFKD, the client computation overhead is the same as in the FedAvg algorithm, and the communication overhead is As the number of model parameters increases daily... Therefore, BFKD is much more efficient in communication than the standard FedAvg method, without introducing too much computational cost.

[0138] For consensus communication overhead analysis, by Figure 5 As can be seen, the specific computation process of the FSGPBFT consensus algorithm has three stages, where σN is the number of consensus nodes, σ∈(0,1). In the pre-preparation stage, the master node broadcasts a preparation message to all consensus nodes. The number of communications in the consensus network during this stage is (σN-1). In the preparation stage, when the pre-preparation message sent by the master node is verified, the replica nodes send pre-preparation messages to other consensus nodes. The number of communications in the consensus network during this stage is (σN-1)·(σN–1). In the response stage, the master node receives and verifies the verification message from the consensus nodes. The number of communications in the consensus network during this stage is (σN-1). Therefore, the improved FSGPBFT consensus algorithm completes the consensus process with (σN-1)·(σN+1) communications. The FSGPBFT consensus algorithm can adjust the scale of consensus nodes as needed, and its communication overhead is much lower than that of the traditional PBFT consensus algorithm.

[0139] For convergence analysis, the distributed optimization model of the BFKD algorithm is defined as follows:

[0140]

[0141] Where N is the total number of all AIoT devices, L CE and L KL This represents the cross-entropy loss function and the Kullback-Leibler divergence loss function w. i These are the local model parameters for the client.

[0142] For loss function L CE and L KL Make the following five assumptions:

[0143] Assumption 1: L CE and L KL Both are L-smooth functions. For all v and w, we have

[0144]

[0145]

[0146] Assumption 2: L CE and LKL Both are μ-strongly convex functions. For all v and w, we have

[0147]

[0148]

[0149] Assumption 3: ε i and δ i All data are randomly and uniformly sampled from the local data of the i-th device. The variance of the stochastic gradient in each device is bounded: when the local devices are i = 1, ..., N, we have...

[0150]

[0151]

[0152] Assumption 4: The square norm of the expectation of the stochastic gradient is uniformly bounded, and has the following condition for i = 1, ..., N and t = 1, ..., T-1:

[0153]

[0154]

[0155] The update of the proposed BFKD model can be expressed by the following formula:

[0156]

[0157] definition

[0158]

[0159]

[0160] Therefore, for and have

[0161]

[0162] In the above equation, since the expectation of the second term in the expansion is 0, the subsequent convergence analysis mainly focuses on the first term A1 and the third term A3. First, the constraint on the boundary A1 can also be expanded as follows:

[0163]

[0164] Since the uncertainty of the A1 boundary is independent of the first term, we will now analyze the boundaries of the second term B1 and the third term B2 in the above equation. At this point, the boundary constraints of B1 are as follows:

[0165]

[0166] in

[0167]

[0168] Using the Cauchy-Schwarz inequality and the arithmetic and geometric mean (AM-GM) inequality constraints, we can obtain:

[0169]

[0170] Therefore, from the above three equations, B1 can be expressed as

[0171]

[0172] Assumption 1 states that L CE and L KL Since they are all L-smooth functions, the corresponding local device gradient can be expressed as a constraint of the following form:

[0173]

[0174]

[0175] Therefore, based on the convexity of |||| and the above equation, B2 can be constrained as follows:

[0176]

[0177] Therefore, by combining the calculations of the above formula, A1 can be expressed as

[0178]

[0179] As shown in the above equation, A1 requires constraints on the boundary C of the last two terms. Definition

[0180]

[0181]

[0182] C can be divided into the following 3 items:

[0183]

[0184] In the above formula, for convenience, the first two terms in C are abbreviated as D. In the boundary constraint D, we have:

[0185]

[0186] The first inequality in the above formula comes from L.CE The convexity of L is determined by the second inequality derived from the AM-GM inequality, and the third inequality derived from the constraints corresponding to the local device gradients mentioned above. The same method is used to constrain L. KL The relevant items can be obtained as follows:

[0187]

[0188] At this point, combining the above formula, the boundary C can be simplified:

[0189]

[0190] In the last inequality, we can state the following three facts:

[0191] and

[0192]

[0193]

[0194] Reviewing the expression for A1 and substituting C into it, we get:

[0195]

[0196] Assuming 3, the stochastic gradient L in device i is... CE and L KL variance and This is the upper bound. Therefore, for boundary A3, we have:

[0197]

[0198] Since the BFKD algorithm requires communication once every T SGD steps, for any t≥0, there exists a t0≤t such that t-t0≤T-1 and This holds true for all k = 1, 2, ..., N. And for all t - t0 ≤ T - 1, ... Based on non-increasing η t We can obtain:

[0199]

[0200] Therefore, the above three formulas can be combined to obtain the following formula:

[0201]

[0202]

[0203] Set the decreasing step size of the BFKD algorithm to exist And when γ>0, it satisfies η t ≤2η t+T Therefore, proof is needed here. in Next, we will use inductive reasoning to analyze Δ. t Let's provide a concrete proof. First, when t=1, The equation holds. Assume the conclusion holds when t > 1, then when t = t + 1:

[0204]

[0205] Then, through the L-smoothness of Φ(), we can obtain:

[0206]

[0207] in and

[0208] In summary, for strongly convex smooth functions, the BFKD algorithm... The convergence rate converges to the global optimum. For the case involving only some devices, the convergence rate with only some devices is the same as the convergence rate with all devices.

[0209] This specification also provides a consensus security analysis, assuming... is n i The received message. Therefore... During the request phase, each client sends message m to the master node n0. i The master node packages and sorts these messages. During the pre-preparation phase, the master node n0 broadcasts a message to all replica nodes. If consensus node n i If it is normal, then at this time

[0210] otherwise in, It is node n i Regarding the message Verification. Given CN ≥ 3f + 1, assuming CN = 4 and f = 1, then we can obtain:

[0211] During the preparation phase, if, but otherwise in It is node n i Verification of the prepared messages. At this stage, any node participating in the consensus will also receive prepared messages from other nodes when it sends its own. Therefore, for node n... i The following relationship exists, where <prepared>n i This refers to the number of correctly prepared messages:

[0212] <prepare> n0= <prepare>(M,D)n1+ <prepare>(M,D)n2+ <prepare>(M,D)n3

[0213] <prepare> n1= <prepare>(M,D)n0+ <prepare>(M,D)n2+ <prepare>(M,D)n3

[0214] <prepare> n2= <prepare>(M,D)n0+ <prepare>(M,D)n1+ <prepare>(M,D)n3

[0215] <prepare> n3= <prepare>(M,D)n0+ <prepare>(M,D)n1+ <prepare>(M, D)n2

[0216] Assuming that more than 2 / 3 of the nodes are benevolent, the following equation always holds:

[0217]

[0218] Therefore, we can obtain the following result: <prepare>n i ≥2f(i=0,1,2,3)

[0219] In the response phase, based on the prepared <prepare>n i ≥2f, at least nodes n0, n1, and n2 can correctly process message M, therefore master node n0 will receive messages that satisfy the following formula.

[0220] <response>n i ≥2f(i=0,1,2,3)

[0221] As can be seen from the above, reaching a consensus is secure. Based on the above proof, it can be seen that FSGPBFT can guarantee the reliability of nodes, the correctness of the results in each round, and prevent tampering and forgery.

[0222] In addition, this specification provides an embodiment of the flight operation data sharing method based on federated knowledge distillation provided herein. This embodiment was tested on a server (desktop computer) equipped with an AMD EPYC 7402 CPU and two NVIDIA GeForce RTX 3080Ti super GPUs, each with 12GB of DRAM and 256GB of RAM. The experimental code for this embodiment was written in Python 3.9 and PyTorch 1.13.0. This embodiment tested the data mining performance of BFKD in two different environments and also tested its consensus performance.

[0223] In the first environment, the public data is generated from the MNIST dataset, while the private data is a subset of FEMNIST. This embodiment considers the iid case, where each private dataset is randomly sampled from FEMNIST, and the non-iid case, where each participant is given only one author's letter during training but is asked to classify all authors' letters at test time.

[0224] In the second environment, the public dataset is generated from the CIFAR10 dataset, while the private dataset is a subset of CIFAR100 with 100 subclasses belonging to 20 superclasses; for example, bears, leopards, lions, tigers, and wolves belong to the large carnivores category. In the iID case, each participant's task is to classify the test images into the correct subclass. The non-iID case is more challenging: during training, each participant has data from one subclass of each superclass; at test time, participants need to classify general test data into the correct superclass. For example, a participant who has only seen wolves during training is asked to correctly classify lions as large carnivores. Therefore, it must rely on information conveyed by other participants.

[0225] In each environment, 10 aviation IoT devices were designed with their own unique convolutional neural networks, which may vary in the number of kernels and layers, as detailed in Table 1. This embodiment uses the Adam optimizer with an initial learning rate of 0.001. In each round of collaborative training, a subset of size 8000 is randomly selected. As a foundation for communication, this embodiment considers soft-tag poisoning attacks in the experiment, whereby malicious clients upload soft-tag updates with Gaussian noise to hinder soft-tag aggregation, thereby affecting the convergence performance of each model.

[0226] Table 1. Information on Convolutional Neural Networks in Aviation IoT Devices

[0227]

[0228] To demonstrate the overall performance of BFKD, Tables 2-5 present the test results of the BFKD client model on the FMNIST and CIFAR100 datasets for both independent and non-independent identically distributed (IOD) local datasets. To illustrate BFKD's resistance to poisoning attacks, Tables 2-5 also show the test results for the Proportion of Malicious Participants (PoMP) with a malicious client ratio of 10%-30%. Furthermore, to more intuitively illustrate the overall performance of the BFKD algorithm, Figures 7a-7d Box plots of the overall client test results for the two datasets under different distributions are given.

[0229] Table 2. Test results of client data being independently and identically distributed on the FEMNIST dataset under different proportions of malicious clients.

[0230]

[0231] Table 3. Test results of client data non-independent and identically distributed on the FEMNIST dataset under different proportions of malicious clients.

[0232]

[0233] Table 4. Test results of client data being independently and identically distributed on the CIFAR100 dataset under different proportions of malicious clients.

[0234]

[0235] Table 5. Test results of client data non-independent and identically distributed on the CIFAR100 dataset under different proportions of malicious clients.

[0236]

[0237] Table 2 shows the test results of client data being independently and identically distributed on the FEMNIST dataset under different malicious client ratios in this specification; Table 3 shows the test results of client data being non-independent and identically distributed on the FEMNIST dataset under different malicious client ratios in this specification; Table 4 shows the test results of client data being independently and identically distributed on the CIFAR100 dataset under different malicious client ratios in this specification; and Table 5 shows the test results of client data being non-independent and identically distributed on the CIFAR100 dataset under different malicious client ratios in this specification. Figure 7a This is a schematic diagram illustrating the test results of client data being independently and identically distributed on the FEMNIST dataset under different malicious client ratios, corresponding to Table 2. Figure 7b This is a schematic diagram illustrating the test results of client data with non-independent and identical distribution under different malicious client ratios in the FEMNIST dataset, corresponding to Table 3. Figure 7c This is a diagram illustrating the test results of client data being independently and identically distributed on the CIFAR100 dataset under different malicious client ratios, corresponding to Table 4. Figure 7d This is a schematic diagram illustrating the test results of client data being non-independently and identically distributed on the CIFAR100 dataset under different malicious client ratios, corresponding to Table 5.

[0238] exist Figure 7a In Table 2-d, the rectangles represent the upper and lower quartiles of the ten client test results, and the short lines above and below the rectangles represent the maximum and minimum values ​​of the ten client test results. The solid lines within the rectangles represent the mean of the ten client test results. The dashed lines within the rectangles represent the median of the ten client test results. Dots represent outliers. (From Tables 2-5 and...) Figure 7a The results from ~d show that, overall, the test accuracy for each client decreases as the proportion of malicious clients increases. The average accuracy for the two datasets decreased by 0.0220, 0.1066, 0.0244, and 0.03282 for independent and non-independent identically distributed data, respectively. The soft labels used to identify malicious clients prevent other models from learning the knowledge gained from training on the local data. For independent and identically distributed data, reducing the amount of training data of the same type results in a smaller decrease in accuracy. For non-independent and identically distributed data, it means that knowledge of some unseen data types may not be obtained, thus leading to a larger decrease in accuracy. Taking the case without poisoning attacks as an example, the differences between the maximum and minimum values ​​of the ten models in the two datasets for the two distributions are 0.0761, 0.1802, 0.0727, and 0.1551, respectively. This difference is due to the heterogeneity of the models and the differences in local data. Overall, BFKD exhibits good resistance to poisoning attacks.

[0239] To further illustrate the knowledge transfer effect of the BFKD method, specifically its performance improvement for each client, two representative client models are selected. Figure 8a ~d presents a comparison of the best and worst performing client models in the no-malicious-client test with the results of training and testing on the local dataset alone.

[0240] Figure 8a This diagram illustrates a comparison between test results of two types of client data independently and identically distributed on the FEMNIST dataset and test results of training on a local dataset alone, under different proportions of malicious clients, as described in this specification. Figure 8b This diagram illustrates a comparison between test results of two types of client data (non-independent and identically distributed) on the FEMNIST dataset and test results of training on a local dataset alone, under different proportions of malicious clients as described in this specification. Figure 8c This diagram illustrates a comparison between test results of two types of client data independently and identically distributed on the CIFAR100 dataset and test results of training on a local dataset alone, under different proportions of malicious clients, as described in this specification. Figure 8d This diagram illustrates the comparison between test results of different client data with different proportions of malicious clients on the CIFAR100 dataset and test results of training on a local dataset alone, under different proportions of malicious clients.

[0241] from Figure 8a The results in ~d show that even under attack, BFKD outperforms training alone. When trained on local data alone, the model converges smoothly, but its test accuracy is low due to the limited amount of local data. This is because the amount of local data is small when the data is independently and identically distributed (i.i.d.). When the data is not independently and identically distributed (i.i.d.), not only is the amount of local data limited, but it also only contains specific data types, excluding others. The convergence process of BFKD is more volatile, especially when the data is not independently and identically distributed. This indicates that BFKD transfers knowledge to various client models through knowledge distillation, allowing them to escape locally optimal model parameters and train better model parameters. When the data is not independently and identically distributed, BFKD shows a more significant performance improvement compared to locally trained models, further demonstrating the knowledge transfer performance of the BFKD algorithm. BFKD can improve model performance by allowing other clients to learn relevant knowledge while protecting privacy and keeping data within the local machine.

[0242] To further illustrate the consensus efficiency of the BFKD algorithm, this embodiment also tested the performance of the PBFT and FSGPBFT consensus algorithms, simulating different numbers of consensus nodes in AIoT by setting different numbers of nodes. In this embodiment, the initial score of each node was set to 100, and the score value of each node after each consensus was determined using the aforementioned algorithm. In the FSGPBFT algorithm, the ratio of consensus nodes to candidate nodes was set to 1:1. This embodiment tested and analyzed the consensus latency and communication overhead of the two consensus algorithms, and the results are presented in [the table / document / etc.]. Figure 9 middle.

[0243] Figure 9 This diagram illustrates consensus latency and communication overhead in this specification. Consensus latency is the time required from when a client node sends a transaction request to the master node to when the client receives the transaction confirmation message. This embodiment uses the average consensus latency of 20 trials as the experimental result to ensure the generalizability of the results. As shown in Figure 9, the consensus latency of FSGPBFT is significantly lower than that of PBFT. When the number of nodes is 100, the average consensus latency of FSGPBFT is 0.65s, while the consensus latency of PBFT is approximately 8s. With the increase in the number of consensus nodes, the consensus latency of the PBFT algorithm increases exponentially, while the consensus latency of FSGPBFT increases more gradually. This is because FSGPBFT reduces the amount of information sent between nodes and reduces the number of nodes participating in the consensus. In FSGPBFT, response and reply phases ensure that each node can be in a consistent state. Communication overhead refers to the traffic generated when nodes in the system execute the consensus algorithm. Figure 9 The results show that as the number of consensus nodes increases, the communication overhead of both algorithms tends to increase. The more nodes there are, the more obvious the advantage of FSGPBFT becomes. When the number of nodes is 100, the communication overhead of FSGPBFT in completing the consensus process is approximately 0.14 times that of PBFT. This further illustrates that FSGPBFT has higher consensus efficiency while maintaining similar security.

[0244] In summary, this invention addresses the data privacy protection and secure sharing issues of aviation IoT devices under heterogeneous equipment conditions. It proposes a flight operation data sharing method based on federated knowledge distillation. First, generated data is used as the public dataset for federated knowledge distillation to protect public data privacy. Second, clients upload soft tags to the server instead of model parameters to prevent gradient backpropagation attacks. Furthermore, a clustering algorithm is used to select aggregated soft tags, improving resistance to poisoning attacks. Finally, blockchain is used to record and preserve the above processes, further ensuring data security. In addition, this invention proposes a consortium blockchain FSGPBFT consensus algorithm, rotating the master node and consensus node set to improve consensus efficiency and incentivize clients to upload more public data and honestly participate in federated knowledge distillation. Theoretical analysis and experimental results on the public dataset show that, compared to traditional blockchain-based federated learning algorithms, BFKD has lower communication complexity, higher consensus efficiency and convergence performance, and can be matched to heterogeneous model scenarios. This paper only tested the public dataset; further deployment and application of this method in real-world aviation IoT applications will be conducted in future studies.

[0245] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 2 A method for sharing flight operation data based on federated knowledge distillation is provided.

[0246] This specification also provides a structural diagram of the computer device shown in Figure 7. As shown in Figure 7, at the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 2 A method for sharing flight operation data based on federated knowledge distillation is provided.

[0247] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0248] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.< / response> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepare> < / prepared>

Claims

1. A method for sharing flight operation data based on federated knowledge distillation, characterized in that, include: Servers and / or aviation equipment that are added to the pre-built blockchain will act as clients, and some clients will upload public data to the blockchain; the public data includes at least one of the client's flight operation data and generated data obtained by the client based on real flight operation data through a generative model, and the public data includes flight operation data records and real labels; Each client obtains public data from the blockchain, trains its neural network model based on the data records and real labels of the public data, tests the public data using the trained neural network model, and uploads the test results as a soft label set to the blockchain. Based on the amount of public data uploaded by each client, a master node is selected from each client. The soft tag sets uploaded by each client are clustered by the master node, and soft tag sets that are different from the soft tag sets of the master node are removed. The remaining soft tag sets are then aggregated in each public data dimension to obtain an aggregated soft tag set, which is then uploaded to the blockchain. Each client obtains public data and aggregated soft tag sets from the blockchain, trains its neural network model based on the data records of the public data and the corresponding soft tags, and trains its neural network model based on local flight operation data to complete a single round of federated knowledge distillation. The neural network models of each client are trained through multiple rounds of iterative federated knowledge distillation, thereby enabling the sharing of flight operation data based on federated knowledge distillation. Each client trains its neural network model based on public data records and real labels using the following formula: in, Let cross-entropy be the loss function. For learning rate, For public datasets of federated knowledge distillation, For the neural network model parameters of the client, ← indicates update; The trained neural network model is then tested on public data according to the following formula: in, For the client i Uploaded soft tag set, For the client's neural network model; The master node clusters the soft tag sets uploaded by each client according to the following formula: in, As cluster center, m The categories for clustering; The aggregated soft tag set is obtained by aggregating the remaining soft tag set across each common data dimension according to the following formula: in, For aggregating soft tag sets, K The number of remaining soft tag sets participating in the aggregation; Based on the public data records and corresponding soft labels, the neural network model for each client is trained using the following formula: in, For learning rate, Let KL divergence loss function be used. The neural network model for each client is trained based on the following formula using local flight operation data: in, This is a local dataset for the client.

2. The flight operation data sharing method based on federated knowledge distillation as described in claim 1, characterized in that, The client generates data based on real flight operation data according to the following objective function: in, For generator, For discriminator, x For real flight operation data, y To generate labels for the data, z To generate data, w For generator parameters, Let the discriminator loss function be... To generate data.

3. The flight operation data sharing method based on federated knowledge distillation as described in claim 1, characterized in that, The master node is selected from each client based on the amount of public data uploaded by each client, specifically including: The overall score of the server nodes contained in the blockchain is initialized using the following formula: in, This is the initial comprehensive score for the consensus nodes. The amount of public data uploaded by the client. Weighting of public data scores; Based on the comprehensive scores of each client, each server node is divided into a consensus node set and a candidate node set; The master node is selected from the consensus node set using a roulette wheel algorithm.

4. The flight operation data sharing method based on federated knowledge distillation as described in claim 3, characterized in that, Some clients upload public data to the blockchain and simultaneously reach consensus on the uploaded public data within the blockchain; the master node uploads aggregated soft tag sets to the blockchain and simultaneously reaches consensus on the aggregated soft tag sets within the blockchain. During the consensus process, the master node reduces the comprehensive score of server nodes corresponding to soft tag sets that are different from the master node's soft tag set during the clustering process, and increases the comprehensive score of server nodes corresponding to soft tag sets that are of the same type as the master node's soft tag set.

Citation Information

Patent Citations

  • Data sharing method and device based on block chain and federal learning

    CN116627970A

  • Communications System, First Node, Second Node, Third Node and Methods Performed Thereby for Handling a Prediction of an Event

    US20230169356A1