Distributed intrusion detection method and system based on flexible secure and trusted federated learning
By combining trust assessment of edge network devices and blockchain verification with interpretable identification results, the security and trust issues in the data sharing process in the mobile Internet are solved, achieving efficient intrusion detection and model aggregation, and ensuring data privacy and system stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-05-28
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies in the mobile internet fail to fully consider flexibility and dynamism, lack comprehensive data privacy and security protection, and lack effective trust assessment mechanisms. This leads to risks of data leakage and tampering during data sharing, making it difficult to guarantee the reliability of node participation in model iteration and the continuous and effective operation of the system.
A distributed intrusion detection method based on flexible, secure, and trustworthy federated learning is adopted. By using edge network devices to assess the trust of training nodes and detect models, malicious updates are identified. Multiple aggregation nodes are selected for model aggregation, and blockchain is used for broadcast verification. Combined with an interpretable identification result trust assessment method, the trustworthiness and robustness of the model are ensured.
This improves the security and robustness of the federated learning process, reduces the risk of data leakage and tampering, ensures the credibility of data sharing and the verifiability of intrusion detection results, and enhances the reliability and accuracy of the system.
Smart Images
Figure CN122293441A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of network data security and sharing technology, specifically to a distributed intrusion detection method and system based on flexible, secure, and trusted federated learning. Background Technology
[0002] In the mobile internet era, mobile devices are fundamentally changing the field of information interaction with their enhanced security, efficiency, and smaller environmental impact. Mobile devices need to share data with other devices, infrastructure, and service providers to improve the efficiency and accuracy of their various functional modules. The emergence of Web3.0 and metaverse technologies has further enhanced the degree of data sharing, allowing users to benefit from it. However, data sharing also brings a series of challenges and risks, primarily including users' high demands for data privacy and security, as well as the ever-increasing network scale and complexity as sharing expands. To obtain more reliable AI models, a large amount of raw data from mobile devices needs to be collected for training; however, ensuring the security and privacy of this data during transmission has become a pressing issue.
[0003] Currently, to ensure client data privacy, some existing technologies attempt to combine data sharing with technologies such as federated learning and blockchain. For example, methods combining intrusion detection with blockchain leverage blockchain's guarantees of data storage and process transparency to address multi-party collaboration issues in data sharing. In industrial IoT scenarios, blockchain technology is used to share data among multiple parties to reduce the risk of data leakage and achieve system decentralization, preventing malicious behavior by a central server from hindering training. Furthermore, there are blockchain-based federated learning system platform architectures that create Merkle trees to ensure the verifiable integrity of client data. Additionally, there are blockchain-based hierarchical federated learning frameworks that improve the security of data sharing architectures by sharing network threat intelligence among clients.
[0004] However, existing technologies have the following problems and shortcomings: First, they fail to fully consider the flexibility and dynamism of the mobile Internet, which limits their application in practice; second, they do not provide comprehensive protection for data privacy and security, and there is a risk of data leakage and tampering; finally, they lack an effective trust assessment mechanism, making it difficult to guarantee the reliability of node participation in model iteration and the continuous and effective operation of the system. Summary of the Invention
[0005] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing a distributed intrusion detection method based on flexible, secure, and trustworthy federated learning, effectively improving system security and robustness while ensuring data privacy and the trustworthiness of data sharing. To solve the above problem, this invention adopts the following technical solution:
[0006] First, this invention proposes a distributed intrusion detection method based on flexible, secure, and trusted federated learning, comprising the following steps:
[0007] S1. Initialize the federated learning network, dividing the nodes into training nodes and aggregation nodes;
[0008] S2. Each training node collects network traffic and trains a local model; it then sends updates to the local model to nearby associated edge network devices.
[0009] S3. The edge network device uses the training node trust evaluation method to detect the local model update and update the list of trusted nodes and malicious nodes. If a new node is added, a new node check is performed. If the difference between the local model update and the global model exceeds a certain threshold, the edge network device determines that the local model update has malicious update or quality problem and refuses the model to participate in the subsequent aggregation steps.
[0010] S4. The aggregation node will aggregate the detected local models, and after the aggregation is completed, it will return the generated global model to the edge network device.
[0011] S5. Edge network devices receive global models generated by each aggregation node, verify them, and broadcast the verified global models via the blockchain.
[0012] S6. All training nodes receive and use the latest global model, start a new round of model training based on local data, obtain the final intrusion detection model, detect intrusions using the final intrusion detection model, and output the intrusion detection results.
[0013] Preferably, step S3 specifically includes the following steps:
[0014] S301. Check if there are any newly added training nodes; if there are new nodes, evaluate the behavior of the node using the training node trust evaluation method; if no malicious behavior is detected, add the new node to the trusted node list; otherwise, if the new node is detected as a malicious node, add the new node directly to the malicious node list.
[0015] S302. During the training phase, the models uploaded by the training nodes need to be evaluated in each round of aggregation; during the model generation process, each training node... Both utilize local datasets Participate in the training of machine learning (ML) models. This represents the set of model parameters for all training nodes. This represents the total number of nodes participating in the training. Indicates the first Local model parameters for each training node; each training node Each will train and generate a local ML model. After each local training session, the edge network device receives all models associated with the training nodes. Using the local test set of edge network devices To verify the accuracy of the training results, Indicates the first The local machine learning model is generated by the training nodes.
[0016] Preferably, the training node trust evaluation method includes the following steps:
[0017] Edge network devices use a multi-core mapping function to perform nonlinear error analysis on the validation benchmark dataset based on a historical global model, generating a dynamically calibrated global reference loss as a benchmark for model reliability evaluation.
[0018] Edge network devices adaptively optimize the local model uploaded by the terminal through weight parameters, fusing the global reference loss and the local expected loss to obtain a dynamic fusion loss. The edge network device calculates the deviation value based on the normalized relative difference criterion according to the global reference loss and the dynamic fusion loss. If the deviation value does not exceed the preset trust threshold, the corresponding local model is marked as a reliable model; otherwise, it is marked as unreliable and rejected from participating in subsequent aggregation steps.
[0019] Preferably, in step S4, during the aggregation phase, each time an aggregation occurs, the edge network device node selects a group of aggregation nodes to complete the aggregation task of the same batch of models; the edge network device assigns the same batch of model aggregation list to each aggregation node, and each aggregation node uses FedAvg to aggregate the models and generate the global model for this round.
[0020] Preferably, step S5 specifically includes the following steps:
[0021] S501: The edge network device receives the global model returned by each aggregation node, uses the aggregation node trust evaluation method to compare the parameter weights of the global model generated by each aggregation node, and detects whether there is an aggregation node that has been poisoned.
[0022] S502. If an aggregation node is detected as an abnormal node, then the aggregation node is removed from the current aggregation task, and the global model generated by it is rejected.
[0023] S503. Broadcast the verified global model via the blockchain so that all training nodes can download it and use it for the next round of model training tasks.
[0024] Preferably, the aggregation node trust assessment method described in step S501 specifically includes the following steps:
[0025] S5011. Vectorize the weights of the global model generated by each aggregation node, that is, expand the weight matrix of each layer into a one-dimensional vector and concatenate them to obtain the long vector corresponding to each aggregation node. ,in It is the number of the global model generated by each aggregation node;
[0026] S5012. Calculate the cosine similarity between the long vectors corresponding to every two aggregation nodes.
[0027] S5013, For each aggregation node Construct a matrix from its cosine similarity to the remaining nodes. ,in , Represents a node and nodes Cosine similarity between them Represents a node and nodes The angle between them; if the matrix middle, The number of values greater than 0 exceeds the matrix. If the number of elements is half of the total number, then it is determined that... It is reliable.
[0028] Preferably, step S6 specifically includes the following steps:
[0029] S601. Distribute the validated global model to each training node as the initial model for the next round of local training.
[0030] S602. Each training node receives the global model and performs the next round of local model training based on local data;
[0031] S603, edge network devices, and the cloud platform jointly monitor the training task execution status of each training node and aggregation node.
[0032] Preferably, the present invention further includes: performing a trust assessment on the intrusion detection results using an interpretable recognition result-based trust assessment method on the intrusion detection model after training;
[0033] The trust assessment method based on interpretable identification results includes:
[0034] Calculate the historical feature contribution ranking of each classification feature learned by the model under historical conditions, and the alarm feature contribution ranking of alarms in actual detection.
[0035] Compare the historical feature contribution ranking with the alarm feature contribution ranking, and calculate the similarity between the two;
[0036] Set a similarity evaluation threshold, and determine whether the intrusion detection result corresponding to the alarm has been tampered with based on the comparison result between the similarity and the similarity evaluation threshold.
[0037] Preferably, step S1 specifically includes the following steps:
[0038] S101. All edge network devices and IoT terminals obtain the initial global model parameters from the cloud platform. The cloud platform ensures that the model parameters are consistent across all network coverage areas. Each edge network device obtains the latest global model through the cloud platform and is responsible for distributing the global model within its network coverage area.
[0039] S102. Based on computing power, the mobile terminal is divided into training nodes and aggregation nodes. The training nodes are responsible for data collection and local model training; the aggregation nodes are responsible for receiving the local models from each training node to generate a global model.
[0040] Furthermore, this invention also proposes a distributed intrusion detection system based on flexible, secure, and trusted federated learning, comprising:
[0041] The initialization and node allocation unit is configured to perform: initialize the federated learning network, and divide the nodes into training nodes and aggregation nodes;
[0042] The model training and uploading unit is configured to perform the following: each training node collects local network traffic data, trains a local model, and uploads the local model update to nearby associated edge network devices.
[0043] The model detection and verification unit is configured to perform the following: receiving local model updates uploaded by each training node from the edge network device, detecting the uploaded local model updates using the training node trust evaluation method, and updating the list of trusted nodes and malicious nodes; if a new node is added, a new node check is performed; if the difference between the local model update and the global model exceeds a set threshold, the edge network device determines that the model update has malicious updates or quality problems and refuses to participate in subsequent aggregation steps.
[0044] The model aggregation unit is configured to perform the following: the aggregation node aggregates the local models that have not been rejected after detection, and after the aggregation is completed, returns the generated global model to the edge network device.
[0045] The aggregation result verification and broadcasting unit is configured to perform the following: the edge network device receives the global model generated by each aggregation node, verifies it, and broadcasts the verified global model through the blockchain for each training node to download;
[0046] The global model update and training unit is configured to: receive and use the latest global model from all training nodes, start a new round of model training based on local data, until the final intrusion detection model is obtained; use the model to detect intrusions and output the intrusion detection results.
[0047] The present invention, by adopting the above technical solution, has the following beneficial effects:
[0048] (1) The present invention performs trust evaluation on the local model updates uploaded by training nodes through edge network devices, which can identify local models with malicious updates or quality problems before model aggregation and refuse them to participate in subsequent aggregation, thereby reducing the impact of abnormal training nodes on the global model and improving the security and robustness of the federated learning process.
[0049] (2) By selecting multiple aggregation nodes to aggregate the same batch of local models that have passed the test, and having edge network devices verify the consistency of the global model parameter weights generated by each aggregation node, this invention can discover aggregation nodes that may have poisoning behavior and reduce the risk of model tampering during the aggregation stage.
[0050] (3) The present invention broadcasts the verified global model through the blockchain, so that each training node can obtain the verified global model. By utilizing the traceability and tamper-proof characteristics of the blockchain, the credibility of the model sharing process is improved.
[0051] (4) After obtaining the final intrusion detection model, the present invention further adopts an interpretable identification result trust evaluation method to compare the historical feature contribution ranking and the alarm feature contribution ranking, and outputs the credibility judgment result of the intrusion detection result, thereby improving the verifiability of the intrusion detection result. Attached Figure Description
[0052] Figure 1 This invention relates to a schematic diagram of the overall architecture of a distributed intrusion detection method based on flexible, secure, and trusted federated learning.
[0053] Figure 2 This invention relates to a flowchart of a distributed intrusion detection method based on flexible, secure, and trusted federated learning. Detailed Implementation
[0054] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.
[0055] Example 1: Reference Figure 1This embodiment provides a distributed intrusion detection method based on flexible, secure and reliable federated learning. This method can be applied to distributed intrusion detection scenarios in mobile Internet or IoT terminal access networks. It is especially suitable for scenarios where multiple terminal nodes are deployed in different edge network coverage areas, the original network traffic data is not suitable for centralized uploading, but the intrusion detection model needs to be trained collaboratively through federated learning.
[0056] To facilitate understanding of the data flow process between nodes in this embodiment, the main input data and output results involved in this embodiment are shown in Table 1:
[0057] Table 1
[0058]
[0059] Based on the above application scenarios and data flow relationships, the method in this embodiment includes the following steps:
[0060] Step S1, initialize the federated learning network and node allocation, including the following steps:
[0061] S101. All edge network devices and IoT terminals obtain the initial global model parameters from the cloud platform. The cloud platform ensures that the model parameters are consistent across all network coverage areas. Each edge network device obtains the latest global model through the cloud platform and is responsible for distributing the global model within its network coverage area.
[0062] S102. Based on computing power, the mobile terminal is divided into training nodes and aggregation nodes. The training nodes are responsible for data collection and local model training; the aggregation nodes are responsible for receiving the local models from each training node to generate a global model.
[0063] Step S2, Model Training and Upload. Each training node collects its own normal network traffic, preprocesses the data, and trains a local network traffic classification model, which is denoted as the local model. After local training is complete, these local models are updated and sent to nearby associated edge network devices.
[0064] Step S3: Edge network device model detection and verification. Edge network devices employ a similarity-based model evaluation method to detect uploaded local models. By calculating the similarity between the uploaded model and the global model, the edge network device can determine the model's credibility and update the lists of trusted and malicious nodes. If a new node is added, a new node check is also required. If the difference between the updated model and the global model exceeds a certain threshold, the model may be considered to have been maliciously updated or has quality issues, and the system will reject this model from participating in subsequent aggregation steps. Details are as follows:
[0065] S301. Check if there are any newly added training nodes; if there are new nodes, evaluate the behavior of the node using the training node trust evaluation method; if no malicious behavior is detected, add the new node to the trusted node list; otherwise, if the new node is detected as a malicious node, add the new node directly to the malicious node list.
[0066] S302. During the training phase, the models uploaded by the training nodes need to be evaluated in each round of aggregation; during the model generation process, each training node... Both utilize local datasets Participate in the training of machine learning (ML) models. This represents the set of model parameters for all training nodes. This represents the total number of nodes participating in the training. Indicates the first Local model parameters for each training node; each training node Each will train and generate a local ML model. After each local training session, the edge network device receives all models associated with the training nodes. Using the local test set of edge network devices To verify the accuracy of the training results, Indicates the first The local machine learning model is generated by the training nodes.
[0067] The specific steps of the training node trust evaluation method are as follows:
[0068] First, edge network devices Based on historical global model The test set is mapped using a multi-kernel mapping function. Feature space transformation is performed. Specifically, the device employs a kernel function. The nonlinear difference between the model output and the true label is mapped to a high-dimensional space, and the weighted reference loss is calculated. :
[0069] (1)
[0070] in, To determine the sample size of the test set, Indicates the first in the test set One sample, It can be configured as a Gaussian kernel or a polynomial kernel to capture the distribution characteristics of model errors.
[0071] Step S3022: Then the edge network device For terminals Uploaded local model Dynamic loss fusion is performed by optimizing the weight parameters. , global reference loss Compared with the expected loss of the local model Conduct a joint assessment:
[0072] (2)
[0073] in, For local loss function, This represents the statistical expectation based on the distribution of the test set.
[0074] Step S3023: Edge network device Model reliability is determined using the NRDC (Normalized Relative Divergence Criterion). The reference loss is calculated. With node loss Normalized differences:
[0075] (3)
[0076] in, An adjustable trust threshold. This is a very small constant used to avoid the denominator being zero. If the condition is met, then... For reliable models, participation in aggregation is allowed; otherwise, it is marked as an anomaly and isolated.
[0077] Step S4, Model Aggregation: The aggregation node is responsible for aggregating qualified local models.
[0078] Step S401: During the aggregation phase, to prevent aggregation nodes from poisoning data, each time aggregation occurs, the edge network device node selects a group of aggregation nodes to complete the aggregation task for the same batch of models. The edge network device assigns each aggregation node... Assign the same batch of model aggregation lists.
[0079] Step S402: Each aggregation node uses FedAvg (Federated Averaging) to aggregate the models and generate the global model for this round.
[0080] Step S403: After aggregation is complete, the aggregation node returns the generated global model to the edge network device.
[0081] Step 5: Verification and Broadcasting of Aggregation Results. The edge network device receives the global model generated by each aggregation node, verifies it, and broadcasts it on the blockchain. The specific steps are as follows:
[0082] Step S501: Trust assessment of aggregated nodes. Poisoning nodes are detected by comparing the weights of the parameters after model aggregation.
[0083] The specific steps of the trust assessment method for aggregation nodes are as follows:
[0084] S5011. Vectorize the weights of the global model generated by each aggregation node, that is, expand the weight matrix of each layer into a one-dimensional vector and concatenate them to obtain the long vector corresponding to each aggregation node. ,in It is the number of the global model generated by each aggregation node;
[0085] S5012. Calculate the cosine similarity between the long vectors corresponding to every two aggregation nodes:
[0086] (4)
[0087] in, It is a model and model The vector dot product, It is the norm of two vectors.
[0088] S5013, For each aggregation node Construct a matrix from its cosine similarity to the remaining nodes. ,in , Represents a node and nodes Cosine similarity between them Represents a node and nodes The angle between them; if the matrix middle, The number of values greater than 0 exceeds the matrix. If the number of elements is half of the total number, then it is determined that... It is reliable.
[0089] Step S502: If an abnormal node is detected, the node is directly excluded.
[0090] Step S503: The verified global model will be broadcast via the blockchain for all training nodes to download and use for the next round of model training.
[0091] Step 6: Global Model Update and Next Round of Training. Once the global model is successfully broadcast, all training nodes will receive and use the latest global model to begin a new round of model training based on their local data. The cloud platform and edge network devices jointly monitor the entire training process to ensure the stable operation of all nodes.
[0092] S601. Distribute the validated global model to each training node as the initial model for the next round of local training.
[0093] S602. Each training node receives the global model and performs the next round of local model training based on local data;
[0094] S603, edge network devices, and the cloud platform jointly monitor the training task execution status of each training node and aggregation node.
[0095] Step 7, Trust Assessment Based on Interpretable Recognition Results. After the previous steps of updating and iteration, the final intrusion detection model is obtained. A trust assessment of the intrusion detection results is then performed on this model using interpretability, which is essentially a "final check." Further interpretability analysis is conducted on the model that passes the similarity assessment. By generating interpretable results from the model output (such as feature contribution), the reliability of the model training is evaluated to ensure that it meets the system requirements.
[0096] Combination Figure 2 The method for calculating the credibility of interpretable results includes three steps, as follows:
[0097] Step S701: Calculate the contribution ranking of each classification feature learned by the model under historical conditions, and the feature contribution ranking of alarms in actual detection.
[0098] Hypothetical sample set The The sample is ,sample The The features are ,sample The The features are ,in and Indexes for different features ( ), The total number of features in the sample.
[0099] Based on historical training data, the model reconstructs the feature contribution calculation logic by introducing the entropy weight method and nonlinear compression function.
[0100] First, calculate the information entropy of each feature to quantify its discriminative power:
[0101] (5)
[0102] Features with lower entropy values contain more information and are given higher weight.
[0103] Next, the standard deviation of each feature is calculated using historical data, which is used to adjust the scale of the nonlinear transformation.
[0104] (6)
[0105] Then, through The function performs nonlinear compression and amplification on the eigenvalues, suppressing noise and enhancing the contribution differences of key features:
[0106] (7)
[0107] in: The total number of historical samples. Features Historical average, The characteristic standard deviation is denoted as .
[0108] This method generates a ranking list of historical feature contributions. At the same time, the same process is used to generate a ranking list for the alarm samples. ,
[0109] Step S702: Compare the two contribution ranking lists and calculate their similarity.
[0110] To measure the deviation between historical and alarm feature rankings, a weighted Spearman rank correlation coefficient is designed, incorporating feature contribution as a weight into the similarity calculation:
[0111] (8)
[0112] in and Representing features respectively Ranking in history and alerts.
[0113] The weights Reflecting feature importance, the ranking differences of high-weight features have a greater impact on similarity, thus making them more sensitive to anomaly identification, as detailed below:
[0114] (9)
[0115] Step S703: Set a similarity assessment threshold to determine whether the detection result has been tampered with.
[0116] Based on historical similarity data Calculate the mean with standard deviation And set dynamic thresholds
[0117] (10)
[0118] In this embodiment, the confidence factor is set to 2, corresponding to a 95% confidence interval. Wherein: This is the confidence factor, which defaults to 2 (corresponding to the 95% confidence interval under a normal distribution).
[0119] Based on the similarity of alarm samples and The size relationship is used to determine whether the detected alarm has been tampered with:
[0120] (11)
[0121] in, This indicates the detection result. When the similarity of the alarm samples meets the similarity evaluation threshold, the intrusion detection result is output as reliable; when the similarity of the alarm samples does not meet the similarity evaluation threshold, the intrusion detection result is output as abnormal or at risk of being tampered with. Therefore, the system can not only output the intrusion detection category or alarm result, but also simultaneously output the reliability judgment result of the detection result.
[0122] Example 2: This example provides a distributed intrusion detection system based on flexible, secure, and trusted federated learning, used to implement the distributed intrusion detection method based on flexible, secure, and trusted federated learning. It includes an initialization and node allocation unit, a model training and uploading unit, a model detection and verification unit, a model aggregation unit, an aggregation result verification and broadcasting unit, and a global model update and training unit, wherein:
[0123] The initialization and node allocation unit is used to initialize the federated learning network and divide the nodes into training nodes and aggregation nodes.
[0124] The model training and uploading unit is used by each training node to collect its own normal network traffic and train the local model. The local model updates are then sent to nearby associated edge network devices.
[0125] The model detection and verification unit is used to detect uploaded local models through edge network devices using a training node trust evaluation method, and to update the lists of trusted and malicious nodes. If a new node is added, a new node check is performed. If the difference between the updated model and the global model exceeds a certain threshold, the model is considered to have been maliciously updated or has quality issues, and the model is rejected from participating in subsequent aggregation steps.
[0126] The model aggregation unit is used to aggregate qualified local models through aggregation nodes. After the aggregation is completed, the generated global model is returned to the edge network device.
[0127] The aggregation result verification and broadcasting unit is used to receive the global model generated by each aggregation node through the edge network device, verify it, and broadcast it on the blockchain.
[0128] The global model update and training unit is used to receive and use the latest global model through all training nodes, start a new round of model training based on local data, obtain the final intrusion detection model, and detect intrusions through the final intrusion detection model.
[0129] This invention divides mobile terminals into model training nodes and model aggregation nodes based on computing power. As training nodes, the mobile terminal only needs to train the traffic classification model using its local data and upload the model updates to the edge network device. Aggregation nodes, coordinated by the edge network device, are responsible for summarizing and integrating local models from multiple training nodes to generate a global model. Throughout the federated learning process, the edge network device plays a crucial coordinating and supervisory role. Figure 2 In each training round, the edge network device not only provides the initial model to the training nodes but also detects and validates the local models uploaded by each node, using a similarity-based trust assessment method to eliminate potentially unreliable models. At the initial stage of training, each edge network device is assigned a specific test dataset, which is used as a standard to determine whether the model updates submitted by the training nodes meet quality requirements. To prevent potential malicious behavior from aggregation nodes, especially poisoning attacks, this architecture employs an aggregation node trust assessment strategy. During each training round, the edge network device selects multiple aggregation nodes to perform model aggregation operations. After aggregation, the edge network device evaluates the aggregated models provided by the multiple aggregation nodes, eliminating abnormal aggregation nodes through model validation and consistency checks to ensure the reliability and accuracy of the global model. After the global model training is complete, an interpretable intrusion detection result trust assessment is performed. This design effectively improves the system's security and robustness while ensuring data privacy and the trustworthiness of shared data.
[0130] It should be noted that the processing flow of Embodiment 2 corresponds to the specific steps of the method provided in Embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the method provided in Embodiment 1 of the present invention.
[0131] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0132] The specific implementation schemes described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific implementation schemes of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present invention should fall within the scope of protection of the present invention.
Claims
1. A distributed intrusion detection method based on flexible, secure, and trusted federated learning, characterized in that, Includes the following steps: S1. Initialize the federated learning network, dividing the nodes into training nodes and aggregation nodes; S2. Each training node collects network traffic and trains a local model; it then sends updates to the local model to nearby associated edge network devices. S3. The edge network device uses the training node trust evaluation method to detect the local model update and update the list of trusted nodes and malicious nodes. If a new node is added, a new node check is performed. If the difference between the local model update and the global model exceeds a certain threshold, the edge network device determines that the local model update has malicious update or quality problem and refuses the model to participate in the subsequent aggregation steps. S4. The aggregation node will aggregate the detected local models, and after the aggregation is completed, it will return the generated global model to the edge network device. S5. Edge network devices receive global models generated by each aggregation node, verify them, and broadcast the verified global models via the blockchain. S6. All training nodes receive and use the latest global model, start a new round of model training based on local data, obtain the final intrusion detection model, detect intrusions using the final intrusion detection model, and output the intrusion detection results.
2. The method according to claim 1, characterized in that, Step S3 specifically includes the following steps: S301. Check if there are any newly added training nodes; if there are new nodes, evaluate the behavior of the node using the training node trust evaluation method; if no malicious behavior is detected, add the new node to the trusted node list; otherwise, if the new node is detected as a malicious node, add the new node directly to the malicious node list. S302. During the training phase, the models uploaded by the training nodes need to be evaluated in each round of aggregation; during the model generation process, each training node... Both utilize local datasets Participate in the training of machine learning (ML) models. This represents the set of model parameters for all training nodes. This represents the total number of nodes participating in the training. Indicates the first Local model parameters for each training node; each training node Each will train and generate a local ML model. After each local training session, the edge network device receives all models associated with the training nodes. Using the local test set of edge network devices To verify the accuracy of the training results, Indicates the first The local machine learning model is generated by the training nodes.
3. The method according to claim 2, characterized in that, The training node trust evaluation method includes the following steps: Edge network devices use a multi-core mapping function to perform nonlinear error analysis on the validation benchmark dataset based on a historical global model, generating a dynamically calibrated global reference loss as a benchmark for model reliability evaluation. Edge network devices adaptively optimize the local model uploaded by the terminal through weight parameters, fusing the global reference loss and the local expected loss to obtain a dynamic fusion loss. The edge network device calculates the deviation value based on the normalized relative difference criterion according to the global reference loss and the dynamic fusion loss. If the deviation value does not exceed the preset trust threshold, the corresponding local model is marked as a reliable model; otherwise, it is marked as unreliable and rejected from participating in subsequent aggregation steps.
4. The method according to claim 3, characterized in that, In step S4, during the aggregation phase, each time the edge network device selects a group of aggregation nodes to complete the aggregation task of the same batch of models; the edge network device assigns the same batch of model aggregation list to each aggregation node, and each aggregation node uses FedAvg to aggregate the models and generate the global model for this round.
5. The method according to claim 4, characterized in that, Step S5 specifically includes the following steps: S501: The edge network device receives the global model returned by each aggregation node, uses the aggregation node trust evaluation method to compare the parameter weights of the global model generated by each aggregation node, and detects whether there is an aggregation node that has been poisoned. S502. If an aggregation node is detected as an abnormal node, then the aggregation node is removed from the current aggregation task, and the global model generated by it is rejected. S503. Broadcast the verified global model via the blockchain so that all training nodes can download it and use it for the next round of model training tasks.
6. The method according to claim 5, characterized in that, The aggregation node trust assessment method described in step S501 specifically includes the following steps: S5011. Vectorize the weights of the global model generated by each aggregation node, that is, expand the weight matrix of each layer into a one-dimensional vector and concatenate them to obtain the long vector corresponding to each aggregation node. ,in It is the number of the global model generated by each aggregation node; S5012. Calculate the cosine similarity between the long vectors corresponding to every two aggregation nodes. S5013, For each aggregation node Construct a matrix from its cosine similarity to the remaining nodes. ,in , Represents a node and nodes Cosine similarity between them Represents a node and nodes The angle between them; if the matrix middle, The number of values greater than 0 exceeds the matrix. If the number of elements is half of the total number, then it is determined that... It is reliable.
7. The method according to claim 1, characterized in that, Step S6 specifically includes the following steps: S601. Distribute the validated global model to each training node as the initial model for the next round of local training. S602. Each training node receives the global model and performs the next round of local model training based on local data; S603, edge network devices, and the cloud platform jointly monitor the training task execution status of each training node and aggregation node.
8. The method according to claim 1, characterized in that, Also includes: After the intrusion detection model has been trained, a trust evaluation method based on interpretable recognition results is used to evaluate the trust of the intrusion detection results. The trust assessment method based on interpretable identification results includes: Calculate the historical feature contribution ranking of each classification feature learned by the model under historical conditions, and the alarm feature contribution ranking of alarms in actual detection. Compare the historical feature contribution ranking with the alarm feature contribution ranking, and calculate the similarity between the two; Set a similarity evaluation threshold, and determine whether the intrusion detection result corresponding to the alarm has been tampered with based on the comparison result between the similarity and the similarity evaluation threshold.
9. The method according to claim 1, characterized in that, Step S1 specifically includes the following steps: S101. All edge network devices and IoT terminals obtain the initial global model parameters from the cloud platform. The cloud platform ensures that the model parameters are consistent across all network coverage areas. Each edge network device obtains the latest global model through the cloud platform and is responsible for distributing the global model within its network coverage area. S102. Based on computing power, the mobile terminal is divided into training nodes and aggregation nodes. The training nodes are responsible for data collection and local model training; the aggregation nodes are responsible for receiving the local models from each training node to generate a global model.
10. A distributed intrusion detection system based on flexible, secure, and trusted federated learning, characterized in that, include: The initialization and node allocation unit is configured to perform: initialize the federated learning network, and divide the nodes into training nodes and aggregation nodes; The model training and uploading unit is configured to perform the following: each training node collects local network traffic data, trains a local model, and uploads the local model update to nearby associated edge network devices. The model detection and verification unit is configured to perform the following: receiving local model updates uploaded by each training node from the edge network device, detecting the uploaded local model updates using the training node trust evaluation method, and updating the list of trusted nodes and malicious nodes; if a new node is added, a new node check is performed; if the difference between the local model update and the global model exceeds a set threshold, the edge network device determines that the model update has malicious updates or quality problems and refuses to participate in subsequent aggregation steps. The model aggregation unit is configured to perform the following: the aggregation node aggregates the local models that have not been rejected after detection, and after the aggregation is completed, returns the generated global model to the edge network device. The aggregation result verification and broadcasting unit is configured to perform the following: the edge network device receives the global model generated by each aggregation node, verifies it, and broadcasts the verified global model through the blockchain for each training node to download; The global model update and training unit is configured to: receive and use the latest global model from all training nodes, start a new round of model training based on local data, until the final intrusion detection model is obtained; use the model to detect intrusions and output the intrusion detection results.