Network intrusion detection method and system based on federated learning and hybrid clustering

Abstract

The invention discloses a federated learning and hybrid clustering network intrusion detection method and system, and relates to the technical field of network intrusion detection. The method comprises the following steps: a server issues a current global model parameter to a client participating in the current round of training; after each client receives the global model parameter, taking the global model parameter as an initialization parameter of a local model, performing training by using local data, introducing a clustering-based soft label generation mechanism and a classification and clustering parallel dual-task learning framework in the training process for training, and updating the local model parameter; the client encrypts and uploads the trained model parameters to the server; the server aggregates all client model parameters, generates a new generation of global model, and issues the parameters back to the client for next round of training; and after all rounds of training are finished, the server issues a final model for the client to carry out network intrusion discrimination. According to the method, the negative influence of data non-independent identical distribution is effectively overcome, and sparse attacks are accurately detected.

Description

Technical Field

[0001] This invention relates to the field of network intrusion detection technology, and in particular to a network intrusion detection method and system based on federated learning and hybrid clustering. Background Technology

[0002] With the widespread adoption of cloud computing, the Internet of Things (IoT), and mobile edge computing, the network attack surface has expanded dramatically, and attack methods have become increasingly complex and covert. Network Intrusion Detection Systems (NIDS), as a core line of defense for network security, are crucially critical in their accuracy. Traditional signature-based detection methods struggle to cope with unknown attacks, while machine learning-based detection methods show great potential due to their ability to learn complex patterns in both normal and abnormal traffic. However, the performance of machine learning models heavily relies on large-scale, diverse training data, creating a fundamental contradiction between improving model performance and protecting data privacy.

[0003] Federated learning provides a framework for resolving this contradiction. It allows multiple clients to train models on local data, uploading only the model parameters (not the raw data) to a server for aggregation, thus achieving collaborative learning while protecting data privacy. However, when applying federated learning to network intrusion detection practices, a key technical challenge emerges: the non-independent and identically distributed (Non-IID) data. Specifically, due to differences in user groups, business types, and security postures, different network nodes (clients) generate vastly different distributions of network traffic data—some nodes may rarely encounter specific types of attacks (such as APT attacks), resulting in extremely sparse or even missing samples of such attacks in the local dataset. Under standard algorithms such as FedAvg, this bias in data distribution causes the aggregated global model to be overly skewed towards clients with large amounts of data, creating a "detection blind spot" for sparse but high-risk attack categories, severely weakening the actual protective capability of the global model. To improve detection accuracy, researchers have designed complex models such as one-dimensional convolutional neural networks (1D-CNN), long short-term memory networks (LSTM), and Transformer-based models. These models excel at capturing local features, long-range dependencies, and global contextual information in network traffic. However, these approaches have inherent limitations:

[0004] Privacy risks: They rely on centralizing data from various locations, which violates data privacy protection principles.

[0005] High deployment costs: Such models typically have large parameters and huge computational and storage overhead, making them difficult to deploy efficiently on edge devices or network gateways with limited computing resources.

[0006] Therefore, for those skilled in the art, designing a high-precision network intrusion detection scheme to address the aforementioned shortcomings is an urgent problem to be solved. Summary of the Invention

[0007] The purpose of this invention is to provide a network intrusion detection method and system that combines federated learning and hybrid clustering to address the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following solution: On one hand, it provides a network intrusion detection method combining federated learning and hybrid clustering, the specific steps of which include the following:

[0009] The server distributes the current global model parameters to all clients participating in this round of training.

[0010] After receiving the global model parameters, each client uses them as the initialization parameters for its local model and trains it using its local private data.

[0011] A clustering-based soft label generation mechanism is introduced during the local model training process. A dual-task learning framework that combines classification and clustering is used for training to update the local model parameters.

[0012] The client encrypts and uploads the trained model parameters to the server.

[0013] The server aggregates all received client model parameters, generates a new generation of global model, and sends the parameters back to the client for the next round of training.

[0014] After all training rounds are completed on the server, the final model is generated and made available to the client.

[0015] Obtain network traffic data and use the final model to determine whether there is a network intrusion.

[0016] Preferably, the local model includes an input layer, a feature extraction layer, a Transformer module, and a loss calculation module; wherein, the input layer includes a convolutional layer, a batch normalization layer, an adaptive average pooling layer, and a fully connected layer; each of the convolutional layers uses a convolutional operation with a kernel size of 3, and the batch normalization layer is introduced after each convolutional layer, and the feature length is unified using an adaptive average pooling layer, and finally, classification is performed through the fully connected layer.

[0017] Preferably, in the feature extraction layer, a residual connection structure is designed to add the flattened features of the convolutional layer to the output of the fully connected layer of the main branch of the local model through a linear transformation.

[0018] Preferably, the loss calculation module includes a classification header and a clustering header. The classification loss is calculated using the classification header, and the clustering loss is calculated using the clustering header. A joint loss is calculated based on the classification loss and the clustering loss. The clustering loss is calculated only when the clustering header is configured to be enabled and soft labels are generated; otherwise, only the classification loss is used.

[0019] Preferably, the training process of the local model is as follows:

[0020] The input network traffic data is processed by a one-dimensional convolutional neural network. The convolutional layer automatically learns the local spatiotemporal patterns in the sequence, and its convolutional kernel slides in one dimension to extract local features.

[0021] The local features are processed using a residual connection structure and added to the output of the fully connected layer of the main branch. The ReLU activation function is then used to perform a nonlinear transformation on the features.

[0022] The multi-head self-attention mechanism in the Transformer module is used to capture the dependencies between different features in the input sequence, thereby achieving long-distance feature modeling.

[0023] The features processed by the Transformer module are simultaneously fed into two parallel task heads: the classification head and the clustering head, to obtain the classification loss and the clustering loss, respectively.

[0024] The joint loss is calculated based on the classification loss and the clustering loss. During training, the model parameters are updated by the optimizer based on the gradient of the joint loss.

[0025] Preferably, the multi-head self-attention mechanism in the Transformer module takes 128-dimensional features as input and sets 4 attention heads.

[0026] On the other hand, a network intrusion detection system combining federated learning and hybrid clustering is provided, including a parameter distribution module, a local receiving module, a local model training module, an encrypted upload module, an update module, and a detection module; wherein,

[0027] The parameter distribution module is used by the server to distribute the current global model parameters to all clients participating in this round of training.

[0028] The local receiving module is used by each client to receive the global model parameters, use them as the initialization parameters of the local model, and train the model using local private data.

[0029] The local model training module is used to introduce a clustering-based soft label generation mechanism during the local model training process, and to train the local model using a dual-task learning framework that combines classification and clustering, thereby updating the local model parameters.

[0030] The encrypted upload module is used by the client to encrypt and upload the updated model parameters obtained after training to the server.

[0031] The update module is used by the server to aggregate all received model updates, generate a new generation of global model, and send the parameters back to the client for the next round of training. After all rounds of training are completed, the server generates the final model.

[0032] The detection module is used to acquire network traffic data and use a new generation of global models to determine whether there is a network intrusion.

[0033] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: a hybrid model architecture integrating clustering soft labels and dual task heads is designed, and a collaborative training algorithm based on joint loss function is proposed. Under the federated learning framework, the negative impact of non-independent and identically distributed data is effectively overcome, providing accurate and robust detection capabilities for security protection in distributed network environments. This helps to achieve security perception that is more in line with the actual threat situation, while ensuring data privacy and meeting the low resource requirements of edge environments. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the method of the present invention;

[0036] Figure 2 This is a local model diagram of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] The purpose of this invention is to provide a network intrusion detection method that combines federated learning and hybrid clustering, such as... Figure 1 As shown, the specific steps include the following:

[0039] S1. The server distributes the current global model to all clients participating in this round of training.

[0040] S2. After receiving the global model parameters, each client uses them as the initialization parameters for its local model and trains it using local private data.

[0041] S3. In the local model training process, a cluster-based soft label generation mechanism is introduced, and a dual-task learning framework that combines classification and clustering is used for training to update the local model parameters.

[0042] S4. The client updates and encrypts the model parameters obtained after training, and uploads them to the server.

[0043] S5. The server aggregates all received client model parameters, generates a new generation of global model, and sends the parameters back to the client for the next round of training.

[0044] S6. After all training rounds have been completed, the server distributes the final model generated for client use.

[0045] S7. Obtain network traffic data and use the final model to determine whether there is a network intrusion.

[0046] Federated learning is a distributed machine learning paradigm. Its core lies in multiple clients collaboratively training a global machine learning model under the coordination of a server, without needing to centralize their local training datasets to the server. This process is achieved by repeatedly executing the following steps: each client downloads the current global model from the server, trains it on its local dataset, and then uploads the trained model parameters (such as gradients or parameters) to the server; the server aggregates the models from all clients, generates an improved global model, and distributes it again for the next round of training; after all rounds of training are completed, the server generates the final model and distributes it to the clients for use.

[0047] This invention utilizes a federated learning framework to enable multiple network nodes to collaboratively train an intrusion detection model while protecting the privacy of each node's local network traffic data.

[0048] Furthermore, the horizontal federated learning framework, acting as the system's coordination hub, manages the model exchange and aggregation process between clients and servers, ensuring privacy protection during the learning process. Its operation follows a classic paradigm. The local model, as the system's detection core, is deployed on each client. This model employs a novel multi-task learning architecture designed to fully utilize local data for efficient and robust feature learning. In this invention, the local model aims to fully leverage the local feature extraction capabilities of CNNs and the long-range sequence dependency capture capabilities of Transformers. Simultaneously, a clustering-based method is introduced to generate soft labels and incorporate them into the training process, enhancing the model's robustness to data distribution and thus improving detection capabilities in various complex scenarios.

[0049] The method of this invention is actually divided into two stages: the training stage involves federated learning and neural networks. During training, each client trains a neural network model (a hybrid clustering network) locally using training data. Then, the model parameters are uploaded to the central server for aggregation. After aggregation, the central server reissues the model parameters, and the training is repeated. Finally, the generated final model parameters are issued, and the client obtains a model with better performance. The detection stage involves neural networks but not federated learning. The user inputs network traffic, which is preprocessed and then input into the model. The model performs detection and outputs the results.

[0050] like Figure 2 As shown, the training process of the local model is as follows:

[0051] S31. The input network traffic data is processed by a one-dimensional convolutional neural network, containing two convolutional layers. Each layer uses a convolution operation with a kernel size of 3, and extracts local features stepwise through 64 and 128 filters. To improve the model's convergence speed and generalization ability, batch normalization is introduced after each convolutional layer, and an adaptive average pooling layer is used to unify the feature length, enhancing the model's adaptability to inputs of different lengths.

[0052] S32. In the feature extraction stage, the residual connection structure is used to process local features. The flattened features of the convolutional layer are subjected to a linear transformation and added to the output of the fully connected layer of the main branch. Subsequently, the model uses the ReLU activation function to perform a non-linear transformation on the features.

[0053] The introduction of residual structures can alleviate the vanishing gradient problem common in deep models, and improve the stability of training and the model's feature representation ability.

[0054] S33. Employing the multi-head self-attention mechanism from the Transformer module, the input consists of 128-dimensional features, with four attention heads set. Through the self-attention mechanism, the model can capture the dependencies between different features in the input sequence, achieving long-distance feature modeling.

[0055] S34. The features processed by the Transformer module are simultaneously fed into two parallel task heads: the classification head and the clustering head, to obtain the classification loss and the clustering loss, respectively.

[0056] S35. Calculate the joint loss based on the classification loss and clustering loss. During training, the model parameters are updated by the optimizer based on the gradient of the joint loss.

[0057] Furthermore, in S31, 1D-CNN is used to efficiently extract discriminative local features from network traffic sequences. A one-dimensional convolutional neural network is a deep learning model specifically designed for processing sequential data; its convolutional kernels slide along a one-dimensional plane to extract local features. For network intrusion detection tasks, the input network traffic data (such as traffic statistical feature vectors arranged in chronological order) can be considered as a one-dimensional sequence. 1D-CNN automatically learns local spatiotemporal patterns in the sequence through convolutional layers, downsamples through pooling layers to enhance feature robustness, and finally performs classification through fully connected layers.

[0058] Furthermore, in S33, the Transformer is a deep learning model architecture entirely based on a self-attention mechanism, abandoning traditional recurrent or convolutional structures. Its core component is the self-attention layer, which can calculate the correlation weights between any two positions in a sequence, thereby efficiently capturing long-distance dependencies. While Transformer models are typically composed of stacked encoders and decoders, this invention primarily utilizes its self-attention module for deep encoding of network traffic features, enhancing feature representation capabilities to improve the accuracy of intrusion detection.

[0059] This invention is built upon a federated learning framework, integrating 1D-CNN and Transformer as core feature extractors. Federated learning ensures that the original data never leaves the local machine, fundamentally mitigating the risk of privacy breaches. Meanwhile, the robust local model structure guarantees the ability to learn complex attack patterns from decentralized data. Ultimately, under the constraint of "data available but not visible," it successfully achieved detection accuracy comparable to or even superior to centralized training.

[0060] Furthermore, clustering algorithms are a class of unsupervised machine learning methods whose goal is to divide samples in a dataset into several clusters, such that samples within the same cluster have high similarity, while samples between different clusters have low similarity. Common clustering algorithms include K-Means, DBSCAN, and Gaussian mixture models. In this invention, clustering algorithms are used to discover patterns in local network traffic data on the client side, for example, grouping similar traffic behaviors into one category to generate soft labels for model training or reveal potential anomalous patterns.

[0061] Soft labels are a form of label representation relative to "hard labels" (i.e., single, definitive category labels, such as one-hot vectors). They are typically a probability distribution vector representing the likelihood or confidence that a sample belongs to each category. For example, a sample's soft label could be [Normal: 0.2, Attack: 0.8], rather than a hard "attack" label. In this invention, soft labels are generated from the clustering results of the local model. During the local training phase, they provide richer and smoother supervisory information through a joint loss function, helping to improve the robustness and performance of the global model. This invention introduces a clustering-based soft label generation mechanism. By applying the KMeans clustering algorithm to standardized feature data, the distance from each sample to each cluster center is obtained. Subsequently, an exponential decay function is used to convert these distances into a probability distribution, forming a soft label corresponding to each sample. This mechanism can unsupervisedly mine the inherent distribution structure of the data, generating probabilistic guidance signals for labeled sparse minority class attacks and even novel attack patterns. Through the clustering loss function, the model is explicitly guided to learn more robust feature representations that transcend the limitations of hard labels. Ultimately, the model's sensitivity to client-specific sparse attacks and emerging unknown attacks is significantly enhanced, effectively reducing detection blind spots.

[0062] Furthermore, in S35, a joint loss function is used for optimization. The joint loss function is a comprehensive optimization objective composed of multiple linearly or nonlinearly combined loss components. In machine learning, by jointly optimizing multiple loss functions, the model can simultaneously consider different learning tasks or constraints. In the hybrid clustering network of this invention, the classification loss is calculated as the cross-entropy loss between the logits output of the classification head and the hard labels of the input data; the clustering loss is calculated as the KL divergence loss between the clustering probability distribution output of the clustering head and the soft labels generated based on clustering during the data preprocessing stage. The formula for calculating the joint loss is: JointLoss = α·L class +(1-α)L cluster Where α is a hyperparameter, L class For classification loss, L cluster This represents the clustering loss.

[0063] This invention employs a dual-task learning framework that combines classification and clustering with a weighted joint loss function. This design forces the model parameters to simultaneously optimize the discriminative classification boundary and the intrinsic clustering structure, making the learned feature representations insensitive to changes in data distribution. Ultimately, the aggregated global model can better adapt to the vastly different network traffic patterns (Non-IID) among different clients, demonstrating excellent generalization performance and deployment stability.

[0064] On the other hand, a network intrusion detection system combining federated learning and hybrid clustering is provided, including a weight distribution module, a local receiving module, a local model training module, an encrypted upload module, an update module, and a detection module; wherein,

[0065] The parameter distribution module is used by the server to distribute the current global model parameters to all clients participating in this round of training.

[0066] The local receiving module is used by each client to receive the global model parameters, use them as the initialization parameters of the local model, and train the model using local private data.

[0067] The local model training module is used to introduce a clustering-based soft label generation mechanism during the local model training process, and to train the local model using a dual-task learning framework that combines classification and clustering in parallel, and to update the local model parameters.

[0068] The encrypted upload module is used by the client to encrypt and upload the trained model parameters to the server.

[0069] The update module is used by the server to aggregate all received models, generate a new generation of global model, and send the parameters back to the client for the next round of training. After the server sends all rounds of training, the final model is generated.

[0070] The detection module is used to acquire network traffic data and use the final model to determine whether there is a network intrusion.

[0071] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A network intrusion detection method combining federated learning and hybrid clustering, characterized in that, The specific steps include the following: The server distributes the current global model parameters to all clients participating in this round of training. After receiving the global model parameters, each client uses them as the initialization parameters for its local model and trains it using its local private data. A clustering-based soft label generation mechanism is introduced during the local model training process. A dual-task learning framework that combines classification and clustering is used for training to update the local model parameters. The client encrypts and uploads the trained model parameters to the server. The server aggregates all received client model parameters, generates a new generation of global model, and sends the parameters back to the client for the next round of training. After all training rounds are completed on the server, the final model is generated and made available to the client. Obtain network traffic data and use the final model to determine whether there is a network intrusion.

2. The network intrusion detection method based on federated learning and hybrid clustering according to claim 1, characterized in that, The local model includes an input layer, a feature extraction layer, a Transformer module, and a loss calculation module. The input layer includes a convolutional layer, a batch normalization layer, an adaptive average pooling layer, and a fully connected layer. Each convolutional layer uses a convolution operation with a kernel size of 3. Batch normalization is introduced after each convolutional layer, and an adaptive average pooling layer is used to unify the feature length. Finally, the fully connected layer is used for classification.

3. The network intrusion detection method based on federated learning and hybrid clustering according to claim 2, characterized in that, In the feature extraction layer, a residual connection structure is designed to add the flattened features of the convolutional layer to the output of the fully connected layer of the main branch of the local model through a linear transformation.

4. The network intrusion detection method based on federated learning and hybrid clustering according to claim 2, characterized in that, The loss calculation module includes a classification header and a clustering header. The classification loss is calculated using the classification header, and the clustering loss is calculated using the clustering header. The joint loss is calculated based on the classification loss and the clustering loss.

5. The network intrusion detection method based on federated learning and hybrid clustering according to claim 1, characterized in that, The training process of the local model is as follows: The input network traffic data is processed by a one-dimensional convolutional neural network. The convolutional layer automatically learns the local spatiotemporal patterns in the sequence, and its convolutional kernel slides in one dimension to extract local features. The local features are processed using a residual connection structure and added to the output of the fully connected layer of the main branch. The ReLU activation function is then used to perform a nonlinear transformation on the features. The multi-head self-attention mechanism in the Transformer module is used to capture the dependencies between different features in the input sequence, thereby achieving long-distance feature modeling. The features processed by the Transformer module are simultaneously fed into two parallel task heads: the classification head and the clustering head, to obtain the classification loss and the clustering loss, respectively. The joint loss is calculated based on the classification loss and the clustering loss. During training, the model parameters are updated by the optimizer based on the gradient of the joint loss.

6. The network intrusion detection method based on federated learning and hybrid clustering according to claim 5, characterized in that, The multi-head self-attention mechanism in the Transformer module takes 128-dimensional features as input and sets 4 attention heads.

7. A network intrusion detection system combining federated learning and hybrid clustering, characterized in that, It includes a parameter distribution module, a local receiving module, a local model training module, an encrypted upload module, an update module, and a detection module; among which, The parameter distribution module is used by the server to distribute the current global model parameters to all clients participating in this round of training. The local receiving module is used by each client to receive the global model parameters, use them as the initialization parameters of the local model, and train the model using local private data. The local model training module is used to introduce a clustering-based soft label generation mechanism during the local model training process, and to train the local model using a dual-task learning framework that combines classification and clustering, thereby updating the local model parameters. The encrypted upload module is used by the client to encrypt and upload the trained model parameters to the server. The update module is used by the server to aggregate all received client model parameters, generate a new generation of global model, and send the parameters back to the client for the next round of training. After all rounds of training are completed, the server generates the final model. The detection module is used to acquire network traffic data and use the final model to determine whether there is a network intrusion.

Images