Internet of Things intrusion detection method, model, training method, and electronic device based on Transformer and GNN hybrid model

Abstract

The invention relates to an Internet of Things intrusion detection method, model and training method thereof and electronic device based on a hybrid model of Transformer and GNN, which belongs to the field of deep learning technology. In order to solve the problem of simultaneously learning the statistical characteristics, temporal characteristics and topological characteristics of traffic to utilize network space characteristics, it is used for Internet of Things intrusion detection. The key points are to positionally encode the network traffic data in the Internet of Things, input it into the encoder of the Transformer model, and output a first embedding embedding vector; convert the network flow into a first graph, with the host as the node of the first graph and the network flow as the edge of the first graph; convert the first graph into a second graph through a Line Graph model, with the network flow as the node of the second graph and the relationship between the host pairs as the edge of the second graph; input the second graph into the GNN model, and output a second embedding embedding vector; splice the first embedding embedding vector with the second embedding embedding vector to obtain a splicing vector, input the splicing vector into a classifier, and the classifier outputs the Internet of Things intrusion detection result, so that the topology information can be captured and the correlation between different traffic flows can be captured.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology and relates to an Internet of Things intrusion detection method based on a Transformer and GNN hybrid model. Background Art

[0002] Cybersecurity has always been a critical issue. With the increasing scale and diversity of threats, coupled with the growing trend of remote work and online commerce, 31% of companies worldwide are now attacked by cybercriminals at least once a day. Furthermore, the deep integration of the intelligent Internet of Things (IoT) has significantly increased the security risks faced by complex, heterogeneous network systems such as the Industrial Internet of Things (IIoT) and the Internet of Vehicles (IoV). Cyber ​​threats are becoming increasingly large and diverse, expanding their scope to include phishing campaigns, video conferencing attacks, and distributed denial of service (DDoS). Therefore, an efficient and robust network intrusion detection system is crucial to ensuring the sustainable development and security of cyberspace. An effective and robust intrusion detection system is crucial.

[0003] Intrusion detection algorithms identify intrusions by monitoring network traffic in real time. By monitoring the status and activity of protected systems and employing misuse or anomaly detection, they can identify unauthorized or malicious system and network behavior, providing an effective means of preventing intrusions. As the foundational infrastructure for network perimeter security and the backbone of proactive security protection, IoT intrusion detection, coupled with the increasingly complex and important nature of IoT network environments, requires high reliability, performance, practicality, and accuracy.

[0004] Intrusion detection provides a method for detecting intrusion attacks and privilege abuse by legitimate users. Its application premise is that intrusion behavior and legitimate behavior are distinguishable. That is, the nature of the behavior can be determined by extracting behavioral patterns. Intrusion detection systems must address two challenges: 1. How to adequately and reliably extract data describing behavioral characteristics; 2. How to efficiently and accurately determine the nature of the behavior based on these characteristic data.

[0005] Traditional intrusion detection models fall into two main categories: rule-based (signature-based) and anomaly-based IDS (Intrusion Detection System). While both approaches achieve good results in many scenarios, they both suffer from numerous challenges. Initial rule-based intrusion detection systems detect intrusions based on known attack patterns, resulting in low zero-day attack detection rates, poor scalability, and low detection accuracy. They are difficult to generalize to new attacks, and even minor modifications to the attack sequence can circumvent the rules. Traditional machine learning algorithms, such as Bayesian networks, k-nearest neighbors, and support vector machines, can use small amounts of data to construct feature engineering algorithms to identify intrusions. However, feature engineering is difficult to accurately construct and requires domain expertise to select. This process is not only labor-intensive and expensive, but the quality of the extracted features directly impacts the performance of the final detection system. Furthermore, they suffer from limited robustness against adversarial attacks. Anomaly-based intrusion detection uses machine learning and deep learning techniques to train models to detect trusted connections and network traffic and flag any anomalies.

[0006] While traditional machine learning algorithms are fast and effective in detecting labeled attacks in certain scenarios, their effectiveness declines significantly when new or zero-day attacks emerge, leading to numerous false positives. Furthermore, traditional machine learning methods often require manual feature selection and extensive domain expertise, resulting in relatively shallow learning methods. Furthermore, while supervised learning models can distinguish existing known attacks, they are insufficient to detect the constantly emerging zero-day attacks, particularly in dynamic and heterogeneous IoT-based systems.

[0007] To address these limitations, deep learning methods have demonstrated promising results in a wide range of NIDS (Network Intrusion Detection System) research and applications. These methods include convolutional neural networks, recurrent neural networks, reinforcement learning, and many other variants. Compared to traditional signature-based network intrusion prevention methods, deep learning-based intrusion detection methods offer higher accuracy, lower false positive rates, and greater versatility. Furthermore, compared to machine learning-based network intrusion prevention systems, their flexible deep architecture and end-to-end learning process enable them to capture more complex attack patterns while eliminating labor-intensive and tedious feature engineering. Deep learning techniques learn by creating abstract representations of data. This enables the model to better automatically extract features and produce more accurate results.

[0008] The Transformer is a model based on a multi-head attention mechanism. Originally developed for machine translation, it has surpassed RNN and CNN models in machine translation performance. Using only an encoder-decoder and attention mechanism, the Transformer achieves excellent results. Furthermore, it avoids the slow training associated with iterative training in RNNs and GNNs, instead employing a self-attention mechanism for rapid parallelization. The core of the self-attention mechanism is to capture correlations between vectors. Compared to RNNs, which iteratively acquire correlation information from time series data, self-attention can be implemented through parallel matrix operations.

[0009] Transformer architectures are very powerful for NLP tasks because they are able to capture long-range dependencies and relationships between different elements of a sequence without requiring domain-specific prior knowledge or feature engineering. Although originally designed for NLP, the Transformer architecture has proven to be a general and powerful tool for capturing complex patterns and relationships in various types of sequence data, including but not limited to images, graphics, and speech. This adaptability makes Transformer-based models particularly well-suited for use in intrusion detection systems (NIDS), where data is captured as sequences of packets or flows.

[0010] GNNs are a relatively new subfield of deep neural networks, uniquely capable of leveraging the inherent structure of graph-based data. GNN research is closely related to graph embedding, or network embedding. Network coding aims to represent network nodes as low-dimensional vector representations to maintain network topology and node content information, and facilitate subsequent image and data analysis tasks such as classification and clustering. Meanwhile, GNNs are deep learning models designed to solve graph-related tasks in an end-to-end manner. Training and evaluation data for intrusion detection systems (NIDS) are typically represented as flow records, which can naturally be represented in a graph format. This has established the potential and motivation for exploring GNNs for network intrusion detection. Summary of the Invention

[0011] In order to solve the problem of simultaneously learning the statistical characteristics, temporal characteristics, and topological characteristics of traffic to utilize network space characteristics for IoT intrusion detection, in a first aspect, according to some embodiments of the present application, the IoT intrusion detection method based on the Transformer and GNN hybrid model includes:

[0012] The network traffic data in the Internet of Things is position-encoded and input into the encoder of the Transformer model to output the first embedding vector; wherein, the first embedding vector includes the time slice t0 to the time slice t n The network flows between any host pair within a time period, wherein any of the network flows is associated with the intrusion information of other network flows in the host pair in which it is located;

[0013] Set the time slice t0 to t n During the time slice, each host pair i The network flow is converted into a first graph, where the host is a node of the first graph and the network flow is an edge of the first graph;

[0014] Converting the first graph into a second graph using a Line Graph model, where network flows are nodes of the second graph and relationships between pairs of hosts are edges of the second graph;

[0015] Input the second graph into the GNN model and output the second embedding vector;

[0016] The first embedding embedding vector and the second embedding embedding vector are spliced ​​to obtain a splicing vector, the splicing vector is input into a classifier, and the classifier outputs an Internet of Things intrusion detection result.

[0017] According to the IoT intrusion detection method based on the Transformer and GNN hybrid model in some embodiments of the present application, the network traffic data in the IoT is position-encoded and input into the encoder of the Transformer model to generate a first embedding vector, including

[0018] The network traffic data in the Internet of Things is position-encoded and input into the encoder of the Transformer model, wherein the encoder includes an attention mechanism and a feedforward network, and the attention mechanism includes a mask;

[0019] The position-encoded data is input into the attention mechanism of MASK, and the output of the attention mechanism of MASK is residually connected and normalized with the position-encoded data;

[0020] The data of the residual connection and standardization operation is input into a feedforward network, and the output of the feedforward network is subjected to a residual connection and standardization operation on the data of the residual connection and standardization operation, and the first embedding embedding vector is output.

[0021] According to the IoT intrusion detection method based on the Transformer and GNN hybrid model in some embodiments of the present application, the number of encoder layers is 2, the number of attention mechanism heads is 2, and the activation function is relu;

[0022] The specific formula for position encoding of network traffic data in the Internet of Things is as follows:

[0023]

[0024] in, is the time series data of the input network traffic, Encode the location of the current data;

[0025] in, Depend on and get:

[0026]

[0027]

[0028] The input size of the encoder is , Indicates the time position of the current data, Indicates the dimension position of the current data, Indicates the position code of the even-numbered bits, Indicates the position code of an odd number of bits.

[0029] According to the IoT intrusion detection method based on the Transformer and GNN hybrid model in some embodiments of the present application, the first graph is converted into the second graph through the Line Graph model, including

[0030] Generate an adjacency matrix for each of the first graphs , represents the topological relationship in the graph. If there is an edge between two nodes in the graph, the corresponding element in the matrix is ​​1, otherwise it is 0;

[0031] For the adjacency matrix Perform Laplace normalization and convert the adjacency matrix Add the self-loop to get , and calculate the corresponding diagonal matrix , then construct the normalized matrix ,in, represents the identity matrix, represents the adjacency matrix with self-connection, represents the degree matrix, which is a diagonal matrix whose diagonal elements represent the degree of each node;

[0032] The GCNII layer introduces the initial residual connection and identity mapping to provide the signal of the first graph. Construct a K-order polynomial filter with arbitrary coefficients;

[0033] Line Graph model for the first graph Each edge in the second graph Create a node in the first graph; Any two edges that share the same host node in the second graph Create an edge for the corresponding node in and get the second graph.

[0034] On the second aspect, according to some embodiments of the present application, a Transformer and GNN hybrid model for performing the IoT intrusion detection method includes:

[0035] The encoder of the Transformer model generates a first embedding vector from the position-encoded network traffic data in the Internet of Things; wherein the first embedding vector includes the time interval from time slice t0 to time slice t n The network flows between any host pair within a time period, wherein any of the network flows is associated with the intrusion information of other network flows in the host pair in which it is located;

[0036] Conversion layer, converting time slice t0 to t n During the time slice, each host pair i The network flow is converted into a first graph, where the host is a node of the first graph and the network flow is an edge of the first graph;

[0037] Line Graph model, converting the first graph into a second graph, with network flows as nodes and relationships between host pairs as edges;

[0038] GNN layer, inputs the second graph and generates a second embedding vector;

[0039] A concatenation layer concatenates the first embedding vector and the second embedding vector to obtain a concatenated vector.

[0040] The classifier inputs the concatenated vector and outputs an IoT intrusion detection result.

[0041] On the third side, according to the training method of the Transformer and GNN hybrid model for IoT intrusion detection in some embodiments of the present application, it includes:

[0042] Step 1: Preprocess the local network traffic data, including missing value processing, outlier processing and normalization processing;

[0043] Step 2: Determine the number of feature spaces involved in training and prediction ;

[0044] Step 3: Divide the preprocessed local data into a training set and a test set;

[0045] Step 4: Use the training set to train the Transformer and GNN hybrid model;

[0046] Step 5: Use the test set to test the error of the Transformer and GNN hybrid model;

[0047] Step 6: Repeat steps 4 to 5 until the error no longer changes;

[0048] Step 7: Save the model parameters with the smallest error in the test set as the final model parameters.

[0049] According to the training method of the Transformer and GNN hybrid model for IoT intrusion detection in some embodiments of the present application, the feature space includes protocol, flow duration, number of incoming bytes, number of bytes per packet, and TCP flags.

[0050] According to the training method of the Transformer and GNN hybrid model for IoT intrusion detection in some embodiments of the present application, 70% of the data is used as a training set and 30% of the data is used as a test set.

[0051] According to the training method of the Transformer and GNN hybrid model for IoT intrusion detection in some embodiments of the present application, the optimized objective function is the mean square error between the input value and the predicted value, the Nadam optimizer is selected and the learning rate is set to 0.01, and 100 epochs are trained. After each training is completed, the data set is used for testing.

[0052] In the fourth aspect, according to the electronic device in some embodiments of the present application, the electronic device includes: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the electronic device, enable the electronic device to perform any one of the methods.

[0053] Beneficial Effects: This invention combines transformers and GNNs, using graph neural networks to capture topological information and better model the interactions between topology and IP pairs. This allows the invention to simultaneously learn the statistical, temporal, and topological characteristics of traffic. By incorporating these previously overlooked network spatial features, it enables more efficient learning, analysis, and summary of traffic data characteristics.

[0054] Based on a Line Graph, this method swaps the edges and nodes of the original graph, allowing the GNN to use the original features of the stream data as initial embedding vectors, learn the topological information in the graph, capture the correlation between different traffic flows, and implement IoT intrusion detection by classifying nodes on the new graph structure. Compared to existing edge embedding calculations based on the embedding of two nodes connected by an edge, this method, which samples and aggregates edge information in a given graph, avoids the problem of GNN message aggregation being unable to utilize the original features of the stream data.

[0055] The line graph used in this paper contains a redundant spectrum, which helps reduce the number of medium and large eigenvalues ​​of the unnormalized Laplacian matrix and improves the numerical stability of the model during convolution. Furthermore, the high connection density of the line graph enables convolution operations performed on it to aggregate more neighboring flow information, while the presence of the redundant spectrum also enhances the numerical stability of the model during convolution.

[0056] Compared with the combination of RNN and GNN models, the combination of transformer and GNN models of the present invention can generate a first graph at each moment, thereby being able to quantitatively determine whether a specific network flow in a host pair contains attack information. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a training flowchart based on the Transformer and GNN hybrid model provided by the present invention.

[0058] Figure 2 This is the architecture diagram of the Transformer and GNN hybrid model of the present invention.

[0059] Figure 3 It is a flowchart of the GNN algorithm based on LineGraph of the present invention. DETAILED DESCRIPTION

[0060] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0061] To better model the aforementioned topological information and interactions between IP pairs, this method proposes an IoT intrusion detection approach based on a hybrid model of Transformers and GNNs. Graph neural networks are used to capture topological information. Combining Transformers and GNNs simultaneously learns the statistical, temporal, and topological characteristics of traffic. By incorporating these previously overlooked cyberspace features, it enables more effective learning, analysis, and summarization of traffic data.

[0062] The present invention proposes the following technical solutions to solve the above technical problems: a method for detecting intrusion in the Internet of Things based on Transformer and GNN, comprising the following steps:

[0063] Step 1: Preprocess the local network traffic data, including missing value processing, outlier processing, and normalization processing; and build a Transformer and GNN hybrid model.

[0064] Step 2: Determine the number of feature spaces involved in training and prediction .

[0065] Step 3: Divide the local data preprocessed in step 1 into a training set and a test set.

[0066] Step 4: Use the training set to train the Transformer and GNN hybrid model. In order to retain the original statistical characteristics of the stream data and increase the graph connection density, the graph data in the training process uses Line Graph conversion to extract graph features.

[0067] Step 5: Use the test set to test the error of the Transformer and GNN hybrid model.

[0068] Step 6: Repeat steps 4 and 5 until the error no longer changes.

[0069] Step 7: Save the model parameters with the smallest error in the test set as the final model parameters.

[0070] In step 1, the steps for constructing the Transformer and GNN hybrid model are as follows:

[0071] 1.1 Initial Data Representation. Five-tuple Representation of Network Traffic: Each network traffic record is initially represented using the default five-tuple representation, consisting of source IP address, source port, destination IP address, destination port, and transport protocol. Additionally, statistical attributes such as protocol type, traffic duration, and number of bytes transmitted are recorded. All network traffic is processed as a chronological record of communications between IP pairs. This serialized data format lays the foundation for subsequent time series analysis.

[0072] 1.2 Transformer Time Series Learning: Input the sequence data processed in step 1.1 into the Transformer encoder. Transformers excel at processing time series data and can effectively capture the sequence characteristics of abnormal traffic flows during interactions between IP pairs. After processing this time series data, the Transformer generates embedding vectors that represent the time series characteristics of each traffic data point. These embeddings will serve as the initial features for subsequent GNN nodes.

[0073] Because the goal of the Transformer and GNN hybrid model is to detect intrusion traffic in traffic data, in order to reduce the amount of model calculation, only the encoder layer in the Transformer model is constructed, but MASK is added to the attention mechanism. Set the encoder input size to , the number of encoder layers is 2, the number of attention mechanism heads is 2, and the activation function uses relu. The first step is to position encode the input traffic data. The specific formula is as follows:

[0074]

[0075]

[0076]

[0077] in, For the input traffic statistics, is the position code of the current data, which can be obtained by and get; Indicates the time position of the current traffic data, (Odd and even numbers) represent the dimensional position of the current traffic data. Second, the encoded data is fed into the attention mechanism, which incorporates a mask mechanism similar to the decoder in the Transformer. The purpose of adding the mask is to only consider the time series data to date. Third, the data output from the second step undergoes a residual connection and normalization operation. Fourth, the data output from the third step undergoes a residual connection and normalization operation again, passing it into the feedforward network (consisting only of the linear layer and activation function).

[0078] 1.3 Graph Data Generation. The traffic information between IP pairs within each time slice is represented as a graph structure, with IPs as nodes and traffic as edges, resulting in a graph data representation. Through line graph transformation, the edges and nodes of the original graph are swapped. That is, traffic (edges in the original graph) is converted to new nodes, while the original IP nodes become edges. This transforms the problem into node classification on the new graph structure.

[0079] 1.4 To efficiently generate edge embeddings for graph data (i.e., embeddings for stream data), many previous methods calculated edge embeddings by using the embeddings of the two nodes connected by an edge to sample and aggregate edge information in a given graph. However, this approach cannot utilize the original features of the stream data, such as packet count, IP address, port number, and protocol, when performing message aggregation using GNNs.

[0080] Inspired by the concept of line graph in graph theory, this paper performs equivalent mapping on the edges in graph Gf and maps it into Line Graph . Input the graph data converted from the Line Graph in step 1.3 into the GNN. At this point, the traffic data serves as nodes, and the relationships between IP pairs serve as edges. The GNN uses the raw features of the flow data as initial embedding vectors to learn the topological information in the graph and capture the correlations between different traffic flows.

[0081] With the original picture In comparison, Line Graph It also has some advantageous properties. First, The line graph has a higher edge density than the original graph, meaning that nodes with frequent communication in the network are more prominently represented in the line graph. Secondly, the line graph has a redundant spectrum, which helps reduce the number of medium and large eigenvalues ​​of the unnormalized Laplacian matrix, thereby improving the model's numerical stability during convolution. Furthermore, the high connection density of the line graph enables convolution operations performed on it to aggregate more neighboring flow information. The presence of the redundant spectrum also enhances the model's numerical stability during convolution.

[0082] 1.5 Input the node embeddings generated by the GNN in step 1.4 into the feedforward neural network and perform classification through the Softmax layer. Use the cross entropy loss function to train the entire model to optimize the classification accuracy.

[0083] In step 4, the specific steps of the Line Graph-based GNN algorithm are as follows:

[0084] 4.1 represents network traffic as a series of static graph snapshots captured at regular intervals during the time series of a bidirectional flow. The communication between each pair of IP addresses is recorded as a network flow, with the links between IP addresses treated as different "carrier frequencies" and the network flows sent between different IP addresses as unique "frames." Simultaneous bidirectional flows are sliced ​​to reconstruct different graph snapshots. Each graph snapshot consists of a set of nodes (IP addresses) and a set of edges (network flows). The edge features are based on network flow statistics, such as protocol, flow duration, number of incoming bytes, number of bytes per packet, and TCP flags.

[0085] 4.2 Adjacency Matrix Generation: Generate an adjacency matrix for each graph snapshot to represent the topological relationship in the graph. If there is an edge between two nodes in the graph, the corresponding element in the matrix is ​​1, otherwise it is 0.

[0086] 4.3 As the information capacity of each network snapshot becomes increasingly sparse, directly accumulating graph convolutional layers can lead to severe oversmoothing problems, making it difficult to distinguish different types of network behaviors and affecting the model's detection capabilities. Most existing graph convolution operations generate node embeddings, which cannot directly capture the edge features required for intrusion detection tasks.

[0087] 4.4 To address the above issues, this paper introduces a deeper GCNII layer. This architecture alleviates the overfitting problem by using two structures: residual connections and identity mapping. It also uses a line graph structure to extract spatial information from each network snapshot and effectively capture edge features.

[0088] 4.5 First, in order to process the image signal , perform Laplace normalization on the adjacency matrix of the graph. Add the self-loop to get , and calculate the corresponding diagonal matrix . Then construct the normalized matrix .

[0089] 4.6 By introducing the initial residual connection and identity mapping, GCNII can Construct a K-order polynomial filter with arbitrary coefficients. However, since the signal processed by GCNII It represents the node information of the graph, so it needs to be modified to support the intrusion detection task. In order to solve the problem of edge embedding generation, we refer to the Line Graph concept in graph theory and transform the original graph into The edge equivalent mapping in is Line Graph The specific operation is: Each network flow (edge) in Create a node in Any two edges that share the same host node in Create an edge for its corresponding node.

[0090] 4.7 Since the local regularity of intrusion traffic is not obvious in long sequences. In the actual network environment, the performance of various attacks does not have obvious regularity over a long period of time. Therefore, a sliding window mechanism is used to improve the stability of the model. Specifically, during the training process, each batch contains b network snapshots, from arrive ,deploy a sliding window that allows the next batch to share a certain number of overlapping snapshots with the previous batch.,Sliding window can maintain the regularity between time snapshots and improve the stability of the model over long time intervals.

[0091] Compared with the existing RNN model, the present invention uses the Transformer model, which can convert the t0 time slice to t n During the time slice, each host pair i The network flow is converted into the first graph, that is, the time slice t0 to t n Each time slice t of the time slice i The corresponding multiple first graphs. However, the RNN model can only obtain the time period within the end time t n A first diagram of a time slice. Furthermore, for the chronologically ordered network flows within a host pair, for RNN models, the current network flow can only be associated with the information of the previous network flow. However, for Transformer models, the current network flow can be associated with the information of all (the entire row) network flows between the host pair.

[0092] The existing RNN model and GNN model are parallel, while the Transformer model of the present invention and the GNN model are serial, that is, the output of the Transformer model of the present invention is converted from a first graph to a second graph, input into the GNN model, and the output of the Transformer model and the output of the GNN model are added together.

[0093] Based on the above two points, the existing hybrid model of RNN and GNN models can only be used at the termination time t n To achieve different hosts at the end time t n The network flow learning in (column 1) generates a first graph, and thus can only qualitatively determine whether the network flow of a host pair has attack information.

[0094] The hybrid model of the Transformer model and the GNN model in this invention is serial, at any time t iIt can realize that different hosts can i (a column) of network flow learning, and based on the Transformer model, the current network flow can be associated with the information of all (the entire row) network flows between the host pairs. Therefore, the hybrid model of the present invention can achieve at each time t i A first graph is generated, which can quantitatively determine whether a specific network flow in a host pair contains attack information. In a specific example, by combining Transformer and GNN, the model effectively captures the statistical, temporal and topological characteristics of traffic data. At the same time, Line Graph conversion further enhances the model's ability to learn complex network topology structures, enabling this method to more accurately and efficiently identify abnormal behaviors in real-time intrusion detection tasks. Figure 1 As shown in , the present invention provides a training flowchart based on a hybrid model of Transformer and GNN, which specifically includes the following steps:

[0095] In step 1, the intrusion detection model processes all network traffic into communication records between IP pairs arranged in chronological order, preprocesses the data, including missing value processing, outlier processing, and normalization; and constructs a hybrid model of Transformer and GNN.

[0096] In step 2, the intrusion detection model determines the number of feature spaces involved in training and prediction, including protocol, flow duration, number of incoming bytes, number of bytes per packet, and TCP flags.

[0097] In step three, the intrusion detection model divides the local data preprocessed in step one into 70% of the data as a training set and 30% of the data as a test set.

[0098] Step 4: The intrusion detection model uses the training set to train the Transformer and GNN hybrid model. In order to solve the edge embedding generation problem, the Line Graph concept in graph theory is used to map the edges in the original graph to Line Graph. The optimization objective function is the mean square error (MSE) between the input value and the predicted value. The Nadam optimizer is selected and the learning rate is set to 0.01. The training is repeated for 100 epochs. After each training, the dataset is used for testing. Step 5: Intrusion Detection Use the test set to test the error of the Transformer and GNN hybrid model.

[0099] Step 5: The intrusion detection model uses the test set to test the error of the Transformer and GNN hybrid model.

[0100] Step 6: Repeat steps 4 and 5 until the error no longer changes.

[0101] Step 7: The intrusion detection model saves the model parameters with the smallest error in the test set as the final model parameters. Figure 2 The following is the architecture diagram of the Transformer and GNN hybrid model in step 1. The steps for building the Transformer and GNN hybrid model are as follows:

[0102] 1.1 Construct the basic GCNII layer. This architecture uses residual connections and identity mapping to alleviate overfitting. The dimensions of the hidden layer are set to the number of feature spaces in step 2. The number of stacked layers in the GNN (num_layers) is set to 2.

[0103] 1.2 In order to meet the randomness of traffic data, the output of the last layer of the running result of step 1.1 is mapped through the linear layer, and the dimension is Convert to × The dimension of each two values ​​is a group, which is used as the mean and variance of the normal distribution respectively. Each group randomly obtains a value according to the normal distribution, and there are Dimensional output.

[0104] 1.3 Because the goal of the Transformer and GNN hybrid model is to perform fixed-period power prediction on traffic data, to reduce the computational complexity of the model, only the encoder layer of the Transformer model is constructed, but a mask is added to the attention mechanism. The encoder input size is set to 4, the number of encoder layers is 2, the number of attention mechanism heads is 2, the activation function is ReLU, and all other default values ​​are used. The first step is to positionally encode the input traffic data. The specific formula is as follows:

[0105]

[0106]

[0107]

[0108] in, is the input flow data, is the position code of the current data, which can be obtained by and get; Indicates the time position of the current traffic data, (Odd and even numbers) represent the dimensional position of the current traffic data. Second, the encoded data is fed into the attention mechanism, which incorporates a mask mechanism similar to the decoder in the Transformer. The purpose of adding the mask is to only consider the time series data to date. Third, the data output from the second step undergoes a residual connection and normalization operation. Fourth, the data output from the third step undergoes a residual connection and normalization operation again, passing it into the feedforward network (consisting only of the linear layer and activation function).

[0109] 1.4 Intrusion Detection Input the past into the Transformer model in step 1.3 Time series data at the moment, input the past into the GNN model in step 1.2 Time series data at the moment.

[0110] 1.5 Weight the output results of the two models in step 1.4 as the final output result.

[0111] 1.6 Add activation function layer, activation function layer and the fully connected layer at the end to ensure efficient training and correct output of the neural network.

[0112] Figure 3 Create a timing diagram for the application of the Transformer and GNN hybrid model. The specific steps are as follows:

[0113] (1) Intrusion detection obtains data from the application and preprocesses it.

[0114] (2) Intrusion detection determines the feature space to be processed.

[0115] (3) Create a hybrid model of Transformer and GNN based on intrusion detection, and randomly initialize the parameters.

[0116] (4) Intrusion detection uses a local calculator to train the hybrid model established in step (3) until the error on the test set is less than the threshold, that is, the training is completed.

[0117] (5) Save the model parameters with the smallest error in the test set as the final model parameters.

[0118] (6) Deploy the trained Transformer and GNN hybrid model to the application.

[0119] This paper identifies intrusions by monitoring network traffic in real time. By targeting the topological and temporal correlations of intrusion detection model data, a hybrid Transformer and GNN model is designed. This hybrid model can efficiently train models with excellent accuracy, providing theoretical basis and practical experience for the application of deep learning technology in the field of intrusion detection models.

[0120] Experimental Example: Table 1 presents experimental comparison data for our intrusion detection method and other cutting-edge algorithms. Table 2 shows the comparison of six public datasets used, further detailing these datasets, including the number of network flows and protocols, the number of ports, the number of IP addresses associated with the network topology, and the attack types. In a binary classification experiment, we compared our Transformer and GNN fusion algorithm with the following benchmark network intrusion detection system (NIDS) methods:

[0121] 1) Supervised models: E-GraphSAGE, Extra Tree, Random Forest (RF), Deep Feedforward Neural Network (DNN);

[0122] 2) Semi-supervised model: Transductive Support Vector Machine (TSVM).

[0123] E-GraphSAGE is a NIDS method based on graph neural networks (GNNs) that captures edge features and topological information, while TSVM is a semi-supervised large-margin classification method that maximizes the margin between labeled and unlabeled data through regularization. In the TSVM training set, we used the same 30% labeling ratio as our proposed method. Table 1 shows the comparative results, including accuracy, recall, precision, F1 score, and false positive rate (FAR).

[0124]

[0125]

[0126] As can be seen in Table 1, our method performs well on all six datasets. On the NF-BoT-IoT, NF-CSE-CIC-IDS2018, and NF-CSE-CIC-IDS2018-V2 datasets, our method achieves the best F1 score and accuracy, outperforming other methods. Furthermore, on the NF-ToN-IoT, NF-ToN-IoT-V2, and NF-BoT-IoT-V2 datasets, our model achieves F1 scores of 0.9855, 0.9887, and 0.9989, respectively, very close to the performance of other state-of-the-art methods. Furthermore, when our method is retested on the NF-CSE-CIC-IDS2018-V2 dataset using the same proportion of labeled samples as our method, the accuracy of the RF and DNN models decreases from 99.47% to 91.55% and from 99.21% to 92.45%, respectively. In this case, the performance of these two methods is lower than our 99.80% accuracy. Therefore, these results show that our method has superior binary classification ability and is able to distinguish between normal and malicious network traffic with fewer labeled samples.

[0127] Based on the above embodiments, an embodiment of the present application further provides an electronic device, which enables the electronic device to execute the method provided in the above embodiments when the computer program is run on the electronic device.

[0128] Based on the above embodiments, an embodiment of the present application further provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above embodiments.

[0129] The storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0130] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. An IoT intrusion detection method based on a hybrid model of Transformer and GNN, characterized in that: include: The network traffic data in the Internet of Things is position-encoded and input into the encoder of the Transformer model to output the first embedding vector; wherein, the first embedding vector includes the time slice t0 to the time slice t n The network flows between any host pair within a time period, wherein any of the network flows is associated with the intrusion information of other network flows in the host pair in which it is located; Set the time slice t0 to t n During the time slice, each host pair i The network flow is converted into a first graph, where the host is a node of the first graph and the network flow is an edge of the first graph; Converting the first graph into a second graph using a Line Graph model, where network flows are nodes of the second graph and relationships between pairs of hosts are edges of the second graph; Input the second graph into the GNN model and output the second embedding vector; The first embedding embedding vector and the second embedding embedding vector are spliced ​​to obtain a splicing vector, the splicing vector is input into a classifier, and the classifier outputs an Internet of Things intrusion detection result.

2. The IoT intrusion detection method based on the Transformer and GNN hybrid model according to claim 1 is characterized in that: in, The network traffic data in the Internet of Things is positionally encoded and input into the encoder of the Transformer model to generate the first embedding vector, including The network traffic data in the Internet of Things is position-encoded and input into the encoder of the Transformer model, wherein the encoder includes an attention mechanism and a feedforward network, and the attention mechanism includes a mask; The position-encoded data is input into the attention mechanism of MASK, and the output of the attention mechanism of MASK is residually connected and normalized with the position-encoded data; The data of the residual connection and standardization operation is input into a feedforward network, and the output of the feedforward network is subjected to a residual connection and standardization operation on the data of the residual connection and standardization operation, and the first embedding embedding vector is output.

3. The IoT intrusion detection method based on the Transformer and GNN hybrid model according to claim 2 is characterized in that: The number of encoder layers is 2, the number of attention mechanism heads is 2, and the activation function is relu; The specific formula for position encoding of network traffic data in the Internet of Things is as follows: in, is the time series data of the input network traffic, Encode the location of the current data; in, Depend on and get: The input size of the encoder is , Indicates the time position of the current data, Indicates the dimension position of the current data, Indicates the position code of the even bit, Indicates the position code of an odd number of bits.

4. The IoT intrusion detection method based on the Transformer and GNN hybrid model according to claim 1 is characterized in that: in, The first graph is converted into a second graph through the Line Graph model, including Generate an adjacency matrix for each of the first graphs , represents the topological relationship in the graph. If there is an edge between two nodes in the graph, the corresponding element in the matrix is ​​1, otherwise it is 0; For the adjacency matrix Perform Laplace normalization and convert the adjacency matrix Add the self-loop to get , and calculate the corresponding diagonal matrix , then construct the normalized matrix ,in, represents the identity matrix, represents the adjacency matrix with self-connection, represents the degree matrix, which is a diagonal matrix whose diagonal elements represent the degree of each node; The GCNII layer introduces the initial residual connection and identity mapping to provide the signal of the first graph. Construct a K-order polynomial filter with arbitrary coefficients; Line Graph model for the first graph Each edge in the second graph Create a node in the first graph; Any two edges that share the same host node in the second graph Create an edge for the corresponding node in and get the second graph.

5. A Transformer and GNN hybrid model for IoT intrusion detection for executing the method of claim 1, characterized in that: include The encoder of the Transformer model generates a first embedding vector from the position-encoded network traffic data in the Internet of Things; wherein the first embedding vector includes the time interval from time slice t0 to time slice t n The network flows between any host pair within a time period, wherein any of the network flows is associated with the intrusion information of other network flows in the host pair in which it is located; Conversion layer, converting time slice t0 to t n During the time slice, each host pair i The network flow is converted into a first graph, where the host is a node of the first graph and the network flow is an edge of the first graph; Line Graph model, converting the first graph into a second graph, with network flows as nodes and relationships between host pairs as edges; GNN layer, inputs the second graph and generates a second embedding vector; A concatenation layer concatenates the first embedding vector and the second embedding vector to obtain a concatenated vector. The classifier inputs the concatenated vector and outputs an IoT intrusion detection result.

6. A training method for a hybrid Transformer and GNN model for IoT intrusion detection according to claim 5, characterized in that: include Step 1: Preprocess the local network traffic data, including missing value processing, outlier processing and normalization processing; Step 2: Determine the number of feature spaces involved in training and prediction ; Step 3: Divide the preprocessed local data into a training set and a test set; Step 4: Use the training set to train the Transformer and GNN hybrid model; Step 5: Use the test set to test the error of the Transformer and GNN hybrid model; Step 6: Repeat steps 4 to 5 until the error no longer changes; Step 7: Save the model parameters with the smallest error in the test set as the final model parameters.

7. The training method according to claim 6, characterized in that: The feature space includes protocol, flow duration, incoming bytes, bytes per packet, and TCP flags.

8. The training method according to claim 6, characterized in that: 70% of the data is used as a training set and 30% of the data is used as a test set.

9. The training method according to claim 6, characterized in that: in, The optimization objective function is the mean square error between the input value and the predicted value. The Nadam optimizer is selected and the learning rate is set to 0.

01. The training is repeated for 100 epochs, and the dataset is used for testing after each training.

10. An electronic device, comprising: One or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method described in any one of claims 1-4, 7-9.

Images