Adaptive network intrusion detection and response system based on deep learning

Abstract

The invention relates to an adaptive network intrusion detection and response system based on deep learning, and belongs to the technical field of network intrusion detection. The system comprises a data acquisition module, a data preprocessing module, a feature extraction module, an intrusion detection module and an intrusion response module. The data acquisition module uploads the network flow data in the storage temporary file to the data preprocessing module through libcurl; the data preprocessing module obtains network flow preprocessing data through a data preprocessing model according to the network flow data; a feature extraction module performs feature extraction through an edge residual attention dynamic graph neural network model according to the network traffic preprocessing data to obtain network traffic spatial-temporal features; the intrusion detection module obtains an intrusion detection result through a perceptron model according to the network traffic spatial-temporal characteristics; and the intrusion response module performs intrusion response through a response model according to the intrusion detection result. And adaptive network intrusion detection and response based on deep learning are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of network intrusion detection, and in particular relates to an adaptive network intrusion detection and response system based on deep learning. Background Art

[0002] The emergence of the Industrial Internet of Things has brought new opportunities for global development. At the same time, information security issues have become increasingly prominent. As an effective network security protection measure, intrusion detection technology has become a research hotspot in the field of Industrial Internet of Things security. In the increasingly complex Industrial Internet of Things environment, traditional network intrusion detection models face the problems of poor detection performance, traffic data leakage, and difficulty in coping with complex network environments due to the huge and complex network traffic data. Therefore, efficient and reliable network intrusion detection systems play a vital role in ensuring network security.

[0003] With the development of big data and artificial intelligence technology, machine learning technology has been used to build effective network intrusion detection systems. Traditional machine learning methods usually require manual extraction of traffic features based on expert experience, accompanied by a tedious feature selection process, resulting in unstable feature extraction quality and affecting the performance of the detection system. Network intrusion detection systems based on deep learning technology include convolutional neural networks, recurrent neural networks, reinforcement learning and their many variants, but they mainly focus on the statistical characteristics of network traffic, fail to fully model complex network topology structures and IP communication interactions that evolve over time, and fail to fully capture the spatiotemporal characteristics of network traffic, resulting in reduced detection accuracy. Therefore, how to fully consider the spatiotemporal characteristics of network traffic and the complex network environment to improve the accuracy of network intrusion detection and response is a problem that needs to be solved at present. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an adaptive network intrusion detection and response system based on deep learning. The purpose of the present invention can be achieved by the following technical solutions: An adaptive network intrusion detection and response system based on deep learning, including a data acquisition module, a data preprocessing module, a feature extraction module, an intrusion detection module, and an intrusion response module; The data acquisition module is used to sniff network traffic through libpcap and store the network traffic to obtain a temporary storage file, and upload the network traffic data in the temporary storage file to the data preprocessing module through libcurl; The data preprocessing module is used to obtain network traffic preprocessing data through a data preprocessing model according to the network traffic data; The feature extraction module is used to extract features from the network traffic preprocessing data through an edge residual attention dynamic graph neural network model to obtain network traffic spatiotemporal features; The intrusion detection module is used to obtain intrusion detection results through a sensor model according to the spatiotemporal characteristics of the network traffic; The intrusion response module is used to perform intrusion response through a response model according to the intrusion detection result.

[0005] Preferably, obtaining network traffic preprocessing data through a data preprocessing model according to the network traffic data includes: Obtain data details through the Pandas package, and remove null values ​​and duplicate values ​​from the network traffic data according to the data details to obtain network traffic cleansing data; The network traffic preprocessing data is obtained by normalizing the network traffic cleaning data.

[0006] Preferably, the extracting features from the network traffic preprocessing data through an edge residual attention dynamic graph neural network model to obtain the network traffic spatiotemporal features includes: A network traffic dynamic spatiotemporal graph is constructed based on the network traffic preprocessing data by using a discrete time dynamic graph method; According to the network traffic dynamic spatiotemporal graph, the network traffic spatial features are obtained through an edge residual attention mechanism graph neural network; The network traffic spatiotemporal characteristics are obtained through a bidirectional gated cyclic unit according to the network traffic dynamic spatiotemporal graph and the network traffic spatial characteristics.

[0007] Preferably, the network traffic preprocessing data includes communication time, output IP address, output port, and network flow statistics.

[0008] Preferably, the network traffic dynamic spatiotemporal graph is constructed by discrete time dynamic graph method according to the network traffic preprocessing data, and comprises: Using the output IP address and the output port as dynamic graph nodes, and using the network flow statistics information as dynamic graph edges; A dynamic graph is constructed by constructing the dynamic graph edges and the dynamic graph nodes; Obtaining a dynamic graph snapshot by slicing the dynamic graph according to the communication time; The network traffic dynamic spatiotemporal graph is obtained according to the dynamic graph snapshot through a placeholder insertion method.

[0009] Preferably, the placeholder insertion method includes obtaining empty edges of a dynamic graph snapshot, presetting placeholder tags, obtaining the dynamic space-time graph by inserting placeholder edges into the empty edges of the dynamic graph snapshot, and obtaining the network traffic dynamic space-time graph by marking the dynamic space-time graph with the placeholder tags.

[0010] Preferably, obtaining the network traffic spatial features through an edge residual attention mechanism graph neural network according to the network traffic dynamic spatiotemporal graph includes: Obtaining node features of a network traffic dynamic space-time graph, edge features of a network traffic dynamic space-time graph, and a weight matrix, and obtaining a series connection sequence through a series connection operation according to the node features of the network traffic dynamic space-time graph and the edge features of the network traffic dynamic space-time graph; An attention weight is calculated by an attention mechanism according to the series sequence; A normalized attention coefficient is calculated by a softmax function according to the attention weight; Obtaining aggregate information by calculating the normalized attention coefficient and the weight matrix through an activation function; The network traffic spatial feature is obtained by feature updating calculation according to the aggregation information and the network traffic dynamic spatiotemporal graph edge feature.

[0011] Preferably, the obtaining the network traffic spatiotemporal features through a bidirectional gated cyclic unit according to the network traffic dynamic spatiotemporal graph and the network traffic spatial features comprises: A forward hidden feature is calculated by a forward gated recurrent unit according to the network traffic spatial feature; Obtaining inverse hidden features by calculating the inverse gated recurrent unit according to the network traffic spatial features; The network traffic spatiotemporal features are obtained by concatenating the forward hidden features and the reverse hidden features.

[0012] Preferably, performing an intrusion response through a response model according to the intrusion detection result includes: Obtaining a Snort response mode as a rule tree, presetting an add rule action, and adding the add rule action to the rule tree by adding a ListHead node to obtain a completed rule tree; An intrusion response is performed through the completion rule tree according to the intrusion detection result.

[0013] Preferably, the adding rule action includes adding a Drop action and adding a Reject action; the adding a Drop action includes discarding the intrusion data packet and giving an alarm message; the adding a Reject action includes discarding the data packet and sending a data packet to the source address to terminate the current session.

[0014] The beneficial effects of the present invention are: (1) Network traffic data is cleaned by Pandas to remove null values ​​and duplicate values. At the same time, data consistency is enhanced through normalization, which can improve the accuracy and stability of subsequent feature extraction and intrusion detection.

[0015] (2) The spatiotemporal characteristics of network traffic are extracted through the edge residual attention dynamic graph neural network model, which can capture the dynamic relationships and complex dependencies in the traffic and enhance the intrusion detection model's ability to identify abnormal behaviors. At the same time, the spatiotemporal characteristics are detected in combination with the perceptron model, which can quickly locate intrusion behaviors and improve the accuracy and response speed of intrusion detection.

[0016] (3) By integrating the attention mechanism in the edge residual attention dynamic graph neural network model into the edge, the edge data is regarded as the key factor, and the node is regarded as the transmission interface of the adjacent edge data, thus avoiding the increase of computational cost by aggregating node features; at the same time, adding residual connections in the edge update calculation can better adapt to the complex network structure in network intrusion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0018] Figure 1 It is a flowchart of a deep learning-based adaptive network intrusion detection and response system of the present invention. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0020] See also Figure 1 , an adaptive network intrusion detection and response system based on deep learning, including a data acquisition module, a data preprocessing module, a feature extraction module, an intrusion detection module, and an intrusion response module; The data acquisition module is used to sniff network traffic through libpcap and store the network traffic to obtain a temporary storage file, and upload the network traffic data in the temporary storage file to the data preprocessing module through libcurl; The data preprocessing module is used to obtain network traffic preprocessing data through a data preprocessing model according to the network traffic data; The feature extraction module is used to extract features from the network traffic preprocessing data through an edge residual attention dynamic graph neural network model to obtain network traffic spatiotemporal features; The intrusion detection module is used to obtain intrusion detection results through a sensor model according to the spatiotemporal characteristics of the network traffic; The intrusion response module is used to perform intrusion response through a response model according to the intrusion detection result.

[0021] Specifically, the libpcap and libcurl are open source libraries, the libpcap is used to capture network data packets, and the libcurl is used to transmit data; the sniffing refers to the process of extracting useful information by capturing and analyzing data packets transmitted on the network, which relies on the libpcap implementation.

[0022] In this embodiment, the adaptive network intrusion detection and response system based on deep learning includes a client and a server. The client includes a monitoring module and a sniffing module, which are used to collect network traffic data; the monitoring module includes monitoring and reading data in the Zookeeper database, and judging whether the Zookeeper database is updated. If yes, the sniffing module is started; if not, the sniffing module is in a blocked state. The sniffing module sniffs the network traffic through the libpcap and dumps the network traffic to a temporary file; when collecting the quota network traffic data, the network traffic data is transmitted to the server through the libcurl, and data preprocessing, intrusion detection and response are performed on the server.

[0023] Specifically, the data preprocessing model includes: S201: obtaining detailed data information through a Pandas package, and removing null values ​​and duplicate values ​​from the network traffic data according to the detailed data information to obtain network traffic cleansing data, wherein the detailed data information is detailed information of the network traffic data set; S202: Obtain the network traffic preprocessing data through normalization processing according to the network traffic cleaning data.

[0024] Specifically, the network traffic spatiotemporal features are obtained by extracting features through an edge residual attention dynamic graph neural network model based on the network traffic preprocessing data, including: S301: constructing a network traffic dynamic spatiotemporal graph according to the network traffic preprocessing data by using a discrete time dynamic graph method; S302: Obtaining network traffic spatial features through an edge residual attention mechanism graph neural network according to the network traffic dynamic spatiotemporal graph; S303: Obtain the network traffic spatiotemporal characteristics through a bidirectional gated recurrent unit according to the network traffic dynamic spatiotemporal graph and the network traffic spatial characteristics.

[0025] Specifically, the discrete-time dynamic graph method includes: S301-1: The network traffic preprocessing data includes communication time, output IP address, output port, network flow statistics information, and the network flow statistics information includes transmission protocol, flow duration, incoming flow bytes, and first packet TCP flag; S301-2: using the output IP address and the output port as dynamic graph nodes, and using the network flow statistics information as dynamic graph edges, wherein the dynamic graph edges include dynamic graph green edges and dynamic graph red edges, wherein the dynamic graph green edges indicate that the network flow is normal, and the dynamic graph red edges indicate that the network flow is abnormal; S301-3: constructing a dynamic graph by using the dynamic graph edges and the dynamic graph nodes; S301-4: obtaining a snapshot of the dynamic graph by slicing the dynamic graph according to the communication time; S301-5: Obtain the network traffic dynamic spatiotemporal graph through a placeholder insertion method according to the dynamic graph snapshot.

[0026] Specifically, the placeholder insertion method includes obtaining empty edges of a dynamic graph snapshot, presetting a placeholder label, obtaining the dynamic space-time graph by inserting placeholder edges into the empty edges of the dynamic graph snapshot, wherein the feature of the placeholder edge is 0, and marking the dynamic space-time graph with the placeholder label to obtain the network traffic dynamic space-time graph.

[0027] Specifically, the edge residual attention mechanism graph neural network includes: S302-1: Obtain node features of a dynamic spatiotemporal graph of network traffic, edge features of a dynamic spatiotemporal graph of network traffic, and a weight matrix, and obtain a series connection sequence through a series connection operation according to the node features of the dynamic spatiotemporal graph of network traffic and the edge features of the dynamic spatiotemporal graph of network traffic; The series operation expression is: , in, Z connection represents the concatenated sequence, W k represents the weight matrix of the current layer k, represents the previous layer embedding of node v, represents the upper embedding of the edge between node v and node u, represents the previous layer embedding of node u, || represents the feature concatenation operation, k Indicates the current layer. k-1 Indicates the previous layer; S302-2: Calculating an attention weight through an attention mechanism according to the series sequence; The attention mechanism calculation expression is: , in, is the attention weight, y represents the attention mechanism, Z connection represents the concatenated sequence, k Indicates the current layer. u and v represents a node; S302-3: Calculate a normalized attention coefficient using a softmax function according to the attention weight; The softmax function calculation expression is: , in, is the normalized attention coefficient, softmax Indicates the calculation of the softmax function, k Indicates the current layer. u and v Represents a node, is the attention weight; S302-4: Obtaining aggregate information by calculating an activation function according to the normalized attention coefficient and the weight matrix; The activation function calculation expression is: , in, For the aggregate information, LeakyRelu is the activation function, k represents the current layer, u and v Represents a node, N v for Number of nodes, is the weight matrix, is the normalized attention coefficient, represents the previous layer embedding of node v, represents the upper embedding of the edge between node v and node u, represents the previous layer embedding of node u; S302-5: Obtain the network traffic spatial feature through feature update calculation according to the aggregation information and the network traffic dynamic spatiotemporal graph edge feature; The feature update calculation expression is: , in, z vu is the network traffic spatial feature, z v Represents the aggregation information of node v, z uRepresents the aggregation information of node v, CONCAT Indicates a connection operation. , e vu is the dynamic spatiotemporal graph edge feature of the network traffic, , k Indicates the current layer. u and v Represents a node; Specifically, the bidirectional gated recurrent unit comprises: S303-1: Calculate a forward hidden feature through a forward gated recurrent unit according to the network traffic spatial feature; The forward gated recurrent unit calculation expression is: , in, is the hidden feature at the positive time t, GRU represents the forward gated recurrent unit, is the hidden feature at the positive time t-1, is the network traffic spatial feature, u and v Represents a node; S303-2: Obtaining a reverse hidden feature by calculating the reverse gated recurrent unit according to the network traffic spatial feature; The calculation expression of the reverse gated recurrent unit is: , in, To hide the features at time t in reverse, GRU represents the forward gated recurrent unit, To hide the features at the reverse time t+1, is the network traffic spatial feature, u and v Represents a node; S303-3: Obtain the network traffic spatiotemporal features by concatenating the forward hidden features and the reverse hidden features.

[0028] Specifically, performing an intrusion response through a response model according to the intrusion detection result includes: S401: Acquire a Snort response mode as a rule tree, preset an add rule action, and add the add rule action to the rule tree by adding a ListHead node to obtain a completed rule tree; S402: Performing an intrusion response according to the intrusion detection result by completing the rule tree.

[0029] Specifically, the adding rule action includes adding a Drop action and adding a Reject action, which specifically include: The added Drop action includes discarding the intrusion data packet and giving an alarm message; The adding of the Reject action includes discarding the data packet and sending a data packet to the source address to terminate the current session.

[0030] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0031] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0032] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0033] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. An adaptive network intrusion detection and response system based on deep learning, characterized in that: It includes data acquisition module, data preprocessing module, feature extraction module, intrusion detection module and intrusion response module; The data acquisition module is used to sniff network traffic through libpcap and store the network traffic to obtain a temporary storage file, and upload the network traffic data in the temporary storage file to the data preprocessing module through libcurl; The data preprocessing module is used to obtain network traffic preprocessing data through a data preprocessing model according to the network traffic data; The feature extraction module is used to extract features from the network traffic preprocessing data through an edge residual attention dynamic graph neural network model to obtain network traffic spatiotemporal features; The intrusion detection module is used to obtain intrusion detection results through a sensor model according to the spatiotemporal characteristics of the network traffic; The intrusion response module is used to perform intrusion response through a response model according to the intrusion detection result.

2. The deep learning-based adaptive network intrusion detection and response system according to claim 1, characterized in that: The obtaining of network traffic preprocessing data by using a data preprocessing model according to the network traffic data comprises: Obtain data details through the Pandas package, and remove null values ​​and duplicate values ​​from the network traffic data according to the data details to obtain network traffic cleansing data; The network traffic preprocessing data is obtained by normalizing the network traffic cleaning data.

3. The deep learning-based adaptive network intrusion detection and response system according to claim 1, characterized in that: The network traffic spatiotemporal features are obtained by extracting features from the network traffic preprocessing data through an edge residual attention dynamic graph neural network model, including: A network traffic dynamic spatiotemporal graph is constructed based on the network traffic preprocessing data by using a discrete time dynamic graph method; According to the network traffic dynamic spatiotemporal graph, the network traffic spatial features are obtained through an edge residual attention mechanism graph neural network; The network traffic spatiotemporal characteristics are obtained through a bidirectional gated cyclic unit according to the network traffic dynamic spatiotemporal graph and the network traffic spatial characteristics.

4. The deep learning-based adaptive network intrusion detection and response system according to claim 3, characterized in that: The network traffic preprocessing data includes communication time, output IP address, output port, and network flow statistics.

5. The deep learning-based adaptive network intrusion detection and response system according to claim 4, characterized in that: The network traffic dynamic spatiotemporal graph is constructed by using a discrete time dynamic graph method according to the network traffic preprocessing data, and comprises: Using the output IP address and the output port as dynamic graph nodes, and using the network flow statistics information as dynamic graph edges; A dynamic graph is constructed by constructing the dynamic graph edges and the dynamic graph nodes; Obtaining a dynamic graph snapshot by slicing the dynamic graph according to the communication time; The network traffic dynamic spatiotemporal graph is obtained according to the dynamic graph snapshot through a placeholder insertion method.

6. The deep learning-based adaptive network intrusion detection and response system according to claim 5, characterized in that: The placeholder insertion method includes obtaining empty edges of a dynamic graph snapshot, presetting placeholder tags, inserting placeholder edges into the empty edges of the dynamic graph snapshot to obtain the dynamic spatiotemporal graph, and marking the dynamic spatiotemporal graph with the placeholder tags to obtain the network traffic dynamic spatiotemporal graph.

7. The deep learning-based adaptive network intrusion detection and response system according to claim 3 is characterized in that: The obtaining of network traffic spatial features through edge residual attention mechanism graph neural network according to the network traffic dynamic spatiotemporal graph includes: Obtaining node features of a network traffic dynamic space-time graph, edge features of a network traffic dynamic space-time graph, and a weight matrix, and obtaining a series connection sequence through a series connection operation according to the node features of the network traffic dynamic space-time graph and the edge features of the network traffic dynamic space-time graph; An attention weight is calculated by an attention mechanism according to the series sequence; A normalized attention coefficient is calculated by a softmax function according to the attention weight; Obtaining aggregate information by calculating the normalized attention coefficient and the weight matrix through an activation function; The network traffic spatial feature is obtained by feature updating calculation according to the aggregation information and the network traffic dynamic spatiotemporal graph edge feature.

8. The deep learning-based adaptive network intrusion detection and response system according to claim 3, characterized in that: The obtaining of the network traffic spatiotemporal features through a bidirectional gated loop unit according to the network traffic dynamic spatiotemporal graph and the network traffic spatial features includes: A forward hidden feature is calculated by a forward gated recurrent unit according to the network traffic spatial feature; Obtaining inverse hidden features by calculating the inverse gated recurrent unit according to the network traffic spatial features; The network traffic spatiotemporal features are obtained by concatenating the forward hidden features and the reverse hidden features.

9. The deep learning-based adaptive network intrusion detection and response system according to claim 1, characterized in that: The performing an intrusion response through a response model according to the intrusion detection result includes: Obtaining a Snort response mode as a rule tree, presetting an add rule action, and adding the add rule action to the rule tree by adding a ListHead node to obtain a completed rule tree; An intrusion response is performed according to the intrusion detection result by completing the rule tree.

10. The deep learning-based adaptive network intrusion detection and response system according to claim 9, characterized in that: The adding rule action includes adding a Drop action and adding a Reject action; the adding a Drop action includes discarding the intrusion data packet and giving an alarm message; the adding a Reject action includes discarding the data packet and sending a data packet to the source address to terminate the current session.

Images