Intrusion detection system and method for edge device level lightweight pruning neural network

Abstract

The invention provides an intrusion detection system and method for an edge device level lightweight pruning neural network, the scheme provides a lightweight pruning strategy of a neural network, the neural network can be easily deployed on edge devices while the detection performance of the neural network is ensured by reducing the scale of a neural network model, and the detection efficiency of the neural network is improved. The computing pressure of the edge device is effectively reduced, so that the edge terminal device can autonomously detect the abnormal traffic, and the problems that the detection efficiency is low and the detection is easily influenced by a network communication environment because the detection can be carried out only by a cloud in a traditional scheme are effectively solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of network intrusion detection, and in particular to an intrusion detection system and method for edge device-level lightweight pruned neural networks. Background Art

[0002] The inventors found that most existing network intrusion detection systems use centralized cloud detection solutions, which use the powerful computing power of the cloud to discover network threats and block network attacks in a timely manner. However, the limitations of this method are that the communication between the cloud and the device is slow and highly dependent on a good network communication environment. In particular, in the case of a flood attack (DoS), the network communication conditions are very poor, making it difficult to meet the needs of high-real-time network intrusion detection.

[0003] Secondly, with the increasing demand for network intrusion detection tasks and the continuous development of deep learning technology, most of the existing neural network-based detection models have the characteristics of complexity, large number of parameters, and large models. These detection models can usually only be deployed in cloud computing centers, and are difficult to be deployed on edge devices with low power consumption and limited computing resources. Although there are some pruning strategies in existing methods to prune neural networks, these pruning strategies usually consider removing edges with smaller absolute weights from the network, or pruning edges with smaller weight changes in the neural network based on the impact of the back-propagation process on previous neurons. This type of weight pruning method will limit the performance of the neural network, making it easy for the neural network to learn polarized parameters during training, affecting the stability of neural network training, and may cause the gradient update to show exponential decay or exponential growth during the back-propagation process, resulting in significant degradation of the performance of the neural network and reducing the fitting and generalization capabilities of the neural network. Summary of the invention

[0004] The embodiment of the present invention provides an intrusion detection system and method for edge device-level lightweight pruned neural networks to solve the problem that traditional solutions can only be deployed in cloud computing centers for detection by the cloud, resulting in low detection efficiency and susceptibility to the influence of the network communication environment; and traditional neural network pruning strategies limit the performance of neural networks, making it easy for neural networks to learn polarization parameters during training, affecting the stability of neural network training, and may cause gradient updates to show exponential decay or exponential growth during backpropagation, resulting in significant degradation of neural network performance and reducing the fitting and generalization capabilities of neural networks.

[0005] According to a first aspect of an embodiment of the present invention, there is provided an intrusion detection system for edge device-level lightweight pruned neural network, including:

[0006] A neural network pre-training module is set in the cloud, which is used to pre-train a preset network abnormal traffic detection model based on a neural network, and to prune the pre-trained network abnormal traffic detection model based on a neural network to obtain a network abnormal traffic detection model based on a lightweight pruned neural network, and send it to the edge device;

[0007] Wherein, the pre-training is based on a pre-constructed training set, each sample in the training set includes a network traffic data feature vector and its corresponding classification result, and the network traffic data feature vector includes several traffic features; the pruning process is specifically as follows: calculating the importance score of each traffic feature in the network traffic data feature; based on the importance score of each traffic feature, calculating the feature pruning probability of the traffic feature; determining the traffic feature to be pruned based on the pruning probability of each traffic feature; at the same time, by calculating the pruning probability of each neuron in each hidden layer in the pre-trained neural network-based network abnormal traffic detection model, determining the neurons to be pruned in each hidden layer; by pruning the traffic features to be pruned and the neurons to be pruned, a network abnormal traffic detection model based on a lightweight pruned neural network is obtained;

[0008] as well as,

[0009] An edge-end pruned neural network intrusion detection module is set on the edge device, which is used to receive a network abnormal traffic detection model based on a lightweight pruned neural network from the cloud, and perform intrusion detection on the network traffic based on the network abnormal traffic detection model.

[0010] Furthermore, the importance score of the traffic feature is calculated using the following formula:

[0011]

[0012] in, Indicates the nth i Traffic characteristics Importance score, n v Represents the total number of features, n y Indicates the number of classification categories, Indicates calculation of the nth i Input features and nth j Input features The Spearman correlation coefficient between Indicates calculation of the nth i Input features With the kth label y k The Spearman correlation coefficient.

[0013] Furthermore, the calculation of the feature pruning probability of the traffic feature is specifically performed using the following formula:

[0014]

[0015] in, Indicates the nth i Traffic characteristics The feature pruning probability of .

[0016] Furthermore, the calculation of the pruning probability of the neuron is specifically performed using the following formula:

[0017]

[0018] in, represents the jth neuron h in the kth hidden layer of the neural network j The pruning probability of represents the number of neurons in the kth layer, M k-1 [p,q] represents the mask matrix M of the same size as the weight matrix of the pre-trained neural network-based network anomaly traffic detection model k The value of the element in the p-th row and q-th column.

[0019] Furthermore, the network abnormal traffic detection model based on neural network specifically includes an input layer, a hidden layer and an output layer, wherein the hidden layer uses a forward propagation mechanism to calculate each neuron and updates the neurons through a preset activation function; the output layer uses a Sigmoid function as a classifier to judge the abnormality of network traffic.

[0020] Furthermore, for the edge pruned neural network intrusion detection module, its intrusion detection process is specifically as follows: feature extraction is performed on the network traffic data to be detected to obtain a network traffic data feature vector; the network traffic data feature vector is used as the input of a network abnormal traffic detection model based on a lightweight pruned neural network to obtain an abnormal judgment result of the network traffic, thereby realizing edge detection of network intrusion.

[0021] Furthermore, the network traffic data feature vector includes but is not limited to a data type, an operation type, and an event timestamp.

[0022] According to a second aspect of an embodiment of the present invention, an intrusion detection method for an edge device-level lightweight pruned neural network is provided, which is based on the above-mentioned intrusion detection system for an edge device-level lightweight pruned neural network, including:

[0023] Collect and receive network traffic data in real time;

[0024] Extracting features from the network traffic data to obtain a network traffic data feature vector;

[0025] The network traffic data feature vector is used as the input of a network abnormal traffic detection model based on a lightweight pruned neural network to obtain an abnormal judgment result of the network traffic and realize edge detection of network intrusion.

[0026] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored and running on the memory, wherein when the processor executes the program, the intrusion detection method for an edge device-level lightweight pruned neural network is implemented.

[0027] According to a fourth aspect of an embodiment of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the intrusion detection method for an edge device-level lightweight pruned neural network is implemented.

[0028] One or more of the above technical solutions have the following beneficial effects:

[0029] (1) The present invention provides an intrusion detection system and method for a lightweight pruned neural network at the edge device level. The scheme provides a lightweight pruning strategy for a neural network. By reducing the scale of the neural network model, while ensuring the detection performance of the neural network, the neural network can be easily deployed on the edge device, effectively reducing the computing pressure of the edge device, thereby enabling the edge terminal device to autonomously detect abnormal traffic, effectively solving the problem that traditional schemes can usually only be detected by the cloud, resulting in low detection efficiency and being easily affected by the network communication environment.

[0030] (2) The lightweight pruning strategy in the solution described in the present invention determines the importance score of the input unit according to the correlation coefficient between the input unit and the reconstructed input in the lightweight pruning neural network, so as to eliminate the performance impact of unimportant features on the neural network detection performance, while retaining important features. While ensuring the neural network detection performance, it effectively reduces the scale of the neural network model and reduces the demand of the neural network model for device computing performance.

[0031] (3) The solution described in the present invention provides a feature pruning probability calculation strategy, which can keep the existing propagation path of the neural network unchanged with a higher pruning probability and remove neurons with similar contributions. This strategy can effectively reduce the size of the neural network. The pruned neural network can be deployed on terminal devices with limited computing power while ensuring the ability to detect abnormal traffic.

[0032] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 It is a schematic diagram of the overall process of an intrusion detection system for edge device-level lightweight pruned neural network described in an embodiment of the present invention;

[0035] Figure 2 The present invention is a schematic diagram of the construction process of the network abnormal traffic detection model based on the lightweight pruned neural network described in the embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0037] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0038] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0039] In one or more embodiments, Figure 1 As shown, this embodiment provides an intrusion detection system for edge device-level lightweight pruned neural network, which specifically includes a neural network pre-training module set in the cloud and an edge-end pruned neural network intrusion detection module set in the edge device, wherein:

[0040] A neural network pre-training module is set in the cloud, which is used to pre-train a preset network abnormal traffic detection model based on a neural network, and to prune the pre-trained network abnormal traffic detection model based on a neural network to obtain a network abnormal traffic detection model based on a lightweight pruned neural network, and send it to the edge device;

[0041] Wherein, the pre-training is based on a pre-constructed training set, each sample in the training set includes a network traffic data feature vector and its corresponding classification result, and the network traffic data feature vector includes several traffic features; the pruning process is specifically as follows: calculating the importance score of each traffic feature in the network traffic data feature; based on the importance score of each traffic feature, calculating the feature pruning probability of the traffic feature; determining the traffic feature to be pruned based on the pruning probability of each traffic feature; at the same time, by calculating the pruning probability of each neuron in each hidden layer in the pre-trained neural network-based network abnormal traffic detection model, determining the neurons to be pruned in each hidden layer; by pruning the traffic features to be pruned and the neurons to be pruned, a network abnormal traffic detection model based on a lightweight pruned neural network is obtained;

[0042] Specifically, the neural network pre-training module set in the cloud includes a lightweight feature extraction unit, a neural network pre-training unit and a lightweight neural network pruning training unit, wherein:

[0043] The lightweight feature extraction unit is used to extract lightweight network abnormal traffic features. Specifically, this module relies on the third-party network traffic capture tool Scapy to capture the communication data packets between IoT devices to form a data set X = {x1,…,x i ,…,x N}. After that, the network data packets are preprocessed to obtain representative features such as data type features, operation type features, event timestamp features, etc. that can describe the traffic status. The feature vector is represented by word vector technology as follows: Among them, v ni Indicates the nth i This lightweight feature extraction scheme can minimize computational overhead, making it suitable for real-time monitoring and analysis of large-scale networks, while effectively distinguishing benign traffic from abnormal traffic.

[0044] The neural network pre-training unit is used to pre-train a neural network-based network abnormal traffic detection model in the cloud. This module constructs a training set based on the lightweight features obtained in the previous step, and trains the neural network-based abnormal traffic detection model based on the constructed training set.

[0045] The network abnormal traffic detection model based on neural network in the neural network pre-training unit includes: an input layer, a hidden layer and a Sigmoid function output layer, wherein:

[0046] The input layer inputs each piece of data into the artificial neural network in turn. Each piece of data consists of a lightweight feature vector x i = Indicates that, n v is the total number of features;

[0047] The hidden layer uses the forward propagation mechanism to calculate each neuron. The process is expressed as: j =Wx i +b, where o j represents a hidden layer neural unit, W represents a weight matrix, b represents the offset, and uses z j =g(o j ) formula updates the neuron, where z j It represents the output result of the neural unit after the activation function is calculated. g() represents the ReLU activation function, which can alleviate the gradient disappearance problem to a limited extent.

[0048] The Sigmoid function output layer uses the Sigmoid function to output the final abnormal traffic classification result y k =Sigmoid(z j ), where k ranges from 0 to 1 in the binary classification case, indicating that there are 2 classification labels in total. When y1=0, it indicates benign traffic, and y1=1 indicates abnormal traffic;

[0049] The lightweight neural network pruning training unit includes an important feature score estimation subunit, a feature pruning probability calculation subunit and a lightweight pruning neural network construction subunit. Specifically:

[0050] The important feature score estimation subunit uses the Spearman method to calculate the correlation coefficient of two features: Among them, n is the number of levels, and d is the number of level differences between two columns of paired variables. Then, calculate the importance score of each feature: in, Indicates the nth i Features Importance score, n v Represents the total number of features, n y Indicates the number of classification categories, Indicates calculation of the nth i Input features and nth j Input features The Spearman correlation coefficient between Indicates calculation of the nth i Input features With the kth label y k The Spearman correlation coefficient.

[0051] The feature pruning probability calculation subunit calculates the probability of each feature v i Importance Rating Calculate its pruning probability: in, is the importance score of each feature calculated previously. After calculating the pruning probability of all features, we get the pruning probability sequence: Finally, the preset threshold is β, when When The value of is set to 0 to eliminate the influence of this feature on the lightweight pruned neural network. After repeating the operation on all features, the new feature vector is input into the lightweight pruned neural network.

[0052] The lightweight pruning neural network construction subunit constructs a mask matrix M = {M1, ..., M k ,…,M K}, the matrix is ​​a binary array used to selectively ignore or emphasize specific parts of the data, and improve the attention of the lightweight pruned neural network to key information. Based on the mask matrix, a new lightweight pruned neural network with K hidden layers of the same size as the pre-trained neural network can be constructed, where K is a fixed value determined by the initial network design. After that, a pruning probability is calculated for each neuron in the pre-trained neural network for pruning the neural network. The pruning probability of a neuron can be calculated as: in, represents the jth neuron h in the kth hidden layer of the neural network j The pruning probability of represents the number of neurons in the kth layer, M k-1 [p,q] represents the mask matrix M k-1 Since the mask matrix stores a binary array, the mask matrix M is calculated by this formula. k-1 Sum all elements in row p After that, normalization operation is performed It can be calculated that a neuron h j The pruning probability of . Among them, the mask matrix M k-1 The number of all elements in the pth row represents all the elements in the previous layer of the lightweight pruned neural network that are related to the neuron h j The number of connected neurons. Therefore, when a neuron is closely connected to the previous neural network layer, the neuron h j The pruning probability of can illustrate the importance of the neuron. Finally, the preset threshold is β. When The value of is 0 to eliminate the influence of the neuron on the lightweight pruned neural network. After the above processing, the pre-trained neural network can be simplified to obtain a new lightweight pruned neural network.

[0053] An edge-end pruned neural network intrusion detection module is set on the edge device, which is used to receive a network abnormal traffic detection model based on a lightweight pruned neural network from the cloud, and perform intrusion detection on the network traffic based on the network abnormal traffic detection model.

[0054] In the specific implementation, for the edge pruned neural network intrusion detection module, its intrusion detection process is specifically as follows: feature extraction is performed on the network traffic data to be detected to obtain a network traffic data feature vector; the network traffic data feature vector is used as the input of a network abnormal traffic detection model based on a lightweight pruned neural network to obtain an abnormal judgment result of the network traffic, thereby realizing edge detection of network intrusion.

[0055] Specifically, the edge-end pruning neural network intrusion detection module is used to continuously monitor and inspect network traffic, and includes: a lightweight feature extraction unit and a lightweight pruning neural network detection unit, wherein:

[0056] The lightweight feature extraction unit is used to extract lightweight network abnormal traffic features. This module has the same functions as the lightweight feature extraction module deployed in the cloud.

[0057] Lightweight pruned neural network detection unit, which is used to detect abnormal network traffic on edge devices. The cloud sinks the lightweight pruned neural network and deploys it on the edge, so that resource-constrained edge devices can use lightweight deep learning to detect abnormal network traffic.

[0058] In one or more implementations, this embodiment provides an intrusion detection method for edge device-level lightweight pruned neural network, which is based on the above-mentioned intrusion detection system for edge device-level lightweight pruned neural network, including:

[0059] Collect and receive network traffic data in real time;

[0060] Extracting features from the network traffic data to obtain a network traffic data feature vector;

[0061] The network traffic data feature vector is used as the input of a network abnormal traffic detection model based on a lightweight pruned neural network to obtain an abnormal judgment result of the network traffic and realize edge detection of network intrusion.

[0062] In further embodiments, there is also provided:

[0063] An electronic device includes a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the above embodiment. For the sake of brevity, no further description is given here.

[0064] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0065] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0066] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in the above embodiment is completed.

[0067] The method in the above embodiment can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0068] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in the present embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0069] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An intrusion detection system based on lightweight pruned neural network at edge device level, characterized by: include: A neural network pre-training module is set in the cloud, which is used to pre-train a preset network abnormal traffic detection model based on a neural network, and to prune the pre-trained network abnormal traffic detection model based on a neural network to obtain a network abnormal traffic detection model based on a lightweight pruned neural network, and send it to the edge device; Wherein, the pre-training is based on a pre-constructed training set, each sample in the training set includes a network traffic data feature vector and its corresponding classification result, and the network traffic data feature vector includes several traffic features; the pruning process is specifically as follows: calculating the importance score of each traffic feature in the network traffic data feature; based on the importance score of each traffic feature, calculating the feature pruning probability of the traffic feature; determining the traffic feature to be pruned based on the pruning probability of each traffic feature; at the same time, by calculating the pruning probability of each neuron in each hidden layer in the pre-trained neural network-based network abnormal traffic detection model, determining the neurons to be pruned in each hidden layer; by pruning the traffic features to be pruned and the neurons to be pruned, a network abnormal traffic detection model based on a lightweight pruned neural network is obtained; as well as, An edge-end pruned neural network intrusion detection module is set on the edge device, which is used to receive a network abnormal traffic detection model based on a lightweight pruned neural network from the cloud, and perform intrusion detection on the network traffic based on the network abnormal traffic detection model.

2. The intrusion detection system for edge device-level lightweight pruned neural network according to claim 1, characterized in that: The importance score of the traffic feature is calculated using the following formula: in, Indicates the nth i Traffic characteristics Importance score, n v Represents the total number of features, n y Indicates the number of classification categories, Indicates calculation of the nth i Input features and nth j Input features The Spearman correlation coefficient between Indicates calculation of the nth i Input features With the kth label y k The Spearman correlation coefficient.

3. The intrusion detection system for edge device-level lightweight pruned neural network according to claim 1, characterized in that: The calculation of the feature pruning probability of the traffic feature is specifically based on the following formula: in, Indicates the nth i Traffic characteristics The feature pruning probability of .

4. The intrusion detection system for edge device-level lightweight pruned neural network according to claim 1, characterized in that: The calculation of the pruning probability of the neuron is specifically based on the following formula: in, represents the jth neuron h in the kth hidden layer of the neural network j The pruning probability of represents the number of neurons in the kth layer, M k-1 [p,q] represents the mask matrix M of the same size as the weight matrix of the pre-trained neural network-based network anomaly traffic detection model k The value of the element in the p-th row and q-th column.

5. The intrusion detection system for edge device-level lightweight pruned neural network according to claim 1, characterized in that: The network abnormal traffic detection model based on neural network specifically includes an input layer, a hidden layer and an output layer, wherein the hidden layer uses a forward propagation mechanism to calculate each neuron and updates the neuron through a preset activation function; the output layer uses a Sigmoid function as a classifier to judge the abnormality of network traffic.

6. The intrusion detection system for edge device-level lightweight pruned neural network according to claim 1, characterized in that: For the edge-end pruned neural network intrusion detection module, its intrusion detection process is specifically as follows: feature extraction is performed on the network traffic data to be detected to obtain a network traffic data feature vector; the network traffic data feature vector is used as the input of a network abnormal traffic detection model based on a lightweight pruned neural network to obtain an abnormal judgment result of the network traffic, thereby realizing edge-end detection of network intrusion.

7. The intrusion detection system for edge device-level lightweight pruned neural network according to claim 1, characterized in that: The network traffic data feature vector includes a data type, an operation type, and an event timestamp.

8. An intrusion detection method for edge device-level lightweight pruned neural network, based on the intrusion detection system for edge device-level lightweight pruned neural network as claimed in any one of claims 1 to 7, characterized in that: include: Collect and receive network traffic data in real time; Extracting features from the network traffic data to obtain a network traffic data feature vector; The network traffic data feature vector is used as the input of a network abnormal traffic detection model based on a lightweight pruned neural network to obtain an abnormal judgment result of the network traffic and realize edge detection of network intrusion.

9. An electronic device comprising a memory, a processor and a computer program stored and running on the memory, characterized in that: When the processor executes the program, the intrusion detection method for edge device-level lightweight pruned neural network as described in claim 8 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the intrusion detection method for edge device-level lightweight pruned neural network as described in claim 8 is implemented.

Images