Lightweight iot malicious traffic recognition method based on cross-view knowledge distillation

Abstract

This invention discloses a lightweight IoT malicious traffic identification method based on cross-view knowledge distillation. The method first captures and preprocesses IoT network traffic, then extracts features from the flow-level information view and packet spatiotemporal sequence view. By constructing a multi-view teacher network, the features of each view are deeply encoded and fused. Furthermore, this invention constructs a single-view lightweight student network, implicitly learning multi-view knowledge during training through a cross-view knowledge distillation mechanism to achieve high-precision malicious traffic classification. This method features lightweight model, high accuracy, and low complexity, making it particularly suitable for deployment in resource-constrained IoT environments and of great significance for maintaining IoT network security.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, specifically a lightweight IoT malicious traffic identification method based on cross-view knowledge distillation. Background Technology

[0002] With the rapid development of IoT technology, the number of IoT devices is growing exponentially, leading to an explosive increase in network traffic. This growth trend has not only promoted the digital transformation of various industries but also brought unprecedented cybersecurity challenges. Cyber ​​attackers are using diverse techniques and methods to launch increasingly complex and covert attacks targeting the inherent security vulnerabilities of IoT devices, such as limited computing and storage capabilities and potential security vulnerabilities in firmware, posing a severe test to the security of the IoT environment.

[0003] In existing IoT traffic identification technologies, early simple analysis methods based on ports or load have gradually become ineffective due to the widespread adoption of dynamic port allocation and data encryption technologies. Although traditional machine learning-based methods have improved identification capabilities to some extent through automated feature learning, they are still limited by the complexity of feature engineering and the learning capacity of the models when dealing with complex patterns.

[0004] In recent years, deep learning-based methods have demonstrated great potential in the field of IoT traffic identification due to their superior feature extraction and generalization capabilities. However, the high computational and storage resource requirements of these methods limit their practical application in resource-constrained IoT environments. Furthermore, existing technologies often analyze traffic from a single perspective, ignoring the high heterogeneity and diversity of IoT devices, resulting in an inability to comprehensively capture traffic behavior patterns and characteristics.

[0005] Therefore, there is an urgent need for a lightweight method that can adapt to resource-constrained environments and provide high-precision malicious traffic identification. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a lightweight IoT malicious traffic identification method based on cross-view knowledge distillation. This method aims to improve the accuracy of identification by comprehensively utilizing traffic data from multiple perspectives, while reducing the complexity of the model and resource consumption, in order to adapt to resource-constrained IoT environments.

[0007] The technical solution to achieve the purpose of this invention is as follows: In a first aspect, this invention provides a lightweight IoT malicious traffic identification method based on cross-view distillation, comprising the following steps:

[0008] Step 1: Capture benign and malicious traffic flowing through the IoT platform, and process the input traffic samples into session traffic based on the 5-tuple information, and perform deduplication and grouping operations on them; the 5-tuple is the source address, destination address, source port, destination port, and protocol 5-tuple;

[0009] Step 2: For the captured IoT traffic, construct a flow-level information view representation based on the flow duration and the average length of the data packets in the flow; construct a packet spatiotemporal sequence view representation based on the data packet length and the direction of the data packets.

[0010] Step 3: Based on the multi-view traffic representation obtained in Step 2, construct a multi-view teacher model, train it, and save it;

[0011] Step 4: Input the spatiotemporal sequence view of the packet and construct a single-view student model by combining it with a depthwise separable convolutional network;

[0012] Step 5: Use a cross-view knowledge distillation strategy to train and test the single-view student model to achieve accurate identification of malicious traffic in the Internet of Things.

[0013] In a second aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the first aspect.

[0014] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] Fourthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0016] Compared with existing technologies, the significant advantages of this invention are as follows: 1) The cross-view knowledge distillation mechanism adopted in this invention not only improves the accuracy of identifying malicious traffic in the Internet of Things (IoT), but also effectively reduces the complexity and resource consumption of the model by optimizing the model structure, enabling the model to be efficiently deployed on IoT devices or nodes with limited computing and storage resources; 2) By comprehensively utilizing the features of both flow-level information and packet sequence views, this invention can comprehensively capture the behavioral patterns of IoT traffic, effectively identify various abnormal behaviors including encrypted traffic, and ensure user privacy security without touching user data throughout the process; 3) The packet sequence encoder designed in this invention adopts depthwise separable convolution technology, replacing traditional convolution operations, which significantly reduces the computational burden of the model. Furthermore, by applying convolution kernels of different sizes, the encoder can accurately capture the spatiotemporal features of data packets at different time scales, enhancing the model's ability to identify traffic patterns.

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0019] Figure 1 This is a framework diagram of the lightweight IoT malicious traffic identification method based on cross-view distillation of the present invention. Detailed Implementation

[0020] like Figure 1 As shown, this invention proposes a lightweight IoT malicious traffic identification method based on cross-view distillation, which includes the following steps:

[0021] Step 1: Capture benign and malicious traffic data flowing through the IoT platform; classify and sessionify the captured traffic data using the five-tuple information (source IP address, destination IP address, source port, destination port, and transport protocol); perform preprocessing operations such as deduplication and grouping on the sessionified traffic data to prepare for subsequent feature extraction.

[0022] Step 2: For the captured IoT traffic, construct a flow-level information view representation based on the flow duration, the average length of the data packets in the flow, etc.; construct a packet spatiotemporal sequence view representation based on the data packet length and the direction of the data packets.

[0023] Here, the extracted flow-level information view features include, but are not limited to, 31 features such as flow duration and number of packets, and these features are standardized. The specific features are shown in Table 1. For the packet spatiotemporal sequence view, packets are marked according to the sending direction: traffic sent from IoT devices is marked as +1, and received traffic is marked as -1.

[0024] Table 1. Features extracted from the flow level information view and their descriptions

[0025] serial number Feature Description serial number Feature Description 01 Flow duration 17 Total length of downlink data packets 02 Uplink data packet count 18 Average downlink packet length 03 Downlink data packet count 19 Downlink packet length standard deviation 04 Total number of bidirectional data packets 20 Total length of bidirectional data packets 05 Average delay of uphill private rooms 21 Average bidirectional packet length 06 Total standard deviation of delay in uphill private rooms 22 Two-way package length standard deviation 07 Average total downlink latency of private rooms 23 Uplink bytes per second 08 Downlink private room delay total standard deviation 24 Downlink bytes per second 09 Total average delay of two-way private rooms 25 Bidirectional byte transfer per second 10 Total standard deviation of delay in two-way private rooms 26 Uplink data packet header bytes 11 Uplink packets per second 27 Downlink packet header bytes 12 Downlink packets per second 28 Bidirectional data packet header bytes 13 Bidirectional packet transmission per second 29 Uplink data packet header byte percentage 14 Total length of uplink data packets 30 Downlink data packet header byte percentage 15 Average uplink packet length 31 Bidirectional data packet header byte percentage 16 Upward package length standard deviation

[0026] Step 3: Using the multi-view features extracted in Step 2, construct a multi-view teacher model.

[0027] Here, the multi-view teacher model consists of a multi-view encoding module, a multi-view fusion module, and a classifier. The multi-view encoding module is divided into a stream-level view encoder and a bag sequence view encoder. The stream-level view encoder is used to encode stream-level information view features and is composed of multiple linear layers combined with residual connections. The bag sequence view encoder is used to encode bag spatiotemporal sequence view features and is composed of multiple parallel residual convolutional blocks of different scales. Each residual convolutional block contains a 1D convolutional layer, a BN layer, a ReLU activation function, and a residual connection. The kernel size of each residual convolutional block is different, with 3, 5, and 7 kernels respectively, to capture spatiotemporal features at different scales.

[0028] Here, the multi-view fusion module is based on a two-stage fusion strategy. The first stage of fusion relies on a multi-head cross-attention mechanism to linearly map the features of different views, generate Query, Key and Value, and fuse information through the cross-attention mechanism. The second stage of fusion concatenates the features obtained in the first stage and then inputs them into a multilayer perceptron for further feature fusion.

[0029] Here, the classifier of the teacher model consists of two fully connected linear layers; the fused multi-view features are input into the classifier and the multi-view teacher model is trained using the cross-entropy loss function to optimize classification performance;

[0030] Step 4: Input the spatiotemporal sequence view representation of the input packet and construct a single-view student model by combining it with a depthwise separable convolutional network;

[0031] Here, the single-view student model uses only the spatiotemporal sequence view of the bag as input. The single-view student model mainly consists of a lightweight bag sequence encoder and a classifier. The lightweight bag sequence encoder has a similar structure to the bag sequence encoder in the multi-view teacher model, consisting of multiple parallel residual convolutional blocks of different scales. Each residual convolutional block contains a 1D convolutional layer, a BN layer, a ReLU activation function, and residual connections. The 1D convolutional blocks are replaced with depthwise separable convolutions instead of standard convolutions, and depthwise separable convolutions are used in the 1D convolutional layers to reduce computational complexity and the number of parameters. The classifier consists of two fully connected linear layers, and the output probability distribution is z. s ;

[0032] Step 5: Use a cross-view knowledge distillation strategy to train and test the single-view student model to achieve accurate identification of malicious traffic in the Internet of Things.

[0033] Here, cross-view knowledge distillation is divided into two parts: soft loss optimization and hard loss optimization. In this way, the student model can implicitly learn the multi-view knowledge of the teacher model, thereby improving recognition accuracy.

[0034] Soft loss optimization performs knowledge distillation at both the response and feature levels. For knowledge distillation at the response level, the output probability distributions of the teacher and student models are softened using the temperature parameter T, and the difference between them is measured by KL divergence. For knowledge distillation at the feature level, the feature gap is reduced by maximizing the similarity of the output features of the teacher and student models.

[0035] Specifically, regarding knowledge distillation at the response level, assuming the probability distributions of the classifier outputs for the teacher and student models are z, respectively. t and z s The probability distribution of the output is softened by introducing a temperature parameter T, i.e.:

[0036] q t =softmax(z t / T)

[0037] q s =softmax(z s / T)

[0038] Furthermore, the soft probability distribution q of the student model is measured using the KL divergence. s The soft probability distribution q of the teacher model t The difference between them, that is:

[0039] L KD =D KD (q t ||q s )

[0040] Among them, L KD D represents the knowledge distillation loss at the response level. KD Let q represent the KL divergence function. t It is the soft probability distribution output by the teacher model, q s It is the soft probability distribution output by the student model;

[0041] For knowledge distillation at the feature level, assuming the output feature of the multi-view fusion module of the teacher model is f mv The encoder output of the student model is f p By maximizing f mv and f p The similarity between the two features is used to reduce the difference between them, that is:

[0042]

[0043] Thus, the overall soft loss function L is obtained. soft as follows:

[0044] L soft =L KD +aLf

[0045] Where 'a' is a hyperparameter used to balance L KD and L f These two loss items;

[0046] Furthermore, hard loss optimization is achieved through the cross-entropy loss function, namely:

[0047] L hard =L CE

[0048] Among them, L hard L represents the hard loss function. CE Represents the cross-entropy loss function;

[0049] Therefore, the total loss function L of the single-view student model std It is expressed as follows:

[0050] L std =λL soft +(1-λ)L hard

[0051] Wherein, λ is used to control the relative weights of soft loss and hard loss;

[0052] Furthermore, the trained single-view student model can be deployed in resource-constrained IoT environments to identify malicious IoT traffic.

[0053] This invention, through a cross-view knowledge distillation strategy, can not only achieve high-precision identification of malicious traffic in the Internet of Things (IoT), but also effectively reduce the complexity and size of the model, making it suitable for deployment in resource-constrained IoT environments. This is of great significance for maintaining IoT network security.

[0054] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A lightweight IoT malicious traffic identification method based on cross-view knowledge distillation, characterized in that, Includes the following steps: Step 1: Capture benign and malicious traffic flowing through the IoT platform, and process the input traffic samples into session traffic based on the five-tuple information, and perform deduplication and grouping operations on them; The quintuple consists of the source address, destination address, source port, destination port, and protocol quintuple. Step 2: For the captured IoT traffic, construct a flow-level information view representation based on the flow duration and the average length of the data packets in the flow; construct a packet spatiotemporal sequence view representation based on the data packet length and the direction of the data packets. Step 3: Based on the multi-view traffic representation obtained in Step 2, construct a multi-view teacher model, train it, and save it; Step 4: Input the spatiotemporal sequence view of the packet and construct a single-view student model by combining it with a depthwise separable convolutional network; Step 5: Use a cross-view knowledge distillation strategy to train and test the single-view student model to achieve accurate identification of malicious traffic in the Internet of Things.

2. The lightweight IoT malicious traffic identification method based on cross-view knowledge distillation according to claim 1, characterized in that, The flow-level information view representation extracted in Step 2 includes flow duration, number of uplink packets, number of downlink packets, total number of bidirectional packets, total mean of uplink inter-packet delay, total standard deviation of uplink inter-packet delay, total mean of downlink inter-packet delay, total standard deviation of downlink inter-packet delay, total mean of bidirectional inter-packet delay, total standard deviation of bidirectional inter-packet delay, uplink packets per second, downlink packets per second, bidirectional packets per second, total uplink packet length, average uplink packet length, standard deviation of uplink packet length, total downlink packet length, average downlink packet length, and downlink packet length. The following parameters are defined: long standard deviation, total bidirectional packet length, average bidirectional packet length, standard deviation of bidirectional packet length, uplink bytes per second, downlink bytes per second, bidirectional bytes per second, uplink packet header bytes, downlink packet header bytes, bidirectional packet header bytes, percentage of uplink packet header bytes, percentage of downlink packet header bytes, and percentage of bidirectional packet header bytes. These features are then standardized. For the packet spatiotemporal sequence view, the direction of the data packet is marked as +1 or -1 based on whether it was sent or received from an IoT device.

3. The lightweight IoT malicious traffic identification method based on cross-view knowledge distillation according to claim 1, characterized in that, The multi-view teacher model described in step 3 consists of a multi-view encoding module, a multi-view fusion module, and a classifier; The multi-view encoding module is divided into a stream-level view encoder and a packet sequence view encoder. The stream-level view encoder is used to encode stream-level information view features and consists of multiple linear layers combined with residual connections. The packet sequence view encoder is used to encode packet spatiotemporal sequence view features and consists of multiple parallel residual convolutional blocks of different scales. Each residual convolutional block contains a 1D convolutional layer, a BN layer, a ReLU activation function, and a residual connection. The kernel size of each residual convolutional block is different, with 3, 5, and 7 kernels respectively, to capture spatiotemporal features at different scales. The multi-view fusion module is based on a two-stage fusion strategy. The first stage of fusion is based on a multi-head cross-attention mechanism, which linearly maps the features of different views to generate Query, Key and Value, and then fuses the information through the cross-attention mechanism. The second stage of fusion involves concatenating the features obtained in the first stage and then inputting them into a multilayer perceptron for feature fusion. The classifier consists of two fully connected linear layers; the fused multi-view features are input into the classifier and the multi-view teacher model is trained using the cross-entropy loss function.

4. The lightweight IoT malicious traffic identification method based on cross-view knowledge distillation according to claim 1, characterized in that, The single-view student model described in step 4 uses only the spatiotemporal sequence view of the bag as input; the single-view student model mainly consists of a lightweight bag sequence encoder and a classifier; the lightweight bag sequence encoder consists of multiple parallel residual convolutional blocks of different scales, each residual convolutional block contains a 1D convolutional layer, a BN layer, a ReLU activation function and residual connections, wherein the 1D convolutional layer adopts depthwise separable convolution; the classifier consists of two fully connected linear layers.

5. The lightweight IoT malicious traffic identification method based on cross-view knowledge distillation according to claim 4, characterized in that, Step 5 describes cross-view knowledge distillation, which includes two parts: soft loss optimization and hard loss optimization. Soft loss optimization performs knowledge distillation at both the response and feature levels. For knowledge distillation at the response level, the output probability distributions of the teacher and student models are softened using the temperature parameter T, and the difference between them is measured by KL divergence. For knowledge distillation at the feature level, the feature gap is reduced by maximizing the similarity of the output features of the teacher and student models. For knowledge distillation at the response level, assume that the probability distributions of the classifier outputs of the teacher model and the student model are z, respectively. t and z s The probability distribution of the output is softened by introducing a temperature parameter T, i.e.: q t =softmax(z t / T) q s =softmax(z s / T) The soft probability distribution q of the student model is measured using the KL divergence. s The soft probability distribution q of the teacher model t The difference between them, that is: L KD =K KD (q t ||q s ) Among them, L KD D represents the knowledge distillation loss at the response level. KD Let q represent the KL divergence function. t It is the soft probability distribution output by the teacher model, q s It is the soft probability distribution output by the student model; For knowledge distillation at the feature level, assuming the output feature of the multi-view fusion module of the teacher model is f mv The encoder output of the student model is f p By maximizing f mv and f p The similarity between the two features is used to reduce the difference between them, that is: The overall soft loss function is thus obtained as follows: L soft =L KD +aL f Among them, L soft Let L represent the total soft loss function, where α is a hyperparameter used to balance L. KD and L f These two loss items; Hard loss optimization is achieved through the cross-entropy loss function, i.e.: L hard L CE Among them, L hard L represents the hard loss function. CE Represents the cross-entropy loss function; Therefore, the total loss function L of the single-view student model std It is expressed as follows: THE std =λL soft +(1-λ)L hard In the formula, λ is the relative weight used to control soft loss and hard loss.

6. The lightweight IoT malicious traffic identification method based on cross-view knowledge distillation according to claim 1, characterized in that, The trained single-view student model is deployed in an IoT environment to achieve real-time identification and classification of malicious IoT traffic.

7. A computer device comprising a memory, a processor, and a computer program on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-6.

Images