A lightweight IoT malicious behavior identification method and system

Abstract

The application discloses a kind of identification method and system of lightweight IoT malicious behavior, belong to network security technical field.The application is by obtaining the traffic data in the monitoring range of IoT gateway, the traffic data obtained is preprocessed as one-way flow metadata based on equipment, then host-level features are generated based on one-way flow metadata and high-quality host-level features are retained, and then abnormal detection model and malicious family detection model are constructed based on high-quality host-level features;IoT malicious traffic is identified based on high-quality host-level features using abnormal detection model, and the malicious family that traffic originates from is identified based on the high-quality host-level features of IoT malicious traffic using the identification malicious family detection model.The application can identify infected IoT device in the monitoring network segment and the malicious family infected by it.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, specifically relating to a method and system for identifying lightweight IoT malicious behavior. Background Technology

[0002] IoT malware refers to programs or code that target IoT devices and aim to manipulate them for illegal activities. IoT malware locates target devices through IP scanning and spreads by brute-forcing remote login services or exploiting vulnerabilities in other services. Infected devices typically communicate with command and control servers via DGA domains and execute malicious commands. IoT malware often targets distributed denial-of-service attacks, the theft of confidential information, cryptocurrency mining, and the spread of malicious emails.

[0003] Identifying IoT malware is a crucial step in preventing IoT threats, and existing research focuses on identifying IoT malware from a network behavior perspective. Traditional malicious traffic identification typically employs Deep Packet Inspection (DPI) or Deep Flow Inspection (DFI) techniques, such as common Network Intrusion Detection Systems (NIDS) like Snort and Suricata, which identify known malicious traffic by detecting packet headers, packet payloads, or JA3 and JA3S fingerprints in TLS handshakes. With the development of AI technology, machine learning (ML)-based methods have become a research hotspot. Fingerprint-based detection methods struggle to provide sufficient detection levels for unknown malware variants, while ML methods, after supervised or unsupervised training, can identify unknown malicious traffic with similar behavioral patterns. Existing IoT malicious traffic identification methods are mainly based on packet or flow-level machine learning or deep learning identification methods. However, due to the large computational demands of fine-grained traffic processing (packet or flow) and the limited computing resources of IoT devices, existing methods consistently face problems of low efficiency and high cost, making the protection of IoT devices challenging. Since IoT devices are generally low-power, limited in computing power and memory, traditional identification methods, which rely on expensive computing resources, are no longer suitable for IoT scenarios.

[0004] Therefore, to enhance the security of IoT networks, an accurate and efficient method for identifying malicious IoT traffic is needed. This method should be able to identify infected devices within the monitored network segment with low computational cost, high computational efficiency, and high identification accuracy, even in resource-constrained scenarios such as IoT gateways. Summary of the Invention

[0005] The purpose of this invention is to propose a lightweight IoT malicious behavior identification method and system based on host-level high-efficiency traffic processing. This method can identify infected IoT devices and their infected malware families within a monitored network segment, achieving higher computational efficiency with coarser traffic processing granularity. This invention aggregates traffic on a device-by-device basis and characterizes the connection behavior, network activity, and service request type features of IoT malware based on its network behavior. Combined with lightweight machine learning algorithms, it solves the problems of low identification efficiency and high computational cost in existing malicious traffic identification technologies.

[0006] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0007] A method for identifying malicious behavior in lightweight IoT devices includes the following steps:

[0008] S1: Obtain traffic data within the monitoring range of the IoT gateway;

[0009] S2: Preprocess the acquired traffic data into device-based unidirectional flow metadata;

[0010] S3: Generate host-level features based on unidirectional stream metadata;

[0011] S4: Retain high-quality host-level features through feature selection;

[0012] S5: Construct anomaly detection models and malicious family detection models based on high-quality host-level features;

[0013] S6: Uses an anomaly detection model based on high-quality host-level features to identify malicious IoT traffic;

[0014] S7: Use the malicious family identification detection model to identify the malicious family from which the traffic originates, based on the high-quality host-level features of IoT malicious traffic.

[0015] Furthermore, the methods for obtaining traffic data in step S1 include two types:

[0016] Online traffic capture: Traffic data is input in blocks based on a queue approach to capture traffic.

[0017] Offline data reading: Data is read by inputting traffic data in blocks using a sliding window approach.

[0018] Furthermore, the data preprocessing steps in step S2 include:

[0019] Retain one-way traffic sent from hosts within the monitored area to the outside of the gateway, and filter out traffic sent from the outside to the inside;

[0020] For retained unidirectional traffic, traffic packets from the same source IP are aggregated into a host-level flow;

[0021] Metadata is extracted from the host-level stream, including source port, destination IP, destination port, packet length, and timestamp interval information, and a metadata dictionary is constructed.

[0022] Furthermore, the host-level features in step S3 include:

[0023] Connection behavior characteristics include the non-repetition rate of destination IP, source port, and destination port of a packet originating from an IoT device, as well as the mean, variance, skewness, and kurtosis of the frequency list of the destination IP, source port, and destination port.

[0024] Network activity characteristics include the byte sequence of packet blocks originating from an IoT device, the mean, variance, skewness, kurtosis, and packet transmission rate between timestamps;

[0025] The request service type characteristics include the access service type of the packet originating from an IoT device and the distribution of the destination port number. The access service type includes HTTP service, HTTPS service, SSH service, Telnet service, Mail service, DNS service, NTP service, MQTT service, UPnP service, MySQL service, IRC service, and BitTorrent service. The distribution of the destination port number is whether the device accesses the recognized port, registered port, and dynamic port.

[0026] Furthermore, the feature selection step in step S4 includes: calculating the contribution of each host-level feature to the decision criteria during the construction of the decision tree using the random forest algorithm, ranking the importance according to the contribution, and selecting the top few host-level features with the greatest importance as high-quality host-level features.

[0027] Furthermore, the decision-making criterion is to select one of the following: Gini coefficient, information gain, or mean square error.

[0028] Furthermore, the method for constructing an anomaly detection model based on high-quality host-level features in step S5 is as follows: construct a binary classification dataset containing benign traffic and malicious traffic using high-quality host-level features, and train the anomaly detection model using the random forest algorithm on the binary classification dataset.

[0029] Furthermore, the method for constructing a malicious family detection model based on high-quality host-level features in step S5 is as follows: a multi-class dataset containing several malicious family samples is constructed using high-quality host-level features, and the malicious family detection model is trained on the multi-class dataset using the random forest algorithm to obtain the malicious family detection model.

[0030] Furthermore, the malicious families are specifically 11 malicious families: Mirai, Gafgyt, IRCbot, Kenjiro, Okiru, Muhstik, Hideandseek, Trojan, Troii, Hakai, and Hajime.

[0031] A lightweight IoT malicious behavior identification system includes:

[0032] The IoT gateway traffic input module is used to acquire traffic data within the monitoring range of the IoT gateway.

[0033] The host-level metadata extraction module is used to preprocess the acquired traffic data into device-based unidirectional flow metadata.

[0034] The host-level feature vector construction module is used to generate host-level features based on unidirectional streaming metadata;

[0035] The feature selection module is used to retain high-quality host-level features through feature selection;

[0036] The training and evaluation module is used to build anomaly detection models and malicious family detection models based on high-quality host-level features.

[0037] Anomaly Detection Module: Used to identify malicious IoT traffic based on high-quality host-level features using an anomaly detection model; and to identify the malicious family from which the traffic originates using a malicious family identification detection model based on high-quality host-level features of malicious IoT traffic.

[0038] The log output module is used to output logs for anomaly detection and malicious family detection.

[0039] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0040] 1. This invention uses host-level traffic granularity as the identification granularity, which exchanges coarser granularity for higher computational efficiency. Without sacrificing the identification effect, it reduces the computational load caused by traffic processing and significantly improves the system throughput.

[0041] 2. This invention designs host-level features to describe the network behavior of IoT devices from a higher-dimensional perspective. These features are closely integrated with the lifecycle of IoT malware, avoiding the impact of a mixture of benign and malicious traffic from the same device.

[0042] 3. Based on host-level features, this invention further accurately identifies the malicious family from which malicious traffic originates, ultimately achieving the goal of precise location of infected IoT devices. Attached Figure Description

[0043] Figure 1This is a flowchart of a method for identifying lightweight IoT malicious behavior in an embodiment.

[0044] Figure 2 This is a block diagram of a lightweight IoT malicious behavior identification system in this embodiment. Detailed Implementation

[0045] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, a detailed description is provided below in conjunction with the accompanying drawings.

[0046] This invention discloses a method for identifying lightweight IoT malicious behavior. This method obtains traffic within a monitored network segment from an IoT gateway, with the key point being that the traffic is obtained in data blocks, with each data block containing 1000 data packets. The monitored network segment contains both benign and infected IoT devices. Traffic generated by benign devices is marked as benign, while traffic generated by infected devices is a mixture of benign and malicious traffic, which is marked as malicious. Infected devices involve various malware families. The process of this method is as follows: Figure 1 As shown, the specific steps include:

[0047] S1: Obtain traffic data within the monitoring range of the IoT gateway.

[0048] Data can be acquired through online traffic capture and offline data reading. Online capture uses a queue-based approach, inputting data in blocks. Offline reading uses a sliding window-based approach, also inputting data in blocks. A data block is a set of data packets acquired in each data collection session.

[0049] S2: Traffic preprocessing is based on device-specific unidirectional flow metadata.

[0050] Traffic preprocessing includes the following steps:

[0051] S2-1: Perform unidirectional flow filtering. Only retain unidirectional traffic sent from hosts within the monitored area to the outside of the gateway, and filter out traffic flowing from the outside to the inside.

[0052] S2-2: Aggregate by source IP. This results in several host-level flows. Each host-level flow is defined as the set of all packets originating from the same source IP.

[0053] S2-3: Metadata Extraction. For each host-level flow, extract the source port, destination IP, destination port, packet length, and timestamp interval for each data packet to prepare for the next step of extracting host-level features. This step forms a metadata dictionary with device IP as the key and the metadata of its host-level flow as the value.

[0054] S3: Building Host-Level Features

[0055] Based on the metadata dictionary formed in step S2, host-level features are generated. Host-level features mainly target the stage where malicious IoT processes exhibit significant active network behavior, including connection behavior features, network activity features, and requested service type features.

[0056] The feature construction process includes the following steps:

[0057] S3-1: Construction of Connection Behavior Features. This category contains 15 features, namely the non-repetition rate of the destination IP, source port, and destination port of a packet block originating from an IoT device, and the mean, variance, skewness, and kurtosis of the frequency list of the destination IP, source port, and destination port. The non-repetition rate is defined as the ratio of the number of unique elements in a set to the total number of elements. The frequency list is defined as the set of the number of times each element appears in a given set.

[0058] These characteristics are primarily used to identify malware scanning behavior. Many malware programs exhibit horizontal or vertical scanning behavior. Horizontal scanning refers to a botnet scanning multiple hosts for the same port number to search for targets running a specific service. Vertical scanning refers to scanning a large number of ports for the same target to detect the services running on the target host. This scanning behavior will generate connection activity that deviates from normal, benign traffic.

[0059] S3-2: Construction of Network Activity Features. This category contains 9 features, namely the byte sequence of packet blocks originating from an IoT device, the mean, variance, skewness, and kurtosis of the timestamp sequence, and the packet sending rate (packets / second).

[0060] Network activity characteristics describe the amount of data and packet frequency sent by a host within a collection period. A normal host's network activity should be within a suitable range. When a device is infected, IoT malware may perform scans or DoS attacks, generating a large number of packets and significantly increasing network load.

[0061] S3-3: Construction of Request Service Type Features. This category contains 15 features, which are derived from the access service types of packets from an IoT device, including HTTP, HTTPS, SSH, Telnet, Mail, DNS, NTP, MQTT, UPnP, MySQL, IRC, and BitTorrent services, corresponding to a total of 23 ports. It also includes the distribution of the destination port numbers, specifically whether the device accesses the recognized ports (0 to 1023), registered ports (1024 to 49151), and dynamic ports (49151 to 65535).

[0062] The basis for requesting service type characteristics is that a large number of malware programs tend to use different services when conducting CnC communication. This type of characteristic aims to effectively distinguish malicious activity traffic, including CnC communication, from benign traffic at the host level, and to differentiate the network behavior of different malware families.

[0063] S4: Host-level feature optimization

[0064] Feature selection is performed on the host-level traffic in step S3. All features undergo feature selection, with only a small number of high-quality features retained. High-quality features are selected by ranking the feature importance using a Random Forest (RF) algorithm. During the construction of the decision tree in the RF algorithm, the reduction in the criterion for each feature is calculated to determine the construction decision for the current branch. This criterion is calculated by the Gini coefficient by default.

[0065] S5: Establish an anomaly detection model and a malicious family detection model respectively.

[0066] Using the high-quality host-level features obtained in step S4, two datasets can be constructed. Dataset A is a binary classification dataset containing benign and malicious samples, which is directly input into the anomaly detection dataset for training. Dataset B is a multi-class dataset containing 11 malicious family samples, which is input into the malicious family detection dataset for training. Both the anomaly detection model and the malicious family detection model are based on the random forest algorithm. A test set is also constructed to test these two models.

[0067] The 11 malicious families include:

[0068] 1. Mirai: A botnet family that primarily infects IoT devices by scanning open ports and exploiting weak passwords, and is often used for DDoS attacks.

[0069] 2. Gafgyt: A botnet family primarily targeting IoT devices. It spreads by exploiting default or weak passwords and is also commonly used in DDoS attacks.

[0070] 3. IRCBot: A family of malware that communicates and controls via the IRC protocol. It is often used to create botnets, allowing attackers to send commands through the IRC channel to control infected devices.

[0071] 4. Kenjiro: A botnet family that typically also has the ability to scan and exploit vulnerabilities to infect devices.

[0072] 5. Okiru: A market-specific IoT malware family that typically spreads by scanning and exploiting weak passwords.

[0073] 6. Muhstik: A family of botnet malware targeting IoT devices, which infects devices by exploiting known vulnerabilities and incorporates them into the botnet.

[0074] 7. Hideandseek: A sophisticated family of P2P botnet malware capable of establishing direct communication links between infected devices, bypassing traditional command and control servers.

[0075] 8. Trojan: Contains various trojan programs targeting IoT devices, typically used to gain control of devices, steal information, or for use in botnets.

[0076] 9. Troii: A family of malware typically associated with Trojan programs, often disguised as legitimate software to trick users into downloading and installing them.

[0077] 10. Hakai: A family of malware specifically targeting routers, spreading by exploiting router vulnerabilities and incorporating infected devices into botnets.

[0078] 11. Hajime: A sophisticated family of IoT botnet malware that does not have a centralized command and control server and communicates via a P2P network.

[0079] S6: Abnormal Traffic Detection

[0080] Use anomaly detection models to identify traffic generated by IoT malware on the test set.

[0081] S7: Malicious Family Traffic Detection

[0082] For the malicious traffic identified in step S6, its host-level characteristics are input into the malicious family detection model to identify the malicious family from which the traffic originates.

[0083] This invention also discloses a lightweight IoT malicious behavior identification system for performing the above-described identification method. The system's modules are as follows: Figure 2 As shown, it specifically includes:

[0084] The IoT gateway traffic input module is used to acquire traffic data within the monitoring range of the IoT gateway. Specifically, it acquires network traffic online in the form of a queue or reads network traffic offline in the form of a sliding window.

[0085] The host-level metadata extraction module is used to preprocess the acquired traffic data into device-based one-way flow metadata. Specifically, it performs one-way flow filtering, host-level traffic aggregation, and metadata collection and retention on the acquired raw traffic.

[0086] The host-level feature vector construction module is used to generate host-level features based on unidirectional stream metadata, including connection behavior features, network activity features, and request service type features.

[0087] The feature selection module is used to select the best host-level features and filter out high-quality features.

[0088] The training and evaluation module is used to build anomaly detection models and malicious family detection models based on high-quality host-level features. Specifically, it constructs training and testing sets, trains and tests the identification model and the association model, and evaluates their overall accuracy, F1 score, recall rate of specified types, and precision of specified types.

[0089] Anomaly Detection Module: Used to identify malicious IoT traffic based on high-quality host-level features using an anomaly detection model; and to identify the malicious family from which the traffic originates using a malicious family identification detection model based on high-quality host-level features of malicious IoT traffic.

[0090] The log output module is used to output logs for anomaly detection and malicious family detection during the deployment phase.

[0091] Experimental test:

[0092] The experimental tests of this invention are based on the extended dataset IoT-23-Gate of the open-source dataset IoT-23. IoT-23-Gate contains 12.6GB of benign traffic and 16.2GB of mixed traffic. The benign traffic includes 9 IoT devices and 16 PC devices, while the mixed traffic includes 20 IoT devices. The mixed traffic is a mixture of benign and malicious traffic to simulate the real-world network behavior of infected devices in a real-world scenario. In the anomaly detection task, the IoT-23-Gate dataset was processed into 52,583 benign samples and 463,568 malicious samples, and the model classification metric was obtained through 10-fold cross-validation.

[0093] In the malware family classification task, mixed traffic was divided into 11 categories based on malware families. Due to the extreme imbalance of data across different families, the experiment performed oversampling and undersampling on the dataset. Specifically, based on the sample size of each family, the Mirai, Gafgyt, IRCbot, and Kenjiro families were downsampled to 8000 samples, the Okiru family was downsampled to 2000 samples, and the Muhstik, Hideandseek, Trojan, Troii, Hakai, and Hajime families were upsampled to 200 samples. The model classification metric was also obtained through 10-fold cross-validation.

[0094] Experiments demonstrate that the anomaly detection model achieves an accuracy and F1 score exceeding 0.99, while the malicious family detection model achieves an accuracy and F1 score exceeding 0.99 and 0.98, respectively. The system can achieve over 60,000 PPS (packets per second) when processing offline traffic data on a Raspberry Pi 4B. Therefore, the method and system provided by this invention can efficiently and accurately identify IoT malicious software traffic.

[0095] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A method for identifying malicious behavior in lightweight IoT devices, characterized in that, Includes the following steps: S1: Obtain traffic data within the monitoring range of the IoT gateway; S2: Preprocess the acquired traffic data into device-based unidirectional flow metadata; S3: Generate host-level features based on unidirectional stream metadata; host-level features include: Connection behavior characteristics include the non-repetition rate of destination IP, source port, and destination port of a packet originating from an IoT device, as well as the mean, variance, skewness, and kurtosis of the frequency list of the destination IP, source port, and destination port. Network activity characteristics include the byte sequence of packet blocks originating from an IoT device, the mean, variance, skewness, kurtosis, and packet transmission rate between timestamps; The request service type characteristics include the access service type and the distribution of the destination port number of the packet block originating from an IoT device. The access service type includes HTTP service, HTTPS service, SSH service, Telnet service, Mail service, DNS service, NTP service, MQTT service, UPnP service, MySQL service, IRC service and BitTorrent service. The distribution of the destination port number is whether the recognized port, registered port and dynamic port are accessed by the device. S4: Retain high-quality host-level features through feature selection; S5: Construct anomaly detection models and malicious family detection models based on high-quality host-level features; S6: Uses an anomaly detection model based on high-quality host-level features to identify malicious IoT traffic; S7: Use the malicious family identification detection model to identify the malicious family from which the traffic originates, based on the high-quality host-level features of IoT malicious traffic.

2. The identification method as described in claim 1, characterized in that, There are two methods for obtaining traffic data in step S1: Online traffic capture: Traffic data is input in blocks based on a queue approach to capture traffic. Offline data reading: Data is read by inputting traffic data in blocks using a sliding window approach.

3. The identification method as described in claim 1, characterized in that, The data preprocessing steps in step S2 include: Retain one-way traffic sent from hosts within the monitored area to the outside of the gateway, and filter out traffic sent from the outside to the inside; For retained unidirectional traffic, traffic packets from the same source IP are aggregated into a host-level flow; Metadata is extracted from the host-level stream, including source port, destination IP, destination port, packet length, and timestamp interval information, and a metadata dictionary is constructed.

4. The identification method as described in claim 1, characterized in that, The feature selection steps in step S4 include: calculating the contribution of each host-level feature to the decision criteria during the construction of the decision tree using the random forest algorithm, ranking the importance according to the contribution, and selecting the top few host-level features with the greatest importance as high-quality host-level features.

5. The identification method as described in claim 4, characterized in that, The decision criterion is to use one of the following: Gini coefficient, information gain, or mean square error.

6. The identification method as described in claim 1, characterized in that, The method for constructing an anomaly detection model based on high-quality host-level features in step S5 is as follows: construct a binary classification dataset containing benign traffic and malicious traffic using high-quality host-level features, and train the anomaly detection model using the random forest algorithm on the binary classification dataset.

7. The identification method as described in claim 1, characterized in that, The method for constructing a malicious family detection model based on high-quality host-level features in step S5 is as follows: a multi-class dataset containing several malicious family samples is constructed using high-quality host-level features, and the malicious family detection model is trained on the multi-class dataset using the random forest algorithm to obtain the malicious family detection model.

8. The identification method as described in claim 7, characterized in that, The malicious families are specifically 11: Mirai, Gafgyt, IRCbot, Kenjiro, Okiru, Muhstik, Hideandseek, Trojan, Troii, Hakai, and Hajime.

9. A lightweight IoT malicious behavior identification system, used to implement the identification method according to any one of claims 1-8, characterized in that, include: The IoT gateway traffic input module is used to acquire traffic data within the monitoring range of the IoT gateway. The host-level metadata extraction module is used to preprocess the acquired traffic data into device-based unidirectional flow metadata. The host-level feature vector construction module is used to generate host-level features based on unidirectional streaming metadata; The feature selection module is used to retain high-quality host-level features through feature selection; The training and evaluation module is used to build anomaly detection models and malicious family detection models based on high-quality host-level features. Anomaly detection module: Used to identify malicious IoT traffic based on high-quality host-level features using anomaly detection models; The malicious family identification detection model is used to identify the malicious family from which the traffic originates based on the high-quality host-level features of IoT malicious traffic. The log output module is used to output logs for anomaly detection and malicious family detection.

Images