An internet of things traffic collection and malicious traffic classification detection method

Abstract

The application discloses a kind of Internet of Things flow acquisition and malicious flow classification detection method, comprising: based on the polling dormancy length of adaptive determination, utilize data plane development kit to obtain the network traffic of Internet of Things;Based on multiple hash strategy and polling load strategy, the network traffic is shunted, and the data stream after load balancing is obtained;Based on the measurement data of data stream is handled with multiple packet aggregation based on aggregation update strategy, update multistage hash table and generate flow reorganization data packet;Malicious flow classification and detection are carried out to flow reorganization data packet, capture malicious code and generate corresponding flow detection result.The scheme is shunted to network traffic by combining multiple hash strategy and polling load strategy, load balancing is realized between multiple cores, and the utilization and performance of multicore processor are improved;Support one-stop malicious flow detection and malicious code capture, without using honeypot for entrapment type malicious code capture, to improve the efficiency and accuracy of flow detection.

Description

Technical Field

[0001] This invention relates to the field of traffic detection technology, and in particular to a method for collecting Internet of Things (IoT) traffic and classifying and detecting malicious traffic. Background Technology

[0002] In high-speed network scenarios, high-performance network traffic acquisition technology is the cornerstone of traffic analysis and a key technology for network security detection and tracing. As network scale continues to grow, the performance of traffic acquisition and analysis systems based on traditional network traffic acquisition frameworks can no longer keep pace with the expansion of traffic. Furthermore, traditional frameworks suffer from limitations in bypassing the kernel protocol stack and require multiple copying operations of data packets during transmission and reception, leading to significant waste of system resources. This makes network traffic acquisition and analysis components the bottleneck in various traffic analysis and attack detection systems. Simultaneously, with the increasing prevalence of network usage scenarios, network traffic is experiencing explosive growth, placing higher demands on network traffic security detection. The ability to quickly and efficiently detect abnormal traffic amidst a large volume of data is therefore crucial.

[0003] Currently, the main method for detecting IoT attacks and capturing malicious code is deep learning. However, this method has the following problems: (1) For continuous high-speed network traffic, simply collecting traffic and using detection algorithms will cause an imbalance in CPU core and memory load, which will lead to performance problems in traffic collection and make it impossible to achieve high-speed real-time abnormal traffic detection; (2) It can only detect attack traffic and malicious code separately, and most malicious code capture requires honeypot technology for trapping. Summary of the Invention

[0004] This invention provides a method for collecting IoT traffic and classifying and detecting malicious traffic, enabling high-speed real-time collection of IoT traffic and supporting one-stop malicious traffic detection and malicious code capture without the need for honeypots for trapping malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0005] According to one aspect of the present invention, a method for collecting Internet of Things (IoT) traffic and classifying and detecting malicious traffic is provided, the method comprising:

[0006] Based on the adaptively determined polling sleep duration, the network traffic of the Internet of Things is obtained using the data plane development kit;

[0007] Network traffic is distributed based on a multi-hash strategy and a round-robin load balancing strategy to obtain a load-balanced data stream;

[0008] Based on the aggregation update strategy, the measurement data of the data stream is aggregated into multiple packets, the multi-level hash table is updated, and traffic reassembly data packets are generated.

[0009] Malicious traffic is classified and detected by reconstructing traffic data packets, malicious code is captured, and corresponding traffic detection results are generated.

[0010] According to another aspect of the present invention, an Internet of Things (IoT) traffic collection and malicious traffic classification and detection device is provided, the device comprising:

[0011] The traffic acquisition module is used to acquire IoT network traffic using the data plane development kit based on an adaptively determined polling sleep duration.

[0012] The load balancing module is used to distribute network traffic based on multi-hash strategy and round-robin load strategy to obtain a load-balanced data stream;

[0013] The traffic reassembly module is used to perform multi-packet aggregation processing on the measurement data of the data stream based on the aggregation update strategy, update the multi-level hash table, and generate traffic reassembly data packets.

[0014] The malicious traffic detection module is used to classify and detect malicious traffic from reconstructed data packets, capture malicious code, and generate corresponding traffic detection results.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the IoT traffic collection and malicious traffic classification and detection method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the Internet of Things traffic collection and malicious traffic classification and detection method according to any embodiment of the present invention.

[0020] The technical solution of this invention acquires IoT network traffic using a data plane development kit based on an adaptively determined polling sleep duration; it distributes network traffic using a multi-hash strategy and a polling load strategy to obtain a load-balanced data stream; it performs multi-packet aggregation processing on the measurement data of the data stream based on an aggregation update strategy, updates the multi-level hash table, and generates traffic reconstructed data packets; it classifies and detects malicious traffic in the reconstructed data packets, captures malicious code, and generates corresponding traffic detection results. This technical solution maximizes polling efficiency and minimizes power consumption by using an adaptively determined polling sleep duration to collect network traffic; it achieves load balancing across multi-core processors by combining a multi-hash strategy and a polling load strategy to distribute network traffic, improving the utilization and performance of multi-core processors; it enhances the performance and efficiency of traffic reconstructing by performing multi-packet aggregation processing on the measurement data of the data stream based on an aggregation update strategy; and it supports one-stop malicious traffic detection and malicious code capture without the need for honeypot-based malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an IoT traffic collection and malicious traffic classification and detection method provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of an IoT traffic collection and malicious traffic classification and detection method provided in Embodiment 2 of the present invention;

[0025] Figure 3 This is a flowchart of traffic acquisition based on the DPDK framework provided in Embodiment 2 of the present invention;

[0026] Figure 4 This is a network topology diagram provided according to Embodiment 2 of the present invention;

[0027] Figure 5 This is a system architecture diagram provided according to Embodiment 2 of the present invention;

[0028] Figure 6 This is a threat detection flowchart for malicious traffic targeting the APT family, provided according to Embodiment 2 of the present invention;

[0029] Figure 7 This is a schematic diagram of the structure of an Internet of Things traffic collection and malicious traffic classification and detection device according to Embodiment 3 of the present invention;

[0030] Figure 8 This is a schematic diagram of the structure of an electronic device that implements the IoT traffic collection and malicious traffic classification and detection method according to embodiments of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1 This is a flowchart of an IoT traffic collection and malicious traffic classification and detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations involving high-speed real-time collection of IoT traffic and classification and detection of malicious traffic and capture of malicious code. This method can be executed by an IoT traffic collection and malicious traffic classification and detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1As shown in the figure, the IoT traffic collection and malicious traffic classification and detection method provided in this embodiment includes the following steps:

[0035] S110: Based on the adaptively determined polling sleep duration, the network traffic of the Internet of Things is obtained using the data plane development kit.

[0036] The Data Plane Development Kit (DPDK) is a set of function libraries and drivers for fast packet processing, which can greatly improve data processing performance and throughput, and increase the efficiency of data plane applications. The polling sleep duration can be set as the polling interval for the DPDK polling mode.

[0037] In this embodiment of the invention, the data packet rate of network traffic can be predicted, and the polling sleep duration of the DPDK polling mode can be adaptively adjusted according to the predicted data packet rate, thereby calling the DPDK framework to collect continuous high-speed network traffic of the Internet of Things. This embodiment, by using an adaptively determined polling sleep duration to collect network traffic, can maximize polling efficiency and minimize power consumption.

[0038] S120. Based on the multi-hash strategy and the round-robin load strategy, network traffic is distributed to obtain the load-balanced data stream.

[0039] Among these strategies, multi-hash strategy refers to using multiple hash calculations to statically and evenly distribute network data across the CPU cores of a multi-core processor, preventing the majority of network traffic from being distributed to the fewest CPU cores. Round-robin load balancing strategy refers to dynamically adjusting the distribution of network traffic based on the CPU core load. By adjusting the scale of network traffic distributed to CPU cores in real time, it can effectively reduce the possibility of CPU core overload caused by sudden traffic spikes and improve load balancing performance.

[0040] In this embodiment of the invention, upon receiving network traffic within the DPDK framework, a pre-configured multi-hash strategy can be invoked to perform multi-hash calculations, thereby distributing the network traffic more evenly across the corresponding CPU processing cores and avoiding collisions that occur during a single hash operation. To further improve the efficiency and performance of CPU cores in handling traffic processing tasks, a round-robin load balancing strategy can be invoked to poll and evaluate the load of each CPU processing core, migrating heavily loaded cores. Ultimately, under load balancing, the datagram flow is completed, and flow table information is saved. This embodiment, by combining a multi-hash strategy and a round-robin load balancing strategy to distribute network traffic, can achieve load balancing across multiple cores, improving the utilization and performance of multi-core processors.

[0041] S130. Based on the aggregation update strategy, perform multi-packet aggregation processing on the measurement data of the data stream, update the multi-level hash table, and generate traffic reassembly data packets.

[0042] The aggregation update strategy refers to a strategy used for fine-grained stream reassembly of data streams, which can be implemented through SIMD instruction sets and multi-packet aggregation techniques. A multi-level hash table refers to a hash table used to store network traffic measurement data, and the multi-level hash table is logically hierarchically related. Measurement data may include, but is not limited to: counts, maximum values, minimum values, TCP stream sequence numbers, etc.

[0043] In this embodiment of the invention, to accelerate flow reassembly efficiency, a pre-configured aggregation update strategy can be invoked to perform multi-packet aggregation processing on the measurement data of the collected data stream. For example, the parallelism and vectorization characteristics of the SIMD instruction set can be utilized to process multiple collected data packets simultaneously, improving processing efficiency. Simultaneously, multi-packet aggregation technology is used to merge the measurement data (such as counts, maximum values, minimum values, TCP flow sequence numbers, etc.) of multiple data packets in the same flow into a single update operation. This update operation is then used to update the measurement data to a pre-configured multi-level hash table, generating reassembled traffic data packets. This embodiment improves the performance and efficiency of traffic reassembly by employing an aggregation update strategy to perform multi-packet aggregation processing on the measurement data of the data stream.

[0044] S140. Classify and detect malicious traffic from the reconstructed data packets, capture malicious code, and generate corresponding traffic detection results.

[0045] In this embodiment of the invention, in order to capture and detect malicious traffic in network traffic, the reconstructed data packets after flow reconstruction can be finely classified to identify the malicious traffic. Then, malicious code in the malicious traffic can be captured by calling network packet analysis tools such as Wireshark. By obtaining the behavioral feature sequence of the malicious code when it runs in the sandbox, the corresponding traffic detection result can be determined by calling threat traffic detection models such as support vector machines, such as whether the current network traffic is APT threat traffic.

[0046] The technical solution of this invention acquires IoT network traffic using a data plane development kit based on an adaptively determined polling sleep duration; it distributes network traffic using a multi-hash strategy and a polling load strategy to obtain a load-balanced data stream; it performs multi-packet aggregation processing on the measurement data of the data stream based on an aggregation update strategy, updates the multi-level hash table, and generates traffic reconstructed data packets; it classifies and detects malicious traffic in the reconstructed data packets, captures malicious code, and generates corresponding traffic detection results. This technical solution maximizes polling efficiency and minimizes power consumption by using an adaptively determined polling sleep duration to collect network traffic; it achieves load balancing across multi-core processors by combining a multi-hash strategy and a polling load strategy to distribute network traffic, improving the utilization and performance of multi-core processors; it enhances the performance and efficiency of traffic reconstructing by performing multi-packet aggregation processing on the measurement data of the data stream based on an aggregation update strategy; and it supports one-stop malicious traffic detection and malicious code capture without the need for honeypot-based malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0047] Example 2

[0048] Figure 2 This is a flowchart of an IoT traffic collection and malicious traffic classification and detection method provided in Embodiment 2 of the present invention. It is further optimized and extended based on the above embodiments and can be combined with various optional technical solutions in the above embodiments. For example... Figure 2 As shown in the figure, the IoT traffic collection and malicious traffic classification and detection method provided in this embodiment includes the following steps:

[0049] S210: Based on the adaptively determined polling sleep duration, the network traffic of the Internet of Things is obtained using the data plane development kit.

[0050] In this embodiment of the invention, the traffic acquisition process based on the DPDK framework is as follows: Figure 3 As shown. Meanwhile, the network topology of the technical solution in this embodiment of the invention is as follows: Figure 4As shown, it demonstrates DPDK's detection of traffic information across multiple nodes. An Autonomous System (AS) is a collection of one or more IP networks; a Provider Edge (PE) router is located at the edge of the carrier's network, connected to the customer's network, and primarily used to provide VPN (Virtual Private Network) services or MPLS (Multiprotocol Label Switching) tunnels; an Autonomous System Boundary Router (ASBR) is located at the boundary between different ASs, responsible for routing and forwarding between ASs; and a Customer Edge (CE) router is located at the edge of the customer's network, connected to the carrier's network, and used to transmit customer network traffic to the carrier's network.

[0051] Furthermore, based on the above embodiments of the invention, the process of determining the polling sleep duration may include the following steps:

[0052] Step A1: Call the preset Kalman filter algorithm to predict the data packet rate of network traffic and obtain the predicted data packet rate;

[0053] Step A2: Initialize the sliding window of the data plane development kit, and use the packet prediction rate, total time required for polling, number of packets acquired during polling, packet processing time, number of polling attempts, and time required to acquire packets from the network card queue during polling within the sliding window to determine the average polling sleep duration within the sliding window;

[0054] Step A3: Adjust the average polling sleep duration using a preset exponential smoothing method to obtain the polling sleep duration.

[0055] Specifically, network traffic packet rate data from the previous and current moments in network log records can be selected to establish a state-space equation for the network packet rate. Based on the obtained state-space equation, a Kalman filter prediction and update process is performed to obtain the current state estimate and error covariance using the input data. Then, based on the obtained state estimate, the network traffic packet rate is predicted to obtain the predicted packet rate.

[0056] Furthermore, to adapt to different traffic environments, this embodiment employs an adaptive intermittent polling method based on packet rate prediction to achieve adaptive intermittent polling with packet-switching sleep. The specific process is as follows:

[0057] 1) Calculate the total time required for the i-th poll. Let the time length of the sliding window be... The packet prediction rate at time t is The polling sleep duration for the i-th poll is The total time required for the i-th poll is... The calculation method is as follows: ,in, This represents the time required to retrieve data packets from the network interface card queue during polling. This represents the number of data packets obtained in the i-th polling round. This represents the time required to process the j-th data packet. It must also satisfy the following formula: , Where H represents the total number of polling iterations within the sliding window. Indicates a unit of time.

[0058] 2) Calculate the average polling sleep duration within the sliding window. : Make the sliding window , , and Use the average value of the corresponding parameters within the sliding window respectively , , and Instead, it assumes that the processing time for each data packet is consistent, i.e. Then we have:

[0059] ; ;

[0060] The average polling sleep time within the sliding window The following can be determined:

[0061]

[0062] 3) Adjusting the average polling sleep duration using a preset exponential smoothing method: To adapt the polling cycle to changes in the data packet arrival rate, the polling sleep duration is finally adjusted using a preset exponential smoothing method, i.e.: .in, This is the smoothing coefficient. It's important to understand that the choice of sliding window length affects the accuracy and overhead of adjusting the polling sleep duration; a trade-off must be struck in practical applications to select an appropriate sliding window length.

[0063] S220: Invoke the pre-configured multi-hash strategy to perform hash calculations on the network traffic packets in order to allocate the network traffic to the corresponding CPU processing cores.

[0064] In this embodiment of the invention, S220 specifically includes the following steps:

[0065] S2201: The Toeplitz hash algorithm is used to calculate the hash value of the network traffic data packets collected in S210 to obtain the corresponding CPU core number and record it in the key-core hash table; then, the CPU core queries the key-stream hash table for the corresponding hash value based on the four-tuple of the data packet (source address SIP, destination address DIP, source port SPORT, destination port DPORT) to obtain the specific TCP session stream and record it in the session stream hash table.

[0066] S2202. For newly arriving data packets, use the hash lookup algorithm corresponding to the session stream. This can be achieved using the packet's quadruple (source address SIP, destination address DIP, source port SPORT, destination port DPORT) to calculate: .

[0067] S2203. Modify the classic hash algorithm, adjust the size of its hash space to suit node performance, and use the following hash function to determine the final generated hash result HRES:

[0068]

[0069]

[0070]

[0071] HKEY2 can be used to control the length of the output results.

[0072] S2204. After determining the hash function in S2203, modify the hash key used by RSS in DPDK to the corresponding value required by the S-RSS algorithm. When the data packet arrives at the network interface layer, DPDK will perform Toeplitz Hash calculation and allocate the network data packet to the rx queue corresponding to different capture cores according to different hash calculation results.

[0073] S2205. After the CPU core corresponding to S2204 obtains the data, it first searches the key-stream hash table in the corresponding data stream reassembly core of the current core, and determines whether the network stream corresponding to the current data packet already exists in the corresponding stream table. If the corresponding network stream can be found, the network stream data is added to the corresponding entry in the stream table. If the corresponding network stream is not found in the corresponding stream table, it searches the key-core hash table to see if the stream data has been distributed to other CPU cores for processing through a load balancing scheme. If the network stream can be found in the key-core hash table, the stream data is delivered to the rte_ring circular queue of the corresponding CPU core. Otherwise, it means that the stream data corresponds to a new TCP connection for the traffic acquisition module and the reassembly module.

[0074] S2206. When processing a new TCP connection, calculate the overall CPU load balancing. If the CPU load balancing exceeds the threshold, remove the overloaded CPU cores and recalculate the Toeplitz hash value. After calculation, add the data stream and the CPU core mapping relationship obtained from the hash result to the key-core hash table, send the traffic data to the rte_ring of the corresponding CPU core, and append the data stream information to the key-stream hash table of the corresponding core.

[0075] S2207. Evaluating CPU load balancing: The proposed session integrity-oriented load balancing evaluation index is used as the load balancing evaluation index. In a typical multi-core processor, the processing power of each core can be considered to be at the same level. Therefore, the specific formula for the load balancing degree among the cores of the multi-core processor at time t is as follows:

[0076]

[0077] in, This represents the utilization efficiency of the i-th CPU core at time t, where e is the number of cores in a multi-core CPU.

[0078] In this embodiment, the load balancing module will adopt a pipeline mode to distribute different packet processing tasks to different cores. When the CPU balance does not reach the threshold, i.e., when the load balancing module does not intervene, the traffic data on each acquisition core will be distributed to two flow reassembly cores; when the CPU balance reaches the threshold, i.e., when the load balancing module intervenes, the traffic on each acquisition core may be output to any CPU core running on any flow reassembly core.

[0079] S230. Determine the flow load value corresponding to the network traffic according to the packet receiving rate, message length and final forwarding rate of the network traffic within the preset period.

[0080] In this embodiment of the invention, network traffic (flow) is defined. The streaming load value of the processing core Within the preset period t The packet receiving rate is The message length is , The final forwarding rate is ,but The streaming load value of the processing core The following can be calculated:

[0081]

[0082] Among them, the final forwarding rate The following formula should be satisfied:

[0083]

[0084] In the formula, D, k1, b1, k2, and b2 are constants related to processing capacity, and L is the packet length. D is the baseline value of the final forwarding rate. k1, b1, k2, and b2 describe the linear relationship between the final forwarding rate and the packet length.

[0085] S240. Based on the network traffic processed by each CPU processing core and the corresponding flow load value of the network traffic, determine the core load value of each CPU processing core, and store the core load value and the flow load value of each network traffic of the corresponding CPU processing core in the processing core flow information table corresponding to the CPU processing core.

[0086] In this embodiment of the invention, the load of each CPU processing core can be quantified using the flow load value of each flow processed on the CPU processing core, while the Processing Core Flow Information Table (PCFIT) is used to maintain the flow load value of each CPU processing core. The PCFIT table provides a basis for calculating the core load value and subsequent load migration.

[0087] Assume that a set of processing cores in the system are ,in Let represent any CPU processing core. It is also assumed that this CPU processing core is responsible for processing m streams. Then the CPU processing core Corresponding core load value It is expressed as follows: .

[0088] S250. Classify the corresponding CPU processing cores according to the core load value of each CPU processing core and the preset heavy load threshold.

[0089] In this embodiment of the invention, S250 specifically includes the following steps:

[0090] S2501. Sort all CPU processing cores in descending order according to their corresponding core load values.

[0091] S2502: Start traversing from the CPU processing core with the maximum core load value and determine whether the corresponding core load value is less than the preset heavy load threshold.

[0092] S2503, If so, then all CPU processing cores will be designated as light-load processing cores.

[0093] S2504. If not, then the corresponding CPU processing core is identified as the heavy-load processing core, and the remaining CPU processing cores are traversed.

[0094] Specifically, assuming all CPU processing cores have the same processing power, the preset heavy load threshold is defined as follows: The classification process of CPU processing cores is as follows:

[0095] All CPU processing cores are assigned their corresponding core load values. Sort in descending order; classify CPU cores by traversing from the most heavily loaded CPU core: if the core load value meets... If this indicates that there are no heavily loaded processing cores in the current system, the adaptive load balancing phase can be terminated directly; if the core load value meets the requirements... If so, the CPU core is classified as a heavy-load core, and the process continues to iterate through other CPU cores. In subsequent CPU cores, if the core load value meets... If no CPU core is found to be under light load, the CPU core is classified as a light-load processing core. If no light-load processing core is found after traversing all CPU cores, it means that the current system is under full load and cannot be optimized through adaptive load balancing. In this case, the adaptive load balancing process ends.

[0096] S260. Based on the core classification results and processing core flow information table corresponding to each CPU processing core, perform load migration on the heavily loaded processing cores in the CPU processing cores to obtain the load-balanced data flow.

[0097] In this embodiment of the invention, S260 specifically includes the following steps:

[0098] S2601. When there are both heavy-load processing cores and light-load processing cores in the CPU processing core, the heavy-load processing core with the largest core load value is taken as the source core for load migration, and the network data of the latest record in the processing core flow information table corresponding to the source core is taken as the target migration flow.

[0099] S2602. Sort each light-load processing core in ascending order according to its corresponding core load value. After the light-load processing core with the smallest core load value is determined to receive the target migration flow, if the corresponding new core load value is less than the preset heavy load threshold, the corresponding light-load processing core is determined as the target light-load processing core, and the target migration flow is migrated to the target light-load processing core. At the same time, the core load values ​​are updated.

[0100] S2603. Continue to migrate the load on the source core until the load value of the new core corresponding to the source core is less than the preset heavy load threshold. Select the next heavy load processing core as the new source core and re-execute the load migration process on the new source core until there are no heavy load processing cores in the system or the target light load processing core cannot be found.

[0101] Specifically, when both heavily loaded and lightly loaded processing cores exist within a CPU processing core, the BBFF (Big Box First Fit) algorithm can be invoked to perform load migration based on the processing core classification results and the PCFIT table. Each CPU processing core corresponds to a PCFIT table, and whenever a new flow is assigned to a CPU processing core, the flow's identification information is added to the corresponding processing core's PCFIT table.

[0102] During load migration, when the system selects a flow to migrate from the heaviest processing core with the highest core load value, firstly, the most recently recorded flow in the PCFIT table corresponding to that heaviest processing core is selected as the target migration flow. Then, a target light-load core is selected based on the BBFF algorithm: each light-load processing core is checked in ascending order of core load value to ensure that if a light-load processing core accepts the target migration flow, its core load value will not exceed a preset heavy-load threshold. Finally, the core load values ​​are updated and the heaviest processing cores are rescheduled: when a flow migrates from one core to another, the core load value of the corresponding core is updated. If the core load value of a heaviest processing core is already below the preset heavy-load threshold, the next heaviest processing core is selected as the source core for load migration. These steps are repeated until there are no more heaviest processing cores in the system, or no target light-load processing core can be found.

[0103] S270: Invoke the preset SIMD instruction set and preset multi-packet aggregation technology to merge the measurement data of multiple data packets in the same data stream into a single update operation, and save the generated traffic reconstructed data packet as a PCAP file.

[0104] Among them, the preset SIMD (Single Instruction Multiple Data) instruction set is a parallel processing technique that allows multiple data elements to be processed in parallel within a single instruction operation, thereby improving computational efficiency. The preset multi-packet aggregation technique can merge the measurement results of multiple network packets belonging to the same flow into a single update operation, reducing the number of accesses to the network measurement hash table structure and the probability of collisions.

[0105] In embodiments of the present invention, such as Figure 5 As shown, the data stream obtained in S260 can be used as input, and the parallelism and vectorization characteristics of the preset SIMD instruction set can be used to process multiple network traffic data packets simultaneously, thereby improving processing efficiency. At the same time, the preset multi-packet aggregation technology is used to merge the measurement data (such as count, maximum value, minimum value, TCP flow sequence number, etc.) of multiple data packets in the same flow into an update operation, and the generated traffic reassembled data packet is saved as a PCAP file (a datagram storage format).

[0106] S280. Obtain the pre-configured multi-level hash table and update the measurement data to the multi-level hash table according to the update operation.

[0107] In this embodiment of the invention, a set of adjacent large prime numbers can be selected as the modulus of the hash operation result, and a set of coprime multi-level hash tables with a length equal to the set of large prime numbers can be established to store the network measurement results. The process of updating the measurement data to the multi-level hash table is as follows: When processing a data packet, the hash result calculated by the network card is first obtained, and the first large prime number is taken for modulo operation. If the address in the multi-level hash table does not contain data or the flow identifiers of the two are consistent, the measurement data is updated to the multi-level hash table; if the address in the multi-level hash table already contains data and the flow identifiers of the two are inconsistent, a hash collision occurs. At this time, the second large prime number is taken for modulo operation, and an attempt is made to put it into the next level hash table, and so on; if no place is found in the multi-level hash table, the record with the least amount of data and the longest last update time is evicted and exported.

[0108] S290. Invoke the preset traffic analysis tool to extract the TCP session flow from the PCAP file corresponding to the traffic reconstructing data packet.

[0109] In embodiments of the present invention, such as Figure 6As shown, after network traffic is collected, a malicious traffic detection and analysis process targeting the APT family, along with malicious code capture and threat detection, can be executed to detect malicious traffic and capture malicious code within the network traffic. Specifically, pre-defined traffic analysis tools such as TShark can be used to filter and analyze the PCAP files corresponding to the reconstructed data packets, removing non-TCP traffic, such as UDP traffic and ICMP (Internet Control Message Protocol) traffic. TCP session streams are then filtered out in batches from the network traffic, and each TCP session stream is saved separately as a PCAP file, named with a 5-tuple. For example, if the destination IP address is 59.41.39.125, the destination port is 22, the source IP address is 131.202.240.8, and the source port is 63041, then the file name would be TCP_59-41-39-125_63081_131-202-240-87_22.pcap. Each TCP session stream can be saved separately for subsequent analysis.

[0110] S2100: Call the preset malicious traffic classification model to determine whether the TCP session flow is a malicious TCP session. If it is, extract the destination IP address from the malicious TCP session. If not, terminate the subsequent malicious traffic detection process.

[0111] The preset malicious traffic classification model can be understood as a pre-configured model used to detect whether a TCP session flow is a malicious session flow. The preset malicious traffic classification model may include at least one of the following: k-Nearest Neighbor (KNN), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Linear Support Vector Machine (LSVM), Decision Tree (DT), and Random Forest (RF).

[0112] In this embodiment of the invention, a complete TCP session flow comprises three parts: connection establishment traffic, data transmission traffic, and connection termination traffic. The TCP session flow can be segmented by identifying the identifiers for "connection establishment" and "connection termination." Then, TCP session features (flow-level features) and request header field features (packet header-level features) are extracted from each TCP session flow. The request header field features include packet length features, arrival time features, etc. These features are then fused to form multimodal features, which are then uniformly vectorized using methods such as Self-Organizing Map (SOM) neural networks to convert the multimodal features into session feature vectors. Finally, the obtained session feature vectors are input into a pre-trained, preset malicious traffic classification model for classification, thereby determining whether the TCP session flow is a malicious TCP session. If so, the corresponding destination IP address is extracted from the malicious TCP session; otherwise, the subsequent malicious traffic detection process is terminated.

[0113] S2110. Based on the matching of the destination IP address with the preset malicious IP address database, the TCP session stream is filtered, and the corresponding TCP session stream with the destination IP address being a malicious IP address is retained.

[0114] The pre-configured malicious IP address database can refer to a database that is pre-configured locally and stores a number of known malicious IP addresses.

[0115] In this embodiment of the invention, the destination IP address extracted from the TCP session stream can be used to determine whether the address is a private address. For example, if neither the destination IP address nor the source IP address in step S290 is within the three categories of private addresses, then it is not a private address. If it is a private address, the detection process ends directly and the result is output. If it is not a private address, the API (Application Programming Interface) provided by OTX (Open Threat Exchange) is used to detect the destination IP address. OTX maintains pulses for the IP address for the user. Based on the return result of OTX, it is determined whether the number of pulses associated with the destination IP address is greater than 1. Taking IP addresses 131.202.240.8 and 59.41.39.125 as examples, their pulse counts in the OTX database are queried respectively, and it is determined whether they are malicious addresses based on the return result. If the number of pulses corresponding to the destination IP address is greater than 1, then the destination IP address is considered a malicious IP address, and its corresponding TCP session stream is retained; otherwise, the address is considered a benign address, and the TCP session stream is discarded.

[0116] Meanwhile, a pre-set malicious IP address database can be built locally to filter each TCP session stream based on the destination IP address. TCP session streams with destination IP addresses that are malicious are retained, while those with destination IP addresses are discarded.

[0117] S2120. Call the default Wireshark tool to capture malicious code in the TCP session stream and obtain the behavioral characteristic sequence of the malicious code when it runs in the sandbox.

[0118] In this embodiment of the invention, a preset Wireshark tool can be invoked according to the application layer protocol to capture malicious code in the TCP session stream. To obtain the behavioral feature sequence of the malicious code, the malicious code can be run in a sandbox to monitor its dynamic behavior and record activities such as system calls, file operations, network communication, and process creation during code execution. By extracting unit behavioral feature sequences from several consecutive time slices, multi-sequence description and modeling of APT family malicious behavior can be achieved. Specifically, APT malware behavior is modeled using a quintuple of software name, software operating system environment, malicious behavior set, object of software execution action, and path. The behavioral feature quintuple data from consecutive time slices together constitute a set of behavioral feature sequences. An independent behavioral feature sequence is constructed for each malicious behavior and optimized, redundant feature fields are deleted, and simple special characters are used to replace the software operating environment field. For example, if a suspected malicious code appears in the first-in-first-out sequence of the API call structure in the sandbox as: (23, Virtual AllocEx)→(23, WriteProcess Memory)→…→(23, CreateRemoteThread), then the corresponding abstract behavior structure (i.e., behavior feature sequence) is: (23, system|process,create, inject|remotethread).

[0119] S2130. Input the behavioral feature sequence into the preset support vector machine model to obtain the corresponding attack behavior score, and when the attack behavior score is greater than the preset score threshold, determine the TCP session stream corresponding to the behavioral feature sequence as an APT virus sample.

[0120] In this embodiment of the invention, the generated behavioral feature sequence can be input into a pre-trained preset support vector machine (SVM) model, and the behavioral feature sequence can be expressed as a first-order matrix. The frequency of occurrence of the target sample (TCP session stream) in multiple dimensions of the first-order matrix is ​​statistically analyzed, and attack behavior weights are assigned. Then, the preset SVM model is used to score the attack behavior in multiple dimensions, and the TCP session stream is determined to be an APT virus sample based on the generated attack behavior score. That is, when the attack behavior score is greater than a preset score threshold, the TCP session stream corresponding to the behavioral feature sequence is identified as an APT virus sample, thereby realizing the detection of threat traffic in IoT network traffic.

[0121] Furthermore, based on the above embodiments of the invention, the construction process of the preset support vector machine model may include the following steps:

[0122] Step B1: Extract a preset number of behavioral feature sequence samples of the APT family samples during their operation in the sandbox;

[0123] Step B2: Call the preset Blast algorithm to perform similarity detection on each behavioral feature sequence sample, and take the part of all behavioral feature sequence samples with similarity greater than the preset threshold as common features, and build a malicious behavior feature library based on the common features;

[0124] Step B3: Train the initial support vector machine model using the malicious behavior feature library to obtain the preset support vector machine model.

[0125] Specifically, a predetermined number of APT family samples can be obtained as training data, and behavioral feature sequence samples of the aforementioned APT family samples running in a sandbox can be extracted. Then, a predetermined Blast algorithm is called to perform similarity detection on each behavioral feature sequence sample, and the portion of all behavioral feature sequence samples with a similarity greater than a predetermined threshold (e.g., 90%) is extracted as APT common features. A malicious behavior feature library is then constructed based on the obtained APT common features. Finally, an SVM model is trained based on the data in the malicious behavior feature library to obtain a predetermined support vector machine model for threat traffic detection. This embodiment of the invention constructs a malicious behavior feature library based on the Blast algorithm, realizing multi-sequence description and modeling of malicious behaviors of APT families. The Blast algorithm is used to compare the similarity of behavioral feature sequences, extracting the optimal common features of the APT family. The common features of the APT family are used to fully express the differences between malicious traffic and normal traffic in the feature dimension, providing a foundation for subsequent threat traffic detection.

[0126] The technical solution of this invention proposes a multi-core dynamic and static load balancing method based on multiple hashing and round-robin load balancing. It uses a multiple hashing algorithm to statically distribute network flows to each CPU core, while dynamically adjusting the distribution probability of traffic among multiple cores based on the CPU core load, thereby improving the utilization and performance of multi-core processors. A multi-level hash table structure is used to quickly locate the flow to which a data packet belongs. An aggregation update strategy is employed, and the combination of these two methods realizes a fine-grained flow reassembly method based on a multi-level hash table, thereby reducing the number of hash table updates and the probability of collisions, and improving the performance and efficiency of traffic reassembly. The parallelization and vectorization characteristics of the SIMD instruction set are utilized to process multiple data packets simultaneously, thereby improving processing efficiency. Multi-packet aggregation technology is used to merge the measurement results of multiple network data packets belonging to the same flow into a single update operation, reducing the number of accesses to the network measurement hash table structure and the probability of collisions, accelerating flow reassembly efficiency while ensuring the accuracy of network measurements. To address the challenge of accurately extracting threat features from complex and massive datasets, this invention first achieves refined classification of malicious traffic based on multi-dimensional feature fusion, captures malicious code within the malicious traffic, extracts its behavioral features, and then utilizes a pre-defined support vector machine model to detect threat traffic in IoT. This allows for one-stop malicious traffic detection and malicious code capture, eliminating the need for honeypot-based trapping of malicious code, thereby improving the efficiency and accuracy of traffic detection.

[0127] Example 3

[0128] Figure 7 This is a schematic diagram of an IoT traffic collection and malicious traffic classification and detection device provided in Embodiment 3 of the present invention. Figure 7 As shown, the device includes:

[0129] Traffic acquisition module 31 is used to acquire network traffic of the Internet of Things based on the adaptively determined polling sleep duration using the data plane development kit;

[0130] The load balancing module 32 is used to distribute network traffic based on a multi-hash strategy and a round-robin load strategy to obtain a load-balanced data stream.

[0131] The traffic reassembly module 33 is used to perform multi-packet aggregation processing on the measurement data of the data stream based on the aggregation update strategy, update the multi-level hash table, and generate traffic reassembly data packets.

[0132] The malicious traffic detection module 34 is used to classify and detect malicious traffic from reconstructed data packets, capture malicious code, and generate corresponding traffic detection results.

[0133] The technical solution of this invention involves a traffic acquisition module that uses a data plane development kit to acquire network traffic for the Internet of Things (IoT) based on an adaptively determined polling sleep duration; a load balancing module that distributes network traffic using a multi-hash strategy and a polling load strategy to obtain a load-balanced data stream; a traffic reconstruction module that performs multi-packet aggregation processing on the measurement data of the data stream using an aggregation update strategy, updates the multi-level hash table, and generates traffic reconstruction data packets; and a malicious traffic detection module that classifies and detects malicious traffic in the traffic reconstruction data packets, captures malicious code, and generates corresponding traffic detection results. This technical solution maximizes polling efficiency and minimizes power consumption by using an adaptively determined polling sleep duration to collect network traffic; it achieves load balancing across multiple cores by combining a multi-hash strategy and a polling load strategy to distribute network traffic, improving the utilization and performance of multi-core processors; it enhances the performance and efficiency of traffic reconstruction by using an aggregation update strategy to aggregate the measurement data of the data stream; and it supports one-stop malicious traffic detection and malicious code capture without the need for honeypot-based malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0134] Furthermore, based on the above embodiments of the invention, the process of determining the polling sleep duration includes:

[0135] The preset Kalman filter algorithm is invoked to predict the packet rate of network traffic, and the predicted packet rate is obtained.

[0136] Initialize the sliding window of the data plane development kit, and determine the average polling sleep duration within the sliding window using the packet prediction rate, total polling time, number of packets acquired during polling, packet processing time, number of polling attempts, and time required to acquire packets from the network interface card queue during polling.

[0137] The average polling sleep duration is adjusted using a preset exponential smoothing method to obtain the polling sleep duration.

[0138] Furthermore, based on the above embodiments of the invention, the load balancing module 32 includes:

[0139] The multi-hash calculation unit is used to call a pre-configured multi-hash strategy to perform hash calculations on network traffic packets in order to allocate network traffic to the corresponding CPU processing cores.

[0140] The flow load value determination unit is used to determine the flow load value corresponding to the network traffic according to the packet receiving rate, message length and final forwarding rate of the network traffic within a preset period.

[0141] The core load value determination unit is used to determine the core load value of each CPU processing core based on the network traffic processed by each CPU processing core and the corresponding network traffic flow load value, and to store the core load value and the flow load value of each network traffic of the corresponding CPU processing core in the processing core flow information table corresponding to the CPU processing core.

[0142] The core classification unit is used to classify the corresponding CPU processing cores according to the core load value of each CPU processing core and the preset heavy load threshold.

[0143] The load migration unit is used to perform load migration on the heavily loaded processing cores in the CPU processing cores according to the core classification results and processing core flow information table corresponding to each CPU processing core, so as to obtain the load-balanced data flow.

[0144] Furthermore, based on the above embodiments of the invention, the core classification unit is specifically used for:

[0145] Sort all CPU cores in descending order according to their corresponding core load values;

[0146] Starting from the CPU core with the highest core load value, traverse the system and determine whether the corresponding core load value is less than the preset heavy load threshold.

[0147] If so, then all CPU processing cores will be designated as light-load processing cores.

[0148] If not, the corresponding CPU processing core is identified as a heavy-load processing core, and the remaining CPU processing cores are traversed.

[0149] Furthermore, based on the above embodiments of the invention, the load migration unit is specifically used for:

[0150] When both heavy-load and light-load processing cores exist in the CPU processing core, the heavy-load processing core with the largest core load value is taken as the source core for load migration, and the network data of the latest record in the processing core flow information table corresponding to the source core is taken as the target migration flow.

[0151] Each light-load processing core is sorted in ascending order according to its corresponding core load value. After the light-load processing core with the smallest core load value is determined to receive the target migration flow, if the corresponding new core load value is less than the preset heavy load threshold, the corresponding light-load processing core is determined as the target light-load processing core, and the target migration flow is migrated to the target light-load processing core. At the same time, the core load values ​​are updated.

[0152] Continue to migrate the load on the source core until the load value of the new core corresponding to the source core is less than the preset heavy load threshold. Then select the next heavy load processing core as the new source core and repeat the load migration process on the new source core until there are no heavy load processing cores in the system or the target light load processing core cannot be found.

[0153] Furthermore, based on the above embodiments of the invention, the traffic reassembly module 33 includes:

[0154] The multi-packet aggregation unit is used to call the preset SIMD instruction set and preset multi-packet aggregation technology to merge the measurement data of multiple data packets in the same data stream into the same update operation, and save the generated traffic reassembled data packet as a PCAP file;

[0155] The measurement data update unit is used to obtain a pre-configured multi-level hash table and update the measurement data to the multi-level hash table according to the update operation.

[0156] Furthermore, based on the above embodiments of the invention, the malicious traffic detection module 34 includes:

[0157] The session stream extraction unit is used to call a preset traffic analysis tool to extract TCP session streams from the PCAP file corresponding to the traffic reconstructed data packets;

[0158] The malicious session detection unit is used to call a preset malicious traffic classification model to determine whether the TCP session flow is a malicious TCP session. If it is, the destination IP address is extracted from the malicious TCP session; otherwise, the subsequent malicious traffic detection process is terminated.

[0159] The session stream filtering unit is used to filter TCP session streams based on the matching of the destination IP address with a preset malicious IP address database, and retain the corresponding TCP session streams whose destination IP address is a malicious IP address;

[0160] The behavioral feature sequence acquisition unit is used to call the preset Wireshark tool to capture malicious code in the TCP session stream and obtain the behavioral feature sequence of the malicious code when it runs in the sandbox.

[0161] The APT virus detection unit is used to input behavioral feature sequences into a preset support vector machine model to obtain the corresponding attack behavior score. When the attack behavior score is greater than a preset score threshold, the TCP session stream corresponding to the behavioral feature sequence is identified as an APT virus sample.

[0162] Furthermore, based on the above embodiments of the invention, the pre-defined process for constructing the support vector machine model includes:

[0163] Extract a preset number of behavioral feature sequence samples of APT family samples running in a sandbox;

[0164] The system calls a preset Blast algorithm to perform similarity detection on each behavioral feature sequence sample, and takes the part of all behavioral feature sequence samples with similarity greater than a preset threshold as common features, and constructs a malicious behavior feature library based on the common features;

[0165] The initial support vector machine model is trained using a malicious behavior feature library to obtain a preset support vector machine model.

[0166] The IoT traffic collection and malicious traffic classification and detection device provided in this embodiment of the invention can execute the IoT traffic collection and malicious traffic classification and detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0167] Example 4

[0168] Figure 8 A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0169] like Figure 8 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0170] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0171] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as IoT traffic acquisition and malicious traffic classification and detection methods.

[0172] In some embodiments, the IoT traffic acquisition and malicious traffic classification and detection method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the IoT traffic acquisition and malicious traffic classification and detection method described above can be performed. Alternatively, in other embodiments, processor 41 can be configured to perform the IoT traffic acquisition and malicious traffic classification and detection method by any other suitable means (e.g., by means of firmware).

[0173] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0175] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0177] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0178] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0179] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0180] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for collecting Internet of Things (IoT) traffic and classifying and detecting malicious traffic, characterized in that, The method includes: Based on the adaptively determined polling sleep duration, the network traffic of the Internet of Things is obtained using the data plane development kit; The network traffic is distributed based on a multi-hash strategy and a round-robin load balancing strategy to obtain a load-balanced data stream; The measurement data of the data stream is aggregated using an aggregation update strategy, which updates the multi-level hash table and generates traffic reassembly data packets. The reconstructed data packets are classified and detected for malicious traffic, malicious code is captured, and corresponding traffic detection results are generated. The process of determining the polling sleep duration includes: The packet rate of the network traffic is predicted by calling a preset Kalman filter algorithm to obtain the predicted packet rate. The sliding window of the data plane development kit is initialized, and the average polling sleep duration within the sliding window is determined by using the packet prediction rate, total polling time, number of packets acquired during polling, packet processing time, number of polling attempts, and time required to acquire packets from the network card queue during polling. The average polling sleep duration is adjusted using a preset exponential smoothing method to obtain the polling sleep duration; The polling sleep duration is expressed as follows: ； in, and These represent the polling sleep durations for the i-th and (i+1)-th polling cycles, respectively. Indicates the smoothing coefficient; The average polling sleep duration is expressed as: ； in, This represents the average total time required for each poll within the sliding window; This indicates the time required to retrieve data packets from the network interface card queue during polling; This represents the average number of data packets retrieved in each poll within the sliding window; This represents the average time required to process each data packet. This represents the average packet prediction rate within the sliding window; The process of distributing network traffic based on a multi-hash strategy and a round-robin load balancing strategy to obtain a load-balanced data stream includes: The pre-configured multi-hash strategy is invoked to perform hash calculations on the network traffic packets, so as to allocate the network traffic to the corresponding CPU processing cores; The flow load value corresponding to the network traffic is determined based on the packet reception rate, message length, and final forwarding rate of the network traffic within a preset period. Based on the network traffic processed by each CPU processing core and the corresponding flow load value of the network traffic, the core load value of each CPU processing core is determined, and the core load value and the flow load value of each network traffic corresponding to the CPU processing core are stored in the processing core flow information table corresponding to the CPU processing core. The CPU processing cores are classified according to their core load value and preset heavy load threshold. According to the core classification result corresponding to each CPU processing core and the processing core flow information table, the heavy-load processing cores in the CPU processing cores are subjected to load migration, thereby obtaining the data flow after load balancing. The flow load value corresponding to the network traffic is expressed as follows: ； in, Indicates network traffic The corresponding streaming load value; This represents the network traffic within a preset period t. The packet receiving rate; Indicates network traffic The final forwarding rate; The process of performing multi-packet aggregation processing on the measurement data of the data stream based on the aggregation update strategy, updating the multi-level hash table, and generating traffic reassembly data packets includes: By invoking a preset SIMD instruction set and a preset multi-packet aggregation technology, the measurement data of multiple data packets in the same data stream are merged into a single update operation, and the generated traffic reassembly data packet is saved as a PCAP file; Obtain the pre-configured multi-level hash table, and update the measurement data to the multi-level hash table according to the update operation.

2. The method according to claim 1, characterized in that, The classification of the corresponding CPU processing cores according to the core load value of each CPU processing core and a preset heavy load threshold includes: All CPU processing cores are sorted in descending order according to their corresponding core load values; Starting from the CPU processing core with the maximum core load value, traverse the system and determine whether the corresponding core load value is less than the preset heavy load threshold. If so, then all of the CPU processing cores will be identified as light-load processing cores; If not, the corresponding CPU processing core is identified as a heavy-load processing core, and the remaining CPU processing cores are traversed.

3. The method according to claim 1, characterized in that, The step of performing load migration on the heavily loaded processing cores in the CPU processing cores according to the core classification result corresponding to each CPU processing core and the processing core flow information table includes: When both heavy-load and light-load processing cores exist in the CPU processing core, the heavy-load processing core with the largest core load value is taken as the source core for load migration, and the network data of the latest record in the processing core flow information table corresponding to the source core is taken as the target migration flow. The light-load processing cores are sorted in ascending order according to their corresponding core load values. After the light-load processing core with the smallest core load value accepts the target migration flow, if the corresponding new core load value is less than the preset heavy load threshold, the corresponding light-load processing core is determined as the target light-load processing core, and the target migration flow is migrated to the target light-load processing core. At the same time, the core load values ​​are updated. Continue to migrate the load on the source core until the load value of the new core corresponding to the source core is less than the preset heavy load threshold. Then select the next heavy load processing core as the new source core and re-execute the load migration process on the new source core until there are no heavy load processing cores in the system or the target light load processing core cannot be found.

4. The method according to claim 1, characterized in that, The process of classifying and detecting malicious traffic from the reconstructed data packets, capturing malicious code, and generating corresponding traffic detection results includes: The preset traffic analysis tool is invoked to extract the TCP session flow from the PCAP file corresponding to the traffic reconstructed data packet; A preset malicious traffic classification model is invoked to determine whether the TCP session flow is a malicious TCP session. If it is, the destination IP address is extracted from the malicious TCP session; otherwise, the subsequent malicious traffic detection process is terminated. Based on the matching results between the destination IP address and the preset malicious IP address database, the TCP session streams are filtered, and the TCP session streams corresponding to the destination IP address being a malicious IP address are retained; The system invokes a pre-defined Wireshark tool to capture malicious code in the TCP session stream and obtains a sequence of behavioral characteristics of the malicious code when it runs in the sandbox. The behavioral feature sequence is input into a preset support vector machine model to obtain the corresponding attack behavior score. When the attack behavior score is greater than a preset score threshold, the TCP session stream corresponding to the behavioral feature sequence is identified as an APT virus sample.

5. The method according to claim 4, characterized in that, The construction process of the preset support vector machine model includes: Extract a preset number of behavioral feature sequence samples of APT family samples running in a sandbox; The preset Blast algorithm is invoked to perform similarity detection on each of the behavioral feature sequence samples, and the portion of all the behavioral feature sequence samples with similarity greater than a preset threshold is taken as common features, and a malicious behavior feature library is constructed based on the common features; The malicious behavior feature library is used to train the initially constructed support vector machine model to obtain the preset support vector machine model.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the IoT traffic collection and malicious traffic classification and detection method according to any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the IoT traffic collection and malicious traffic classification and detection method according to any one of claims 1-5.

Images