Internet of Things traffic collection and malicious traffic classification detection method

Abstract

The invention discloses an Internet of Things traffic collection and malicious traffic classification detection method, which comprises the following steps: acquiring network traffic of the Internet of Things by using a data plane development suite based on self-adaptively determined polling dormancy duration; shunting the network flow based on a multiple Hash strategy and a polling load strategy to obtain a data flow after load balancing; carrying out multi-packet aggregation processing on the measurement data of the data stream based on an aggregation updating strategy, updating the multi-level hash table and generating a flow recombination data packet; and malicious traffic classification and detection are carried out on the traffic recombination data packet, malicious codes are captured, and a corresponding traffic detection result is generated. According to the scheme, the network flow is shunted by combining the multi-Hash strategy and the polling load strategy, load balancing is realized among multiple cores, and the utilization rate and the performance of a multi-core processor are improved; the method supports one-stop malicious traffic detection and malicious code capture, and does not need to use a honey pot for trapping type malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

Description

Technical Field

[0001] The present invention relates to the technical field of flow detection, and in particular to a method for collecting flow and classifying malicious flow in an Internet of Things. Background Art

[0002] In high-speed network scenarios, high-performance network traffic collection technology is the cornerstone of traffic analysis and is also a key technology for network security detection and tracing. With the continuous growth of network scale, the performance of traffic collection and analysis systems based on traditional network traffic collection frameworks has been unable to keep up with the speed of traffic expansion. At the same time, in the process of sending and receiving data packets, traditional network traffic collection frameworks cannot bypass the kernel protocol stack and copy data packets multiple times, resulting in a large waste of system resources, making network traffic collection and analysis processing components the bottleneck of various traffic analysis and attack detection systems. At the same time, with more and more scenarios for using the network today, network traffic has also exploded, which puts higher requirements on network traffic security detection. How to quickly and efficiently detect abnormal traffic among a large number of traffic is particularly important.

[0003] At present, the methods for discovering IoT attack behaviors and capturing malicious codes mainly use deep learning. However, such methods have the following problems: (1) For continuous high-speed network traffic, simply collecting traffic and using detection algorithms will cause CPU core and memory load imbalance, which will in turn cause performance problems for traffic collection and fail to achieve high-speed and real-time abnormal traffic detection; (2) Only attack traffic and malicious code can be detected separately, and most malicious code capture requires trapping through honeypot technology. Summary of the invention

[0004] The present invention provides an Internet of Things traffic collection and malicious traffic classification detection method to achieve high-speed real-time collection of Internet of Things traffic, and support one-stop malicious traffic detection and malicious code capture, without the need to use honeypots for trapping malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0005] According to one aspect of the present invention, a method for IoT traffic collection and malicious traffic classification detection is provided, the method comprising:

[0006] Based on the adaptively determined polling sleep duration, the data plane development kit is used to obtain the network traffic of the Internet of Things;

[0007] Divert network traffic based on multiple hash strategies and polling load strategies to obtain load-balanced data flows;

[0008] Perform multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, update the multi-level hash table and generate traffic reassembly data packets;

[0009] Malicious traffic is classified and detected on traffic reassembled data packets, malicious code is captured and corresponding traffic detection results are generated.

[0010] According to another aspect of the present invention, a device for collecting traffic and detecting malicious traffic classification in an Internet of Things is provided, the device comprising:

[0011] A traffic acquisition module, used to acquire the network traffic of the Internet of Things using the data plane development kit based on the adaptively determined polling sleep duration;

[0012] The load balancing module is used to distribute network traffic based on multiple hash strategies and polling load strategies to obtain load-balanced data flows;

[0013] A traffic reassembly module, used for performing multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, updating the multi-level hash table and generating a traffic reassembly data packet;

[0014] The malicious traffic detection module is used to classify and detect malicious traffic on traffic reassembly data packets, capture malicious code and generate corresponding traffic detection results.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the Internet of Things traffic collection and malicious traffic classification detection method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for IoT traffic collection and malicious traffic classification detection described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention uses a data plane development kit to obtain the network traffic of the Internet of Things based on the polling sleep time determined adaptively; divides the network traffic based on the multiple hash strategy and the polling load strategy to obtain the load-balanced data flow; performs multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, updates the multi-level hash table and generates a traffic reassembly data packet; performs malicious traffic classification and detection on the traffic reassembly data packet, captures malicious code and generates corresponding traffic detection results. The technical solution of the embodiment of the present invention can maximize the polling efficiency and minimize the power consumption loss by adopting the polling sleep time determined adaptively to collect network traffic; divides the network traffic by combining the multiple hash strategy and the polling load strategy, can realize load balancing among multiple cores, and improve the utilization and performance of the multi-core processor; performs multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, can improve the performance and efficiency of traffic reassembly; and can support one-stop malicious traffic detection and malicious code capture, without the need to use honeypots for trapping malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flow chart of a method for IoT traffic collection and malicious traffic classification detection provided according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a flow chart of a method for IoT traffic collection and malicious traffic classification detection provided according to Embodiment 2 of the present invention;

[0025] Figure 3 This is a flow chart of traffic collection based on the DPDK framework provided according to the second embodiment of the present invention;

[0026] Figure 4 is a network topology diagram provided according to Embodiment 2 of the present invention;

[0027] Figure 5 is a system architecture diagram provided according to Embodiment 2 of the present invention;

[0028] Figure 6 This is a threat detection flow chart for APT family malicious traffic provided according to the second embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the structure of an IoT traffic collection and malicious traffic classification detection device provided according to Embodiment 3 of the present invention;

[0030] Figure 8 It is a structural schematic diagram of an electronic device that implements the method for collecting Internet of Things traffic and classifying malicious traffic according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 This is a flow chart of a method for collecting IoT traffic and classifying malicious traffic for detection provided in the first embodiment of the present invention. This embodiment is applicable to the case of high-speed real-time collection of IoT traffic and classification detection of malicious traffic and malicious code capture. This method can be performed by an IoT traffic collection and malicious traffic classification detection device. The IoT traffic collection and malicious traffic classification detection device can be implemented in the form of hardware and / or software. The IoT traffic collection and malicious traffic classification detection device can be configured in an electronic device. Figure 1As shown, the first embodiment provides an IoT traffic collection and malicious traffic classification detection method, which specifically includes the following steps:

[0035] S110, based on the adaptively determined polling sleep duration, obtain network traffic of the Internet of Things using a data plane development kit.

[0036] The Data Plane Development Kit (DPDK) is a set of function libraries and driver collections for fast data packet processing, which can greatly improve data processing performance and throughput, and improve the work efficiency of data plane applications. The polling sleep duration can be the polling interval set for the DPDK polling mode.

[0037] In the embodiment of the present invention, the data packet rate of the network traffic can be predicted, and then the polling sleep time of the DPDK polling mode can be adaptively adjusted according to the predicted data packet rate, and then the DPDK framework can be called to collect the continuous high-speed network traffic of the Internet of Things. This embodiment can maximize the polling efficiency and minimize the power consumption loss by using the adaptively determined polling sleep time to collect network traffic.

[0038] S120, diverting network traffic based on a multiple hash strategy and a polling load strategy to obtain a load-balanced data flow.

[0039] Among them, the multi-hash strategy can refer to a strategy that uses multiple hash calculations to statically and evenly distribute network data to each CPU core of a multi-core processor for processing, which can avoid the situation where most network traffic is distributed to a few CPU cores. The polling load strategy can refer to a strategy that dynamically adjusts the distribution of network traffic according to the load of the CPU core. By adjusting the scale of network traffic distributed to the CPU core in real time, it can effectively reduce the possibility of CPU core overload caused by burst traffic and improve the load balancing effect.

[0040] In an embodiment of the present invention, after receiving network traffic in the DPDK framework, a pre-configured multiple hashing strategy can be called to perform multiple hashing calculations on it, thereby distributing the network traffic more evenly to the corresponding CPU processing cores, and avoiding the collision problem caused by a single hashing; in order to further improve the efficiency and performance of the CPU core in processing traffic processing tasks, the polling load strategy can continue to be called to poll and evaluate the load of each CPU processing core, and load migrate the heavy-load processing core, and finally complete the datagram group flow under load balancing, and save the flow table information. This embodiment can achieve load balancing among multiple cores and improve the utilization and performance of multi-core processors by combining multiple hashing strategies and polling load strategies to divert network traffic.

[0041] S130 , performing multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, updating the multi-level hash table and generating a traffic reassembly data packet.

[0042] The aggregation update strategy may refer to a strategy for refining the data stream, which may be implemented by SIMD instruction set and multi-packet aggregation technology. The multi-level hash table may refer to a hash table for storing the measurement data of network traffic, and the multi-level hash table is logically hierarchically associated. The measurement data may include but is not limited to: count, maximum value, minimum value, TCP stream sequence number, etc.

[0043] In an embodiment of the present invention, in order to speed up the efficiency of flow reassembly, a pre-configured aggregation update strategy can be called to perform multi-packet aggregation processing on the measurement data of the collected data flow. For example, the parallelism and vectorization characteristics of the SIMD instruction set can be used to simultaneously process multiple collected data packets to improve processing efficiency; at the same time, multi-packet aggregation technology is used to merge the measurement data of multiple data packets in the same flow (such as count, maximum value, minimum value, TCP flow sequence number, etc.) into one update operation, and the update operation is used to update the measurement data to a pre-configured multi-level hash table, and at the same time generate a flow reassembly data packet after flow reassembly. This embodiment can improve the performance and efficiency of flow reassembly by performing multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy.

[0044] S140, classify and detect malicious traffic on the traffic reassembled data packets, capture malicious code and generate corresponding traffic detection results.

[0045] In an embodiment of the present invention, in order to capture and detect malicious traffic in network traffic, the traffic reassembled data packets after stream reassembly can be finely classified to determine the malicious traffic therein, and then the malicious code in the malicious traffic can be captured by calling network packet analysis tools such as Wireshark, and the behavioral feature sequence of the malicious code when running in the sandbox is obtained, and then the threat traffic detection model such as support vector machine is called to determine the corresponding traffic detection result, such as whether the current network traffic is APT threat traffic.

[0046] The technical solution of the embodiment of the present invention uses a data plane development kit to obtain the network traffic of the Internet of Things based on the polling sleep time determined adaptively; divides the network traffic based on the multiple hash strategy and the polling load strategy to obtain the load-balanced data flow; performs multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, updates the multi-level hash table and generates a traffic reassembly data packet; performs malicious traffic classification and detection on the traffic reassembly data packet, captures malicious code and generates corresponding traffic detection results. The technical solution of the embodiment of the present invention can maximize the polling efficiency and minimize the power consumption loss by adopting the polling sleep time determined adaptively to collect network traffic; divides the network traffic by combining the multiple hash strategy and the polling load strategy, can realize load balancing among multiple cores, and improve the utilization and performance of the multi-core processor; performs multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, can improve the performance and efficiency of traffic reassembly; and can support one-stop malicious traffic detection and malicious code capture, without the need to use honeypots for trapping malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0047] Embodiment 2

[0048] Figure 2 The flowchart of a method for IoT traffic collection and malicious traffic classification detection provided in the second embodiment of the present invention is further optimized and expanded based on the above implementation, and can be combined with various optional technical solutions in the above implementation. Figure 2 As shown, the second embodiment provides an IoT traffic collection and malicious traffic classification detection method, which specifically includes the following steps:

[0049] S210, based on the adaptively determined polling sleep duration, using a data plane development kit to obtain network traffic of the Internet of Things.

[0050] In the embodiment of the present invention, the flow collection process based on the DPDK framework is as follows: Figure 3 At the same time, the network topology structure of the technical solution of the embodiment of the present invention is as shown Figure 4As shown in the figure, it shows DPDK's detection of traffic information of multiple nodes. Among them, the autonomous system (AS) is a collection of one or more IP networks; the provider edge router (PE) is located at the edge of the operator network and connected to the customer network. It is mainly used to provide VPN (Virtual Private Network) services or MPLS (Multiprotocol Label Switching) tunnels; the autonomous system boundary router (ASBR) is located at the boundary between different ASs and is responsible for routing exchange and forwarding between ASs; the customer edge router (CE) is located at the edge of the customer network and connected to the operator network. It is used to transmit customer network traffic to the operator's network.

[0051] Further, based on the above-mentioned embodiment of the invention, the process of determining the polling sleep duration may include the following steps:

[0052] Step A1, calling a preset Kalman filter algorithm to predict the data packet rate of network traffic to obtain a predicted data packet rate;

[0053] Step A2, initializing the sliding window of the data plane development kit, and using the predicted rate of data packets in the sliding window, the total time required for polling, the number of data packets obtained during polling, the data packet processing time, the number of polling times, and the time required to obtain data packets from the network card queue during polling, to determine the average polling sleep time in the sliding window;

[0054] Step A3: Use a preset exponential smoothing method to adjust the average polling sleep time to obtain the polling sleep time.

[0055] Specifically, the network traffic packet rate data of the previous moment and the current moment in the network log record can be selected to establish the state space equation of the network packet rate; based on the obtained state space equation, the Kalman filter prediction and update process is performed, and the input data is used to obtain the state estimate and error covariance at the current moment; then, based on the obtained state estimate, the packet rate of the network traffic is predicted to obtain the predicted packet rate.

[0056] Furthermore, in order to adapt to different traffic environments, this embodiment adopts an adaptive intermittent polling method based on data packet rate prediction to implement intermittent polling with adaptive inter-packet sleep. The specific process is as follows:

[0057] 1) Calculate the total time T required for the i-th poll i:Assume the time length of the sliding window is T win , the predicted packet rate at time t is P t , the polling sleep time of the i-th polling is The total time required for the i-th poll is T i The calculation method is as follows: Among them, T rx Indicates the time required to obtain data packets from the network card queue during polling, n i Indicates the number of packets obtained during the i-th polling. It represents the time required to process the jth data packet. And it needs to satisfy the following formula: Among them, H represents the total number of polling in the sliding window, Δt j Indicates a unit of time.

[0058] 2) Calculate the average polling sleep time in the sliding window Let P in the sliding window t , T i , and n i Use the corresponding parameter average values ​​in the sliding window respectively and Instead, and assuming that the processing time of each data packet is consistent, that is, Then we have:

[0059]

[0060] The average polling sleep time in the sliding window is It can be determined as follows:

[0061]

[0062] 3) Use the preset exponential smoothing method to adjust the average polling sleep time: In order to make the polling cycle adapt to the changes in the arrival rate of data packets, the preset exponential smoothing method is finally used to adjust the polling sleep time, that is: Wherein, α is a smoothing coefficient. It should be understood that the length of the sliding window affects the accuracy and overhead of adjusting the polling sleep duration, and a trade-off needs to be made in practical applications to select an appropriate sliding window length.

[0063] S220. Call a pre-configured multiple hash strategy to perform hash calculation on data packets of network traffic to distribute the network traffic to corresponding CPU processing cores.

[0064] In the embodiment of the present invention, S220 specifically includes the following steps:

[0065] S2201. Use the Toeplitz hash algorithm to calculate the hash value of the data packet of the network traffic collected in S210, obtain the corresponding CPU core number, and record it in the key-core hash table; then query the corresponding hash value in the key-stream hash table in the CPU core according to the four-tuple of the data packet (source address SIP, destination address DIP, source port SPORT, destination port DPORT), so as to obtain a specific TCP session flow, and record it in the session flow hash table.

[0066] S2202: Use the hash finder algorithm corresponding to the session flow for the newly arrived data packet, and use the four-tuple of the data packet (source address SIP, destination address DIP, source port SPORT, destination port DPORT) to calculate: index =Four_tuple_hash(SIP,DIP,SPORT,DPORT).

[0067] S2203. Modify the classic hash algorithm and adjust the size of the hash space of its result to adapt to the node performance. Use the following hash function to determine the final generated hash result HRES:

[0068]

[0069] HRES=HKEY1&HKEY2

[0070] Among them, HKEY2 can be used to control the length of the output results.

[0071] S2204. After determining the hash function in S2203, modify the hash key used by RSS in DPDK to the corresponding value required by the S-RSS algorithm. When the data packet arrives at the network interface layer, DPDK will perform Toeplitz Hash calculation and distribute the network data packet to the rx queue corresponding to different capture cores according to different hash calculation results.

[0072] S2205. After the CPU core corresponding to S2204 obtains the data, it first searches the key-stream hash table in the corresponding data stream reassembly core in the current core, and determines whether the network flow corresponding to the current data packet already exists in the corresponding flow table. If the corresponding network flow can be found, the network flow data is added to the corresponding item in the flow table. If the corresponding network flow is not found in the corresponding flow table, the key-core hash table is queried to see whether the flow data has been distributed to other CPU cores for processing through the load balancing solution. If the network flow can be found in the key-core hash table, the flow data is delivered to the rte_ring circular queue of the corresponding CPU core. Otherwise, it indicates that the flow data corresponds to a new TCP connection for the traffic collection module and the reassembly module.

[0073] S2206. When processing a new TCP connection, calculate the overall load balance of the CPU. If the CPU load balance exceeds the threshold, remove the overloaded cores in the CPU core and recalculate the Toeplitz hash value. After the calculation is completed, add the data flow and the mapping relationship of the CPU core obtained according to the hash result to the key-core hash table, send the traffic data to the rte_ring of the corresponding CPU core, and add the data flow information to the key-stream hash table in the corresponding core.

[0074] S2207, evaluating the load balancing degree of the CPU: using the proposed session integrity-oriented load balancing evaluation index as the load balancing evaluation index. In a general multi-core processor, the processing capabilities of each core can be regarded as the same level. Therefore, the specific formula for the load balancing degree between each core of the multi-core processor at time t is as follows:

[0075]

[0076] Among them, q i (t) represents the utilization efficiency of the i-th CPU core at time t, and e is the number of cores of the CPU multi-core processor.

[0077] In this embodiment, the load balancing module will use pipeline mode to distribute different data packet processing tasks to different cores. When the CPU balance degree does not reach the threshold, that is, the load balancing module is not involved, the traffic data on each collection core will be distributed to two flow reassembly cores; when the CPU balance degree reaches the threshold, that is, the load balancing module is involved, the traffic on each collection core may be output to the CPU core where any flow reassembly core runs.

[0078] S230: Determine a flow load value corresponding to the network traffic according to the packet receiving rate, message length and final forwarding rate of the network traffic within a preset period.

[0079] In the embodiment of the present invention, the network traffic (flow) F is defined as i The flow load value for the processing core is V i , within the preset period t, F i The packet receiving rate is R i , the message length is L i , F i The final forwarding rate is Then F i The flow load value for the processing core is V i It can be calculated as follows:

[0080]

[0081] The final forwarding rate The following formula should be satisfied:

[0082]

[0083] Where D, k1, b1, k2, and b2 are constants related to processing power, and L is the packet length. D is the baseline value of the final forwarding rate. k1, b1, k2, and b2 describe the linear relationship between the final forwarding rate and the packet length.

[0084] S240. Based on the network traffic processed by each CPU processing core and the flow load value of the corresponding network traffic, determine the core load value of each CPU processing core, and store the core load value and the flow load value of each network traffic of the corresponding CPU processing core in the processing core flow information table corresponding to the CPU processing core.

[0085] In the embodiment of the present invention, the flow load value of each flow processed on the CPU processing core can be used to quantify the load of each CPU processing core, and the processing core flow information table (Processing Core Flow Information Table, PCFIT) is used to maintain the flow load value of each CPU processing core. The PCFIT table provides a basis for the calculation of the core load value and the subsequent load migration.

[0086] Assume that a set of processing cores in the system is AC = {C1, C2, ..., C j}, where C j represents any CPU processing core. Assume that the CPU processing core is responsible for processing m flows: AF = {F1, F2, ..., F m}, then the CPU processing core Cj The corresponding core load value ld j It is expressed as follows:

[0087] S250 , classifying the corresponding CPU processing cores according to the core load value of each CPU processing core and a preset heavy load threshold.

[0088] In the embodiment of the present invention, S250 specifically includes the following steps:

[0089] S2501, sorting all CPU processing cores in descending order according to corresponding core load values;

[0090] S2502, starting from the CPU processing core corresponding to the maximum core load value, determining whether the corresponding core load value is less than a preset heavy load threshold;

[0091] S2503: If yes, all CPU processing cores are determined as light-load processing cores;

[0092] S2504: If not, determine the corresponding CPU processing core as a heavy-load processing core, and continue to traverse the remaining CPU processing cores.

[0093] Specifically, assuming that all CPU processing cores have the same processing power, the preset heavy load threshold is defined as ld max , the classification process of CPU processing core is as follows:

[0094] Load all CPU processing cores according to the corresponding core load value ld j Sort in descending order; sort the CPU processing cores by traversing from the most heavily loaded CPU processing core: if the core load value satisfies ld j <ld max , it means that there is no heavy-load processing core in the current system, and the adaptive load balancing stage can be ended directly at this time; if the core load value satisfies ld j ≥ld max , then the CPU processing core is classified as a heavy load processing core, and continues to traverse other processing CPU processing cores; in the subsequent CPU processing cores, if the core load value meets ld j <ld max , the CPU processing core is classified as a light-load processing core; if there is no light-load processing core after traversing all CPU processing cores, it means that the current system is in a fully loaded state and cannot be optimized through adaptive load balancing. At this time, the adaptive load balancing process ends.

[0095] S260 , according to the core classification result and the processing core flow information table corresponding to each CPU processing core, load migration is performed on the heavy-load processing core in the CPU processing core, so as to obtain a load-balanced data flow.

[0096] In the embodiment of the present invention, S260 specifically includes the following steps:

[0097] S2601, when there are both heavy-load processing cores and light-load processing cores in the CPU processing cores, the heavy-load processing core corresponding to the largest core load value is used as the source core of the load migration, and the network data of the latest record in the flow information table of the processing core corresponding to the source core is used as the target migration flow;

[0098] S2602, sorting the light-load processing cores in ascending order according to the corresponding core load values, and after determining that the corresponding light-load processing core with the smallest core load value accepts the target migration flow, when the corresponding new core load value is less than a preset heavy-load threshold, determining the corresponding light-load processing core as the target light-load processing core, migrating the target migration flow to the target light-load processing core, and updating the core load values ​​at the same time;

[0099] S2603. Continue to migrate the load on the source core until the new core load value corresponding to the source core is less than the preset heavy load threshold, and select the next heavy load processing core as the new source core, and then re-execute the load migration process on the new source core until there are no heavy load processing cores in the system, or the target light load processing core cannot be found.

[0100] Specifically, when there are both heavy-load processing cores and light-load processing cores in the CPU processing core, the BBFF (Big Box First Fit) algorithm can be called and load migration operations can be performed based on the processing core classification results and the PCFIT table. Each CPU processing core corresponds to a PCFIT table. Whenever a new flow is assigned to a CPU processing core, the identification information of the flow will be added to the PCFIT table of the corresponding processing core.

[0101] During the load migration process, when the system selects the flow to be migrated from the heavy-load processing core with the largest core load value, first, the most recently recorded flow is selected as the target migration flow in the PCFIT table corresponding to the heavy-load processing core; then, the target light-load core is selected based on the BBFF algorithm: each light-load processing core is checked in order from small to large core load values ​​to ensure that if the light-load processing core accepts the target migration flow, its core load value will not exceed the preset heavy-load threshold; finally, each core load value is updated and the heavy-load processing core is rescheduled: when the flow migrates from one core to another, the core load value of the corresponding core will be updated. If the core load value of the heavy-load processing core is already lower than the preset heavy-load threshold, the next heavy-load processing core is selected as the source core for load migration. Repeat the above steps until there is no heavy-load processing core in the system or the target light-load processing core cannot be found.

[0102] S270 , calling a preset SIMD instruction set and a preset multi-packet aggregation technology, merging the measurement data of multiple data packets in the same data stream into the same update operation, and saving the generated traffic reassembled data packet as a PCAP file.

[0103] Among them, the preset SIMD (Single Instruction Multiple Data) instruction set is a parallel processing technology that allows multiple data elements to be processed in parallel in one instruction operation, thereby improving computing efficiency. The preset multi-packet aggregation technology can merge the measurement results of multiple network data packets belonging to the same flow into one update operation, reducing the number of accesses to the network measurement hash table structure and the probability of conflicts.

[0104] In the embodiment of the present invention, Figure 5 As shown, the data stream obtained in S260 can be used as input, and the parallelism and vectorization characteristics of the preset SIMD instruction set can be used to simultaneously process multiple collected network traffic data packets to improve processing efficiency; at the same time, the preset multi-packet aggregation technology is used to merge the measurement data of multiple data packets in the same stream (such as count, maximum value, minimum value, TCP stream sequence number, etc.) into one update operation, and the generated traffic reassembled data packet is saved as a PCAP file (a datagram storage format).

[0105] S280: Obtain a pre-configured multi-level hash table, and update the measurement data to the multi-level hash table according to an update operation.

[0106] In an embodiment of the present invention, a group of adjacent large prime numbers can be selected as the modulus of the hash operation result, and a group of mutually prime multi-level hash tables with a length of the group of large prime numbers can be established to store the results of network measurement. The process of updating the measurement data to the multi-level hash table is as follows: when processing a data packet, first obtain the hash result calculated by the network card, and take the first large prime number for modulus operation. If there is no data at the address in the multi-level hash table or the flow identifiers of the two are consistent, the measurement data is updated to the multi-level hash table; if there is data at the address in the multi-level hash table and the flow identifiers of the two are inconsistent, a hash conflict occurs, and the second large prime number is taken for modulus operation, and an attempt is made to put it into the hash table of the next level, and so on; if the location for putting it in is still not found in the multi-level hash table, find one of the records with the least amount of data and the longest last update time to evict and export.

[0107] S290. Call a preset traffic analysis tool to extract a TCP session flow from a PCAP file corresponding to the traffic reassembly data packet.

[0108] In the embodiment of the present invention, Figure 6 As shown, after the network traffic is collected, the malicious traffic detection analysis and malicious code capture and threat detection process for the APT family can be performed to realize the malicious traffic detection and malicious code capture in the network traffic. Specifically, the preset traffic analysis tools such as TShark can be called to filter and analyze the PCAP file corresponding to the traffic reassembly data packet, and non-TCP traffic, such as UDP traffic and ICMP (Internet Control Message Protocol) traffic, are removed, and TCP session streams are filtered out from the network traffic in batches, and each TCP session stream is saved separately as a PCAP file, and the file is named with a five-tuple. For example, if the destination IP address is 59.41.39.125, the destination port is 22, the source IP address is 131.202.240.8, and the source port is 63041, then the file name is TCP_59-41-39-125_63081_131-202-240-87_22.pcap. Each TCP session stream can be saved separately for subsequent analysis.

[0109] S2100, calling a preset malicious traffic classification model to determine whether the TCP session flow is a malicious TCP session, if so, extracting the destination IP address from the malicious TCP session, if not, terminating the subsequent malicious traffic detection process.

[0110] Among them, the preset malicious traffic classification model can be understood as a pre-configured model for detecting whether a TCP session flow is a malicious session flow. The preset malicious traffic classification model can include at least one of the following: K-Nearest Neighbor (KNN), logistic regression (LR), linear discriminant analysis (LDA), linear support vector machine (LSVM), decision tree (DT) and random forest (RF).

[0111] In an embodiment of the present invention, a complete TCP session flow includes three parts: connection establishment flow, data transmission flow and connection termination flow. The TCP session flow can be segmented by identifying the identifiers of "establishing a connection" and "terminating a connection". Then, TCP session features (stream-level features) and request header field features (data packet header-level features) are extracted for each TCP session flow, wherein the request header field features include data packet length features, arrival time features, etc. They are fused to form multimodal features, and then uniformly vectorized encoding is performed using a self-organizing map neural network (SOM) and other methods to convert the multimodal features into session feature vectors. Finally, the obtained session feature vector is input into a pre-trained preset malicious traffic classification model for classification, so as to determine whether the TCP session flow is a malicious TCP session. If so, the corresponding destination IP address is extracted from the malicious TCP session. If not, the subsequent malicious traffic detection process is terminated.

[0112] S2110. Based on the matching of the destination IP address and the preset malicious IP address database, the TCP session flow is screened, and the corresponding TCP session flow whose destination IP address is the malicious IP address is retained.

[0113] The preset malicious IP address database may refer to a database pre-configured locally and storing a number of known malicious IP addresses.

[0114] In an embodiment of the present invention, it is possible to determine whether the destination IP address extracted from the TCP session flow is a private address, for example, the destination IP address and the source IP address in the above step S290 are not within the range of the three types of private addresses, that is, it is not a private address; if it is a private address, the detection process is directly terminated and the result is output; if it is not a private address, the API (Application Programming Interface, API) provided by OTX (Open Threat Exchange, open source threat intelligence sharing platform) is used to detect the destination IP address. OTX will maintain the pulse of the IP address for the user, and according to the return result of OTX, it is determined whether the number of pulses associated with the destination IP address is greater than 1. Taking IP addresses 131.202.240.8 and 59.41.39.125 as examples, the number of pulses in the OTX database is queried respectively, and it is determined whether it is a malicious address according to the return result. If the number of pulses corresponding to the destination IP address is greater than 1, the destination IP address is considered to be a malicious IP address, and its corresponding TCP session flow is retained. Otherwise, the address is considered to be a benign address and the TCP session flow is discarded.

[0115] At the same time, a preset malicious IP address database can be built locally to screen each TCP session flow according to the destination IP address. The TCP session flow with a malicious IP address as the destination IP address will be retained, otherwise it will be discarded.

[0116] S2120. Call a preset Wireshark tool to capture the malicious code in the TCP session stream, and obtain a behavioral feature sequence when the malicious code runs in the sandbox.

[0117] In an embodiment of the present invention, a preset Wireshark tool can be called according to the application layer protocol to capture malicious code in a TCP session stream. In order to obtain the behavioral feature sequence of the malicious code, the malicious code can be run in a sandbox, its dynamic behavior can be monitored, and system calls, file operations, network communications, process creation and other activities during the code running can be recorded. By extracting unit behavioral feature sequences under several continuous time slices, a multi-sequence description and modeling of malicious behaviors of the APT family can be achieved. Specifically, the APT malware behavior is modeled through a five-tuple of software name, software operating system environment, malicious behavior set, object and path of software execution action, and the five-tuple data of behavioral features under continuous time slices together constitute a set of behavioral feature sequences. An independent behavioral feature sequence is constructed for each malicious behavior and optimized, redundant feature fields are deleted, and the software operating environment field is replaced with simple special characters. For example, a suspected malicious code appears in the first-in-first-out sequence of the API call structure in the sandbox: (23, Virtual AllocEx) → (23, WriteProcess Memory) → … → (23, CreateRemoteThread), then the corresponding abstract behavior structure (i.e., behavior feature sequence) is: (23, system | process, create, inject | remotethread).

[0118] S2130: Input the behavior feature sequence into a preset support vector machine model to obtain a corresponding attack behavior score, and when the attack behavior score is greater than a preset score threshold, determine the TCP session flow corresponding to the behavior feature sequence as an APT virus sample.

[0119] In an embodiment of the present invention, the generated behavior feature sequence can be input into a pre-trained preset support vector machine (SVM) model, the behavior feature sequence can be expressed in the form of a first-order matrix, the frequency of occurrence of the target sample (TCP session flow) in multiple dimensions of the first-order matrix is ​​counted, and the attack behavior weight difference is assigned; then, the preset SVM model is used to score in multiple dimensions, and the TCP session flow is judged whether it is an APT virus sample based on the generated attack behavior score, that is, when the attack behavior score is greater than the preset score threshold, the TCP session flow corresponding to the behavior feature sequence is determined as an APT virus sample, thereby realizing the detection of threat traffic in the Internet of Things network traffic.

[0120] Further, based on the above-mentioned embodiments of the invention, the construction process of the preset support vector machine model may include the following steps:

[0121] Step B1, extracting a preset number of behavioral feature sequence samples of APT family samples when running in the sandbox;

[0122] Step B2: calling a preset Blast algorithm to perform similarity detection on each behavior feature sequence sample, and taking the part of all behavior feature sequence samples whose similarity is greater than a preset threshold as a common feature, and building a malicious behavior feature library based on the common features;

[0123] Step B3: Use the malicious behavior feature library to train the initialized support vector machine model to obtain a preset support vector machine model.

[0124] Specifically, a preset number of APT family samples can be obtained as training data, and the behavioral feature sequence samples of the above APT family samples when running in the sandbox can be extracted; then the preset Blast algorithm is called to perform similarity detection on each behavioral feature sequence sample, and the part of all behavioral feature sequence samples with a similarity greater than a preset threshold (for example, 90%) is extracted as the APT common feature, and then a malicious behavior feature library is constructed based on the acquired APT common features; finally, the SVM model is trained based on the data in the malicious behavior feature library, so as to obtain a preset support vector machine model for threat traffic detection. The embodiment of the present invention constructs a malicious behavior feature library based on the Blast algorithm, realizes multi-sequence description and modeling of malicious behaviors of the APT family, uses the Blast algorithm to realize the similarity comparison of behavioral feature sequences, extracts the optimal common features of the APT family, and uses the common features of the APT family to fully express the difference between malicious traffic and normal traffic in feature dimensions, providing a basis for subsequent threat traffic detection.

[0125] The technical solution of the embodiment of the present invention, through the proposed multi-core dynamic and static load balancing method based on multiple hashing and polling load, uses a multiple hashing algorithm to statically distribute network flows to each CPU core, and dynamically adjusts the probability of traffic distribution between multiple cores according to the load of the CPU core, so as to improve the utilization and performance of the multi-core processor. The multi-level hash table structure is used to quickly locate the flow to which the data packet belongs, and the aggregation update strategy is adopted. The combination of the two realizes a refined flow reorganization method based on a multi-level hash table, thereby reducing the number of hash table updates and the probability of conflict, and improving the performance and efficiency of flow reorganization. The parallelization and vectorization characteristics of the SIMD instruction set are used to process multiple data packets at the same time, thereby improving the processing efficiency. The multi-packet aggregation technology is used to merge the measurement results of multiple network data packets belonging to the same flow into one update operation, reducing the number of accesses and the probability of conflict to the network measurement hash table structure, accelerating the efficiency of flow reorganization while ensuring the accuracy of network measurement. In order to solve the problem that it is difficult to accurately extract threat features from complex and large data, the embodiment of the present invention first implements refined classification of malicious traffic based on multi-dimensional feature fusion, captures malicious code in malicious traffic, and extracts behavioral features, and then uses a preset support vector machine model to detect threat traffic in the Internet of Things. It can support one-stop malicious traffic detection and malicious code capture, without the need to use honeypots for trapping malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0126] Embodiment 3

[0127] Figure 7 This is a schematic diagram of the structure of an IoT traffic collection and malicious traffic classification detection device provided by the third embodiment of the present invention. Figure 7 As shown, the device comprises:

[0128] A traffic acquisition module 31 is used to acquire network traffic of the Internet of Things using a data plane development kit based on the adaptively determined polling sleep duration;

[0129] A load balancing module 32, used to distribute network traffic based on a multiple hash strategy and a polling load strategy to obtain a load-balanced data flow;

[0130] A traffic reassembly module 33, used to perform multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, update the multi-level hash table and generate a traffic reassembly data packet;

[0131] The malicious traffic detection module 34 is used to classify and detect malicious traffic on traffic reassembly data packets, capture malicious codes and generate corresponding traffic detection results.

[0132] The technical solution of the embodiment of the present invention uses a data plane development kit to obtain the network traffic of the Internet of Things through a traffic acquisition module based on an adaptively determined polling sleep time; the load balancing module performs multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, updates the multi-level hash table and generates a traffic reassembly data packet; the malicious traffic detection module performs malicious traffic classification and detection on the traffic reassembly data packet, captures malicious code and generates corresponding traffic detection results. The technical solution of the embodiment of the present invention can maximize the polling efficiency and minimize the power consumption loss by adopting an adaptively determined polling sleep time to collect network traffic; by combining the multi-hash strategy and the polling load strategy to shunt the network traffic, load balancing can be achieved among multiple cores, and the utilization and performance of the multi-core processor can be improved; by adopting the aggregation update strategy to perform multi-packet aggregation processing on the measurement data of the data flow, the performance and efficiency of the traffic reassembly can be improved; at the same time, it can support one-stop malicious traffic detection and malicious code capture, without the need to use a honeypot for trapping malicious code capture, thereby improving the efficiency and accuracy of traffic detection.

[0133] Further, based on the above-mentioned embodiment of the invention, the process of determining the polling sleep duration includes:

[0134] Calling a preset Kalman filter algorithm to predict the packet rate of network traffic to obtain a predicted packet rate;

[0135] Initialize the sliding window of the data plane development kit, and determine the average polling sleep time in the sliding window using the predicted rate of packets in the sliding window, the total time required for polling, the number of packets obtained during polling, the packet processing time, the number of polling times, and the time required to obtain packets from the NIC queue during polling;

[0136] The average polling sleep time is adjusted by using a preset exponential smoothing method to obtain the polling sleep time.

[0137] Further, based on the above-mentioned embodiment of the invention, the load balancing module 32 includes:

[0138] A multi-hash calculation unit, used to call a pre-configured multi-hash strategy to perform hash calculation on data packets of network traffic, so as to distribute the network traffic to corresponding CPU processing cores;

[0139] A flow load value determination unit, used to determine the flow load value corresponding to the network flow according to the packet receiving rate, message length and final forwarding rate of the network flow within a preset period;

[0140] a core load value determining unit, configured to determine a core load value of each CPU processing core based on each network flow processed by each CPU processing core and a flow load value of the corresponding network flow, and store the core load value and the flow load value of each network flow of the corresponding CPU processing core in a processing core flow information table corresponding to the CPU processing core;

[0141] A core classification unit, for classifying corresponding CPU processing cores according to a core load value of each CPU processing core and a preset heavy load threshold;

[0142] The load migration unit is used to perform load migration on the heavy-load processing cores in the CPU processing cores according to the core classification results and the processing core flow information table corresponding to each CPU processing core, so as to obtain a load-balanced data flow.

[0143] Further, based on the above-mentioned embodiments of the invention, the core classification unit is specifically used for:

[0144] Sort all CPU processing cores in descending order according to their corresponding core load values;

[0145] Start traversing from the corresponding CPU processing core with the maximum core load value, and determine whether the corresponding core load value is less than a preset heavy load threshold;

[0146] If yes, all CPU processing cores are determined as lightly loaded processing cores;

[0147] If not, the corresponding CPU processing core is determined as a heavy-load processing core, and the remaining CPU processing cores are traversed.

[0148] Further, based on the above-mentioned embodiment of the invention, the load migration unit is specifically used for:

[0149] When there are both heavy-load processing cores and light-load processing cores in the CPU processing cores, the corresponding heavy-load processing core with the largest core load value is used as the source core of the load migration, and the network data of the latest record in the flow information table of the processing core corresponding to the source core is used as the target migration flow;

[0150] Sort the light-load processing cores in ascending order according to the corresponding core load values, and after determining that the corresponding light-load processing core with the smallest core load value accepts the target migration flow, when the corresponding new core load value is less than a preset heavy-load threshold, determine the corresponding light-load processing core as the target light-load processing core, migrate the target migration flow to the target light-load processing core, and update the load values ​​of each core at the same time;

[0151] Continue to migrate the load on the source core until the new core load value corresponding to the source core is less than the preset heavy load threshold, and select the next heavy load processing core as the new source core, and then re-execute the load migration process on the new source core until there are no heavy load processing cores in the system or the target light load processing core cannot be found.

[0152] Further, based on the above-mentioned embodiment of the invention, the traffic reassembly module 33 includes:

[0153] A multi-packet aggregation unit, used for calling a preset SIMD instruction set and a preset multi-packet aggregation technology, merging measurement data of multiple data packets in the same data stream into a same update operation, and saving the generated traffic reassembled data packets as a PCAP file;

[0154] The measurement data updating unit is used to obtain a pre-configured multi-level hash table and update the measurement data to the multi-level hash table according to an update operation.

[0155] Further, based on the above-mentioned embodiment of the invention, the malicious traffic detection module 34 includes:

[0156] The session flow extraction unit is used to call a preset traffic analysis tool to extract the TCP session flow from the PCAP file corresponding to the traffic reassembly data packet;

[0157] A malicious session detection unit, used to call a preset malicious traffic classification model to determine whether a TCP session flow is a malicious TCP session, if so, extract a destination IP address from the malicious TCP session, if not, terminate a subsequent malicious traffic detection process;

[0158] A session flow screening unit, used to screen the TCP session flow based on the matching of the destination IP address with the preset malicious IP address database, and retain the corresponding TCP session flow whose destination IP address is the malicious IP address;

[0159] A behavior feature sequence acquisition unit is used to call a preset Wireshark tool to capture malicious code in a TCP session flow and acquire a behavior feature sequence when the malicious code is running in a sandbox;

[0160] The APT virus detection unit is used to input the behavior feature sequence into a preset support vector machine model to obtain a corresponding attack behavior score, and when the attack behavior score is greater than a preset score threshold, determine the TCP session flow corresponding to the behavior feature sequence as an APT virus sample.

[0161] Furthermore, based on the above-mentioned embodiment of the invention, the construction process of the support vector machine model is preset, including:

[0162] Extract a preset number of behavioral feature sequence samples of APT family samples when running in the sandbox;

[0163] Call the preset Blast algorithm to perform similarity detection on each behavior feature sequence sample, and take the part of all behavior feature sequence samples with similarity greater than the preset threshold as the common feature, and build a malicious behavior feature library based on the common features;

[0164] The malicious behavior feature library is used to train the initially constructed support vector machine model to obtain a preset support vector machine model.

[0165] The Internet of Things traffic collection and malicious traffic classification detection device provided in the embodiment of the present invention can execute the Internet of Things traffic collection and malicious traffic classification detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0166] Embodiment 4

[0167] Figure 8 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0168] like Figure 8 As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0169] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0170] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the IoT traffic collection and malicious traffic classification detection method.

[0171] In some embodiments, the IoT traffic collection and malicious traffic classification detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the IoT traffic collection and malicious traffic classification detection method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to perform the IoT traffic collection and malicious traffic classification detection method in any other appropriate manner (e.g., by means of firmware).

[0172] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0173] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0174] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0175] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0176] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0177] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0178] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0179] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for collecting traffic and classifying malicious traffic in the Internet of Things, characterized in that: The method comprises: Based on the adaptively determined polling sleep duration, the data plane development kit is used to obtain the network traffic of the Internet of Things; Diverting the network traffic based on a multiple hash strategy and a polling load strategy to obtain a load-balanced data flow; Performing multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, updating the multi-level hash table and generating a traffic reassembly data packet; Malicious traffic is classified and detected on the traffic reassembled data packets, malicious code is captured and corresponding traffic detection results are generated.

2. The method according to claim 1, characterized in that The process of determining the polling sleep duration includes: Calling a preset Kalman filter algorithm to predict the data packet rate of the network traffic to obtain a predicted data packet rate; Initialize the sliding window of the data plane development kit, and determine the average polling sleep duration in the sliding window by using the predicted rate of the data packets in the sliding window, the total time required for polling, the number of data packets obtained during polling, the data packet processing time, the number of polling times, and the time required for obtaining data packets from the network card queue during polling; The average polling sleep time is adjusted by using a preset exponential smoothing method to obtain the polling sleep time.

3. The method according to claim 1, characterized in that The method of dividing the network traffic based on the multiple hash strategy and the polling load strategy to obtain a load-balanced data flow includes: Calling the pre-configured multiple hash strategies to perform hash calculations on data packets of the network traffic, so as to distribute the network traffic to corresponding CPU processing cores; Determine the flow load value corresponding to the network traffic according to the packet receiving rate, message length and final forwarding rate of the network traffic within a preset period; Determine a core load value of each CPU processing core based on each of the network flows processed by each of the CPU processing cores and a flow load value corresponding to the network flows, and store the core load value and the flow load value of each of the network flows corresponding to the CPU processing core in a processing core flow information table corresponding to the CPU processing core; Classifying the corresponding CPU processing cores according to the core load value of each CPU processing core and a preset heavy load threshold; According to the core classification result corresponding to each of the CPU processing cores and the processing core flow information table, load migration is performed on the heavy-load processing cores in the CPU processing cores, thereby obtaining the data flow after load balancing.

4. The method according to claim 3, characterized in that The classifying the corresponding CPU processing cores according to the core load value of each CPU processing core and the preset heavy load threshold comprises: Sorting all the CPU processing cores in descending order according to the corresponding core load values; Starting from the CPU processing core corresponding to the maximum core load value, determining whether the corresponding core load value is less than the preset heavy load threshold; If yes, all the CPU processing cores are determined as light-load processing cores; If not, the corresponding CPU processing core is determined as a heavy-load processing core, and the remaining CPU processing cores are traversed.

5. The method according to claim 3, characterized in that: The step of performing load migration on a heavy-load processing core in the CPU processing core according to the core classification result corresponding to each of the CPU processing cores and the processing core flow information table includes: When there are both heavy-load processing cores and light-load processing cores in the CPU processing core, the heavy-load processing core corresponding to the largest core load value is used as the source core of the load migration, and the network data of the latest record in the processing core flow information table corresponding to the source core is used as the target migration flow; Sort the light-load processing cores in ascending order according to the corresponding core load values, and after determining that the light-load processing core with the smallest core load value accepts the target migration flow, when the corresponding new core load value is less than the preset heavy-load threshold, determine the corresponding light-load processing core as the target light-load processing core, migrate the target migration flow to the target light-load processing core, and update the core load values ​​of each core at the same time; Continue to perform load migration on the source core until the new core load value corresponding to the source core is less than the preset heavy load threshold, and select the next heavy load processing core as the new source core, and then re-execute the load migration process on the new source core until there is no heavy load processing core in the system, or the target light load processing core cannot be found.

6. The method according to claim 1, characterized in that The method of performing multi-packet aggregation processing on the measurement data of the data flow based on the aggregation update strategy, updating the multi-level hash table and generating a traffic reassembly data packet includes: Calling a preset SIMD instruction set and a preset multi-packet aggregation technology to merge the measurement data of multiple data packets in the same data stream into a same update operation, and saving the generated traffic reassembly data packet as a PCAP file; The pre-configured multi-level hash table is obtained, and the measurement data is updated to the multi-level hash table according to the update operation.

7. The method according to claim 1, characterized in that The malicious traffic classification and detection of the traffic reassembly data packet, capturing malicious code and generating corresponding traffic detection results include: Calling a preset traffic analysis tool to extract a TCP session flow from a PCAP file corresponding to the traffic reassembly data packet; Calling a preset malicious traffic classification model to determine whether the TCP session flow is a malicious TCP session, if so, extracting a destination IP address from the malicious TCP session, if not, terminating a subsequent malicious traffic detection process; Based on the matching of the destination IP address and the preset malicious IP address database, the TCP session flow is screened, and the TCP session flow corresponding to the malicious IP address is retained; Calling a preset Wireshark tool to capture the malicious code in the TCP session stream, and obtaining a behavioral feature sequence when the malicious code is running in the sandbox; The behavior feature sequence is input into a preset support vector machine model to obtain a corresponding attack behavior score, and when the attack behavior score is greater than a preset score threshold, the TCP session flow corresponding to the behavior feature sequence is determined as an APT virus sample.

8. The method according to claim 7, characterized in that The construction process of the preset support vector machine model includes: Extract a preset number of behavioral feature sequence samples of APT family samples when running in the sandbox; Calling a preset Blast algorithm to perform similarity detection on each of the behavior feature sequence samples, and taking the parts of all the behavior feature sequence samples whose similarity is greater than a preset threshold as common features, and building a malicious behavior feature library based on the common features; The malicious behavior feature library is used to train the initially constructed support vector machine model to obtain the preset support vector machine model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the Internet of Things traffic collection and malicious traffic classification detection method described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the Internet of Things traffic collection and malicious traffic classification detection method described in any one of claims 1 to 8 when executed.

Images