An embedded device business identification system

By combining deep packet detection technology and machine learning methods, the existing network service identification problems are solved, and high-accuracy and high-performance service identification is achieved, and network security and service quality assurance are enhanced.

CN115484087BActive Publication Date: 2024-11-05NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211089830.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-07
Publication Date
2024-11-05
Estimated Expiration
2042-09-07

AI Technical Summary

Technical Problem

The existing network service identification methods have problems of low accuracy and poor real-time performance, which cannot meet the technical requirements of high-performance systems. Especially in the face of complex network traffic and frequent network attacks, it is difficult to effectively identify and control user behavior and network security. .

Method used

Deep packet detection technology is combined with machine learning methods, through the collaborative work of edge-side devices and upper-layer devices, network traffic is collected and processed in real time, and deep packet detection and machine learning flow feature detection complement each other, improving the accuracy of business recognition and Real-time.

Benefits of technology

It effectively improves the accuracy and real-time nature of service identification, reduces system burden and storage needs, enhances network security and service quality assurance, and can more accurately identify network traffic types and quickly respond to potential threats.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115484087B_ABST
    Figure CN115484087B_ABST
Patent Text Reader

Abstract

The present invention discloses an embedded device business identification system, which has the advantage of being lightweight, can realize comprehensive analysis of network traffic, and perform real-time identification of network protocols, and then determine the business type, while ensuring real-time performance and maintaining a high accuracy rate. At the same time, in terms of network traffic collection, a diversion design is adopted to adapt to the high time requirements of real-time network traffic reception and reduce the CPU occupancy rate. The system not only reduces the impact of business identification on system functions, but also improves accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of mobile communication technology, and in particular to an embedded device service identification system. Background Art

[0002] Network traffic identification is one of the main means of analyzing network transmission content. Existing network traffic identification methods usually use port detection or feature detection memory information entropy detection. In recent years, with the increasing number of network applications, users' online behavior has become complex and changeable; at the same time, network attacks have become more frequent, and attack methods have emerged in an endless stream, which has brought great hidden dangers to network security. Network service identification can well control user online behavior and resist network attacks to a certain extent. At the same time, service identification is an indispensable part of network intrusion detection. Intrusion detection, as a barrier to defend against network attacks and improve network security, is receiving attention; service identification can provide a basis for intrusion detection, judge the type of service before intrusion detection, ban illegal services, and release normal services. Finally, service identification plays an important role in network capacity planning and communication service quality assurance. By analyzing the types of services in the network, network providers can determine the order of magnitude of service bandwidth. In addition, different services have different QOS requirements, and the network's service capabilities are limited. In order to ensure the service quality of various services, it is necessary to design certain scheduling algorithms and apply service identification to the system's QOS guarantee, so as to promptly discover different service types and provide different QOS according to their service types.

[0003] In addition to playing an important role in network capacity planning and intrusion detection, service identification also plays an important role in helping network operators identify abnormal user behavior. In order to protect customers from internal or external threats, operators must track anomalies that occur on the network. In the past decade, the number of vulnerabilities and attacks on the Internet, whether targeting personal security or national security, has gradually increased. In particular, in recent years, the country has vigorously developed the Internet+ model, making the Internet more closely connected with national infrastructure (such as electricity, gas, water, waste disposal, railways, etc.), which has led to a more complex composition of Internet traffic. Once attacked, the consequences will be immeasurable.

[0004] Most of the current business identification methods have the problems of low accuracy and poor real-time performance, and cannot meet the technical requirements of some high-performance systems. Summary of the invention

[0005] The purpose of the present invention is to provide an embedded device service identification system, which integrates deep packet inspection technology with machine learning methods to improve the accuracy and real-time performance of embedded device service identification, and provide highly reliable and high-performance service support for the functional implementation of subsequent systems.

[0006] In order to realize the above functions, the present invention designs an embedded device service identification system, wherein the target communication area includes edge side devices and upper layer devices, wherein the edge side devices include embedded devices, and the embedded devices include at least one gateway node, each gateway node includes a network card, a memory, a service flow collection module, and a service flow processing module, each service flow processing module is respectively connected to the upper layer device for communication, and the upper layer device includes a service flow identification module, and for the network flow to be identified, the service type of the network flow is identified, including the following steps:

[0007] S1: for each service flow collection module, the service flow collection module collects network flow in real time, and based on the preset time interval, saves the network flow collected in each preset time interval as a network flow packet, wherein the network flow packet is in the form of a pcap offline file, and stores each collected network flow packet in the memory of the gateway node;

[0008] S2: The service flow processing module converts each network flow packet into five-tuple information corresponding to each network flow packet, and constructs a network flow storage queue for storing all network flow packets to be identified;

[0009] S3: The service flow identification module obtains the network flow packets to be identified in the network flow storage queue, identifies the service type of each network flow packet to be identified in the network flow storage queue based on the deep packet inspection technology and the machine learning flow feature detection method, and outputs the service type identification result of each network flow packet to be identified;

[0010] S4: Repeat steps S2 to S3 until the business types of all the network traffic packets to be identified in the network flow storage queue are identified.

[0011] As a preferred technical solution of the present invention: step S1 includes the following steps S11-S12:

[0012] S11: For each gateway node, the service flow collection module uses the tcpdump program that comes with Linux to continuously monitor the network flow of the network card, and at every preset time interval, saves the network flow monitored within the preset time interval as a network flow packet, wherein the network flow packet is in the form of a pcap offline file;

[0013] S12: The network card includes a network card interface controller. The network card interface controller calls the lipcap library to copy each network traffic packet of the network card based on a bypass mechanism. The copied network traffic packets of the network card are subjected to kernel filtering and transmitted to an upper-layer device.

[0014] As a preferred technical solution of the present invention: step S2 includes the following steps S21-S25:

[0015] S21: Initialize the memory of the gateway node;

[0016] S22: The service traffic processing module unpacks each network traffic packet collected by the service traffic collection module, and analyzes and obtains the five-tuples corresponding to each network traffic packet, wherein each five-tuple includes the source IP address, destination IP address, source port, destination port, and packet length of the network traffic packet;

[0017] S23: Calculate the hash value of the five-tuple of each network traffic packet;

[0018] S24: construct a network flow storage queue, and store the hash value of each network flow packet and its five-tuple into the network flow storage queue;

[0019] S25: Repeat steps S22-S24, wherein the business traffic processing module determines whether the hash value of the five-tuple of each network traffic packet collected by the business traffic collection module has been stored in the network flow storage queue; if it has been stored in the network flow storage queue, the network traffic packet is discarded; if it has not been stored in the network flow storage queue, the hash value of each network traffic packet and its five-tuple is stored in the network flow storage queue.

[0020] As a preferred technical solution of the present invention: step S3 includes the following steps S31-S34:

[0021] S31: The service flow identification module obtains each network flow packet in the network flow storage queue;

[0022] S32: Based on deep packet inspection technology, identify the service category of each network traffic packet in the network flow storage queue;

[0023] S33: If the service category of the network traffic packet can be identified based on the deep packet inspection technology, then the service type identification result of the network traffic packet is output; if the service category of the network traffic packet cannot be identified based on the deep packet inspection technology, then the service category of the network traffic packet is identified based on the machine learning flow feature detection method, and the service type identification result of the network traffic packet is output;

[0024] S34: Repeat steps S31 to S33 until the service categories of all network traffic packets in the network flow storage queue are identified.

[0025] As a preferred technical solution of the present invention: the preset time interval is 5 seconds.

[0026] As a preferred technical solution of the present invention: in the process of identifying the business category of each network traffic packet based on deep packet inspection technology, if the deep packet inspection technology fails to identify the business category of each network traffic packet for three consecutive times, then the machine learning flow feature detection method is preferentially used to identify the business category of the network traffic packet. After 5 seconds of identification, the business category identification method is changed to deep packet inspection technology again.

[0027] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0028] (1) The present invention uses a network card interface controller to copy and forward captured network traffic packets, which puts less burden on the system and does not affect the normal operation of the system.

[0029] (2) The present invention effectively improves the accuracy and real-time performance of business identification by combining deep packet inspection technology with machine learning.

[0030] (3) The present invention effectively reduces the traffic that needs to be cached by calculating hash values ​​and queue storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a system diagram of an embedded device service identification system provided according to an embodiment of the present invention;

[0032] Figure 2 is a flow chart of embedded device service identification provided by an embodiment of the present invention;

[0033] Figure 3 is a flow chart of a traffic collection module provided according to an embodiment of the present invention;

[0034] Figure 4 is a flow chart of calculating a hash value of a network traffic packet according to an embodiment of the present invention;

[0035] Figure 5 It is a flowchart of a network traffic packet storage process provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0037] The present invention provides an embedded device service identification system, which is mainly based on deep packet inspection technology (DPI identification) and supplemented by a machine learning flow feature detection method. For the input network traffic packet, after deep packet inspection, if the service type is successfully identified, the service type protocol number is directly returned; if the service type cannot be identified after deep packet inspection, the network traffic packet is used as a cache to prepare for subsequent machine learning flow feature detection. The two identification methods are connected in series. This design method combines the advantages of deep packet inspection technology in real-time and accurate identification of non-encrypted conventional protocols, and the characteristics of machine learning flow feature detection methods that can identify encrypted protocols, to supplement the network traffic packets that are not identified by deep packet inspection. Therefore, compared with the identification method using the machine learning flow feature detection method alone, the number of network traffic packets that need to be cached is reduced, the corresponding storage memory is also reduced, and the complexity of model training is reduced.

[0038] Reference Figure 1 , Figure 2 An embedded device service identification system provided by an embodiment of the present invention includes an edge device and an upper layer device in a target communication area, wherein the edge device includes an embedded device, and the embedded device includes at least one gateway node, each gateway node includes a network card, a memory, a service flow collection module, and a service flow processing module, each service flow processing module is respectively connected to the upper layer device for communication, and the upper layer device includes a service flow identification module, and for the network flow to be identified, the service type of the network flow is identified, including the following steps:

[0039] S1: for each service flow collection module, the service flow collection module collects network flow in real time, and based on the preset time interval, saves the network flow collected in each preset time interval as a network flow packet, wherein the network flow packet is in the form of a pcap offline file, and stores each collected network flow packet in the memory of the gateway node;

[0040] In one embodiment, the preset time interval is 5 seconds.

[0041] Among them, reference Figure 3 , step S1 includes the following steps S11-S12:

[0042] S11: For each gateway node, the service flow collection module uses the tcpdump program that comes with Linux to continuously monitor the network flow of the network card, and at every preset time interval, saves the network flow monitored within the preset time interval as a network flow packet, wherein the network flow packet is in the form of a pcap offline file;

[0043] S12: The network card includes a network card interface controller (NIC). The network card interface controller calls the lipcap library and copies each network traffic packet of the network card based on the bypass mechanism. The original network traffic packet will be parsed and forwarded without interference. For each network traffic packet of the copied network card, it will be filtered by the kernel and transmitted to the upper-layer device.

[0044] S2: The service flow processing module converts each network flow packet into five-tuple information corresponding to each network flow packet, and constructs a network flow storage queue for storing all network flow packets to be identified;

[0045] refer to Figure 4 , step S2 includes the following steps S21-S25:

[0046] S21: Initialize the memory of the gateway node;

[0047] S22: The service traffic processing module unpacks each network traffic packet collected by the service traffic collection module, and analyzes and obtains the five-tuples corresponding to each network traffic packet, wherein each five-tuple includes the source IP address, destination IP address, source port, destination port, and packet length of the network traffic packet;

[0048] S23: Calculate the hash value of the five-tuple of each network traffic packet;

[0049] S24: construct a network flow storage queue, and store the hash value of each network flow packet and its five-tuple into the network flow storage queue;

[0050] S25: Repeat steps S22-S24, wherein the business traffic processing module determines whether the hash value of the five-tuple of each network traffic packet collected by the business traffic collection module has been stored in the network flow storage queue; if it has been stored in the network flow storage queue, the network traffic packet is discarded; if it has not been stored in the network flow storage queue, the hash value of each network traffic packet and its five-tuple is stored in the network flow storage queue.

[0051] S3: Reference Figure 5 The service traffic identification module obtains the network traffic packets to be identified in the network flow storage queue, identifies the service type of each network traffic packet to be identified in the network flow storage queue based on the deep packet inspection technology and the machine learning flow feature detection method, and outputs the service type identification result of each network traffic packet to be identified;

[0052] Step S3 includes the following steps S31 to S34:

[0053] S31: The service flow identification module obtains each network flow packet in the network flow storage queue;

[0054] S32: Based on deep packet inspection technology, identify the service category of each network traffic packet in the network flow storage queue;

[0055] S33: If the service category of the network traffic packet can be identified based on the deep packet inspection technology, then the service type identification result of the network traffic packet is output; if the service category of the network traffic packet cannot be identified based on the deep packet inspection technology, then the service category of the network traffic packet is identified based on the machine learning flow feature detection method, and the service type identification result of the network traffic packet is output;

[0056] S34: Machine learning flow feature detection technology identifies network traffic packets, using a process of training first and then identifying. Pre-train some known business types in advance and save the characteristics of each business type; then use machine learning to determine which business type each network traffic packet belongs to; after successful identification, the identification result will be returned and enter the next identification process;

[0057] Repeat steps S31 to S33 until the service categories of all network traffic packets in the network flow storage queue are identified.

[0058] S4: Repeat steps S2 to S3 until the business types of all the network traffic packets to be identified in the network flow storage queue are identified.

[0059] The present invention combines deep packet inspection technology and machine learning flow feature detection method. Deep packet inspection technology has the characteristics of fast detection speed and light weight, but it cannot detect encrypted traffic. Although the machine learning flow feature detection method has a relatively high accuracy, it consumes a lot of resources and is time-consuming. Therefore, the two are combined, and deep packet inspection technology is used for identification first. If the identification fails, the machine learning flow feature detection method is used. Considering the periodic characteristics of user Internet access, an embedded device service identification system provided by an embodiment of the present invention is a process for identifying the service category of each network traffic packet based on deep packet inspection technology. If the deep packet inspection technology fails to identify the service category of each network traffic packet for three consecutive times, the machine learning flow feature detection method is preferentially used to identify the service category of the network traffic packet. After 5 seconds of identification, the service category identification method is changed to deep packet inspection technology again.

[0060] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. An embedded device business identification system, It is characterized in that The target communication area includes edge devices and upper-layer devices, wherein the edge devices include embedded devices, the embedded devices include at least one gateway node, each gateway node includes a network card, a memory, a service flow collection module, and a service flow processing module, each service flow processing module is respectively connected to the upper-layer device for communication, and the upper-layer device includes a service flow identification module, which implements service type identification of network flow for the network flow to be identified, including the following steps: S1: for each service flow collection module, the service flow collection module collects network flow in real time, and based on the preset time interval, saves the network flow collected in each preset time interval as a network flow packet, wherein the network flow packet is in the form of a pcap offline file, and stores each collected network flow packet in the memory of the gateway node; Step S1 includes the following steps S11 to S12: S11: For each gateway node, the service flow collection module uses the tcpdump program that comes with Linux to continuously monitor the network flow of the network card, and at every preset time interval, saves the network flow monitored within the preset time interval as a network flow packet, wherein the network flow packet is in the form of a pcap offline file; S12: The network card includes a network card interface controller, which calls a lipcap library to copy each network traffic packet of the network card based on a bypass mechanism, performs kernel filtering on each network traffic packet of the copied network card, and transmits the packet to an upper layer device; S2: The service flow processing module converts each network flow packet into five-tuple information corresponding to each network flow packet, and constructs a network flow storage queue for storing all network flow packets to be identified; Step S2 includes the following steps S21 to S25: S21: Initialize the memory of the gateway node; S22: The service traffic processing module unpacks each network traffic packet collected by the service traffic collection module, and analyzes and obtains the five-tuples corresponding to each network traffic packet, wherein each five-tuple includes the source IP address, destination IP address, source port, destination port, and packet length of the network traffic packet; S23: Calculate the hash value of the five-tuple of each network traffic packet; S24: construct a network flow storage queue, and store the hash value of each network flow packet and its five-tuple into the network flow storage queue; S25: repeating steps S22 to S24, wherein the service flow processing module determines whether the hash value of the five-tuple of each network flow packet collected by the service flow collection module has been stored in the network flow storage queue, and if it has been stored in the network flow storage queue, then the network flow packet is discarded; if it has not been stored in the network flow storage queue, then the hash value of each network flow packet and its five-tuple is stored in the network flow storage queue; S3: The service flow identification module obtains the network flow packets to be identified in the network flow storage queue, identifies the service type of each network flow packet to be identified in the network flow storage queue based on the deep packet inspection technology and the machine learning flow feature detection method, and outputs the service type identification result of each network flow packet to be identified; Step S3 includes the following steps S31 to S34: S31: The service flow identification module obtains each network flow packet in the network flow storage queue; S32: Based on deep packet inspection technology, identify the service category of each network traffic packet in the network flow storage queue; S33: If the service category of the network traffic packet can be identified based on the deep packet inspection technology, then the service type identification result of the network traffic packet is output; if the service category of the network traffic packet cannot be identified based on the deep packet inspection technology, then the service category of the network traffic packet is identified based on the machine learning flow feature detection method, and the service type identification result of the network traffic packet is output; S34: repeating steps S31 to S33 until the service categories of all network traffic packets in the network flow storage queue are identified; S4: Repeat steps S2 to S3 until the business types of all the network traffic packets to be identified in the network flow storage queue are identified.

2. The embedded device service identification system according to claim 1, It is characterized in that The preset time interval is 5 seconds.

3. The embedded device service identification system according to claim 1, It is characterized in that The process of identifying the business category of each network traffic packet based on deep packet inspection technology. If the deep packet inspection technology fails to identify the business category of each network traffic packet for three consecutive times, the machine learning flow feature detection method will be used to identify the business category of the network traffic packet. After 5 seconds of identification, the business category identification method will be changed to deep packet inspection technology again.