System and method for analyzing incoming traffic flow

Abstract

The present invention relates to computing systems and to the field of providing information security by preliminarily analyzing an incoming traffic flow to determine whether the traffic belongs to a pre-established connection. The technical result is that a traffic flow is analyzed using an efficient system to determine whether the traffic belongs to a predetermined connection, thereby expanding the scope of prior art means and systems for securing information security of a computing network, the result is achieved by a system for analyzing a flow of traffic to determine whether the traffic belongs to a predetermined connection, parsing a data packet at a channel level, a network level, a transmission level and an application level to obtain data from the data packet, dividing the data obtained from the data packet into clusters, and making a decision on each data packet in the cluster by using a setting rule of network data packet processing, detecting a deviation between the network traffic conforming to the processing rule and a normal data packet traffic configuration file, and refining the setting rule of the network data packet processing.

Description

Technical Field

[0001] The present invention relates to computer technology and provides information security by initially analyzing incoming traffic flows to identify traffic belonging to previously established connections. Background Art

[0002] The prior art discloses devices for protecting the perimeter of a local (enterprise) computer network, called firewalls or network screens - a hardware or software package configured to control and filter network packets passing through them based on predefined rules.

[0003] This solution is similar to the "System for Protecting Computer Networks from Unauthorized Access" disclosed in the patent RU2607997, IPC G06F15 / 163 published on January 11, 2017, in which the system for protecting a computer network from unauthorized access includes a network filter located between two computer networks such that all information exchange between the networks is restricted by filtering rules, and the network filter includes at least two network interfaces for data exchange between clients of the first computer network and clients of the second computer network. The system further includes a traffic processing node that includes control means for inputting traffic filtering rules and storing information about the rules, traffic analysis means for ensuring that incoming information complies with the filtering rules, and switching means for interconnecting the network interfaces to facilitate the passage of information permitted by the filtering rules and block information not permitted by the filtering rules, where the filtering rules prohibit the transit transmission of any packets between the network interfaces, except for the following packets: packets having permitted attributes and addressing parameters in the packet header, the format of the information part of the packet corresponding to a template stored in the memory of the firewall, and the request or response parameters matching a set of permitted values stored in the memory of the firewall.

[0004] The prior art includes the patent "Computer Network with Firewall and Firewall" published on October 20, 2003, with IPC G06F15 / 163, G06F15 / 173. The above patent relates to the field of information security and specifically to the hardware and software components of a firewall for preventing unauthorized access and regulating information exchange between different users of a computer network. The technical result is to improve information security by eliminating or completely hiding the network interface of the protection device. The firewall for a local area network includes at least two network interfaces for packet exchange between segments of the computer network, where the exchange is performed according to a packet filtering program. After processing the packets according to the filtering rules, the firewall retains the original information about the physical and logical addresses of the sender of each packet contained in the packet header. The management program does not assign logical addresses to the network interfaces and does not transmit physical address information to the relevant network segments. To be able to modify the filtering rules, the firewall includes a dedicated management interface, where any change to the filtering parameters can only be made through the said management interface.

[0005] The main disadvantage of known similar solutions is that they do not provide a method for controlling established connections and analyzing incoming traffic flows to determine whether the incoming traffic flow belongs to a previously established connection, which limits their effectiveness in protecting network nodes from penetration through vulnerabilities or preventing users from downloading malicious programs (including viruses), as well as in addressing internal threats and data leaks. Summary of the Invention

[0006] The technical result of the invention is to expand the library of technical means and systems for ensuring information security in a computer network by implementing an effective traffic flow analysis system to identify traffic belonging to previously established connections.

[0007] The said technical result is achieved by a traffic flow analysis system for identifying traffic belonging to previously established connections, performing packet parsing to retrieve data from the packets at the link layer, network layer, transport layer, and application layer, clustering the retrieved data, making decisions for each packet within the cluster based on established network packet processing rules, detecting deviations of network traffic corresponding to the processing rules from the normal traffic profile of the packets, and updating the established network packet processing rules.

[0008] In a preferred embodiment, the traffic flow analysis system includes a network traffic receiving and preliminary analysis unit, a new device detection unit, a Deep Packet Inspection (DPI) device, a signature unit, and a memory unit. Among them, the receiving and preliminary analysis unit is coupled to the DPI device, and the receiving and preliminary analysis unit is configured to perform a preliminary analysis on the incoming traffic flow. If the Deep Packet Inspection (DPI) device identifies that the traffic belongs to an established connection, that is, packets with SYN (Synchronize Sequence Numbers) and ACK (Acknowledgment Field Valid) flags have passed through the control unit, the packets can be allowed to pass without filtering.

[0009] In one embodiment, the system includes a graphical user interface to which the working results are transmitted for the system administrator to check.

[0010] If the packet belongs to a new connection, the packet is forwarded to the DPI device for packet parsing. The DPI device parses the packet according to Internet Protocol (IP) specifications to retrieve data at the data link layer, network layer, transport layer, and application layer.

[0011] In one embodiment, the DPI device is capable of retrieving known application protocols in use, tunnel IP addresses, user names, file names, HTTP links, request URLs, and server return codes.

[0012] In one embodiment of the traffic flow analysis system, the network traffic receiving and preliminary analysis unit is configured to buffer data for temporary storage.

[0013] Examples of data retrieved by the DPI device include: physical host address (MAC address), network protocol type (IPv4 and IPv6), source and destination host IP addresses, transport protocol type (such as IP, ICMP, RIP, DDP, ARP, etc.), source and destination port numbers, and LLC, service flag values, and TCP window values.

[0014] Examples of application layer data include remote access and resource sharing protocols such as HTTP, SMTP, FTP, HTTPS, TELNET, SSH, DNS, and other known protocols.

[0015] The DPI device contains a built-in signature database that has characteristics for comparing the analyzed traffic packets, enabling precise identification of the analyzed application or protocol.

[0016] The signature feature database is implemented with update and refinement capabilities to augment the signature feature database with new applications and protocols.

[0017] Using systems and methods involving a preliminary analysis of incoming traffic flows can significantly reduce packet inspection time, especially in cases where timely analysis is crucial.

[0018] For example, for two-way voice calls, a delay of more than 150 milliseconds is unacceptable and may result in disconnection. The time interval allowed for no traffic can also be specified, which is typically set to 25 seconds by default. If the specified time interval is exceeded, the traffic is classified as belonging to a new connection.

[0019] Additionally, user-defined time intervals allowed for no traffic can be set for each type of traffic and / or protocol. In this way, application layer data and relevant metadata for each network packet can be retrieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a block diagram of a traffic processing system utilizing a firewall system according to the present invention.

[0021] Figure 2 is a block diagram of a method for processing traffic using a firewall system according to the present invention and the information flow between units of the system.

[0022] Figure 3 is an architecture diagram of a firewall system according to the present invention. DETAILED DESCRIPTION

[0023] The present invention includes a hardware and software package for implementing a traffic flow analysis system, as Figure 1 shown, the traffic flow analysis system includes the following units:

[0024] · 11 - Network traffic reception and preliminary analysis unit;

[0025] ● 12 - Deep Packet Inspection (DPI) device;

[0026] · 13 - Data clustering unit;

[0027] · 14 - Memory unit;

[0028] · 15 - Recording unit;

[0029] ● 16 - Graphical user interface, included in one embodiment of the system.

[0030] The network traffic reception and preliminary analysis unit 11 is connected to the DPI unit 12, and the network traffic reception and preliminary analysis unit 11 is configured to perform a preliminary analysis of incoming traffic flows. If the traffic is identified as belonging to an established connection, i.e., packets with SYN and ACK flags have passed through the network traffic reception and preliminary analysis unit 11, the packet can be allowed to pass without filtering.

[0031] The DPI device 12 parses the data packets conforming to the IP protocol specification to retrieve data at the link layer, network layer, transport layer, and application layer. The DPI device 12 is configured to retrieve the application protocol used, tunnel IP address, user name, file name, HTTP link, request URL, and server return code from the traffic.

[0032] Examples of the data retrieved from the traffic include: physical host address (MAC address), network protocol type (IPv4 and IPv6), source and destination host IP addresses, transport protocol type (IP, ICMP, RIP, DDP, ARP, etc.), source and destination port numbers, and LLC, service flag value, and TCP window value. Examples of application layer data include remote access and resource sharing protocols such as HTTP, SMTP, FTP, HTTPS, TELNET, SSH, DNS, etc.

[0033] In this way, the application layer data and the relevant metadata of each network data packet can be retrieved. The DPI device 12 has unique features stored in the built-in signature database. Comparing the samples in the database with the traffic being analyzed can accurately identify the application or protocol. However, since new applications and protocols appear regularly, the signature database should also be updated to ensure high-precision identification.

[0034] The data clustering unit 13 is used to group the packet data into clusters of similar objects, thus simplifying subsequent data processing and decision-making. At the same time, specific analysis methods can be applied to each data group.

[0035] The operation of the firewall system is controlled by a set of rules, for example, Suricata (Suricata is an open-source intrusion detection system (IDS) and intrusion prevention system (IPS) developed by the Open Information Security Foundation (OISF)), and these rules can be stored in the memory unit 14. For each cluster, specific processing rules are created and established, and these rules are different from the rules of other clusters. In this way, chains are formed, and the clusters are redirected to these chains for further processing.

[0036] To make the system run, only a single general rule needs to be created for a specific cluster, after which the cluster can be redirected from the specific cluster to a separate chain. The cluster can also be redirected from one processing chain to another. This optimizes network traffic processing because each data packet in the cluster is processed according to its specific rule. In addition, information about each data packet is sent to the recording unit 15 simultaneously. Furthermore, the traffic flow analysis system is configured to ensure that data packets are not discarded; instead, if any non-compliance with the rule is detected, it is forwarded to unit 13 for further reclustering. Utilizing temporary storage of files in a buffer (not depicted in the figure), the network traffic receiving and preliminary analysis unit 11 verifies whether such a data packet has passed through and marks it accordingly, so as to classify the data packet into different clusters during subsequent clustering and discard it only after reprocessing. The established network data packet processing rules are periodically updated based on the passed traffic and the characteristics of the formed clusters.

[0037] In a preferred embodiment of the present invention, the firewall system includes a graphical user interface 16. In this case, the data packet can be sent to the graphical user interface 16 for the system administrator to check instead of being discarded. The system administrator can directly retrieve all network traffic information from the recording unit 15; can also manually allow the data packet to pass through and forward it to the network traffic receiving and preliminary analysis unit 11, and subsequently update the data packet data in the recording unit 15.

[0038] The recording unit 15 collects data on each data packet received from the DPI device 12, including detailed information on the discarded data packets.

[0039] In a preferred embodiment of the invention, the data to be analyzed includes metadata. Compared with the original network traffic, the compression ratio of storing metadata without actual content is about 1 / 100, which greatly increases the amount of information about the data packets stored.

[0040] Another advantage of the system is its ability to immediately transmit information about the abnormal data packets detected in the clustering unit 13 to the recording unit 15.

[0041] In one of the usage scenarios, the traffic flow analysis system serves as an intrusion detection system (IDS) or method.

[0042] This traffic flow analysis method is implemented through a network traffic reception and preliminary analysis unit, which receives incoming traffic and analyzes the incoming traffic. The traffic reception and preliminary analysis unit is connected to a new device detection unit, a deep packet inspection (DPI) device, a signature unit, and a memory unit. It is characterized in that: the reception and preliminary analysis unit is connected to the DPI device, the memory unit, as well as the new device detection unit and the signature unit connected to the memory unit. At the same time, the traffic reception and parsing unit is connected to the new device detection unit.

[0043] As Figure 2 shown, the intrusion detection method is implemented in a hardware and software package including a traffic flow analysis system, which includes the following units:

[0044] ● 21 - Network traffic reception and parsing unit;

[0045] ● 22 - New device detection unit;

[0046] ● 23 - Signature unit;

[0047] ● 24 - Memory unit.

[0048] The memory unit 24 is designed to store various information and rule sets according to information security policies and is connected to the new device detection unit and the signature unit.

[0049] The network traffic reception and parsing unit 21 is used to collect network traffic at the link layer, that is, traffic from the global Internet or traffic transmitted within an enterprise network (e.g., through a local area network), and is responsible for further traffic parsing according to the IP / TCP stack packet structure. The packet parsing device can be used to analyze packets at the network and transport layers, as well as at the network layer, transport layer, and application layer. First, the incoming traffic is divided into TCP, UDP, or other transport streams. Subsequently, the parser marks these streams and decomposes them into high-level protocols and their respective fields, and normalizes the data as needed. Subsequently, units 22 and 23 analyze the obtained decoded, decompressed, and normalized protocol fields to detect any potential network attacks or malicious activities in the network traffic.

[0050] The new device detection unit 22 is designed to identify and control network devices participating in data exchange and data transmission within the network. Network devices can be identified within the network based on their network addresses (e.g., IP addresses, MAC addresses, URIs (Uniform Resource Identifiers), or any other applicable network identifiers). Therefore, network traffic is only transmitted between the identified devices. The new device detection unit 22 decides whether to allow or block outgoing or incoming traffic based on the presence or absence of network device identifiers in the memory unit 24. If the identifier characterizing the network device exists in the memory unit 24, the traffic is allowed to pass, otherwise the traffic is blocked.

[0051] To detect network attacks, the intrusion detection system employs a signature unit 23 that accesses a memory unit 24 containing signatures of known attacks during packet inspection.

[0052] A signature is defined as a contiguous sequence of a finite number of bytes that is necessary and sufficient for uniquely identifying a specific threat. If any of the analyzed program code matches the known virus code, the memory unit 24 decides to block the packet. In a preferred embodiment, the contents of the packet are compared using the checksum of the packet rather than direct comparison, thereby significantly reducing the number of stored records of known viruses. Those skilled in the art should understand that in order to improve the effectiveness of threat detection, the database of known viruses must be continuously updated because new malicious code and threats emerge every day.

[0053] The characteristics of network attacks are very similar to those of viruses and consist of a set of attributes that enable network attacks to be distinguished from other types of network traffic. For example, conflicting TCP packet flags (such as SYN and FIN) may be set simultaneously. Various attack programs often exploit this flag combination to bypass filters and monitors that only focus on the presence of the SYN flag. The inclusion of the string “GET / cgi-bin / . / etc / passwd” in the data segment of an HTTP packet indicates the exploitation of a path traversal vulnerability. Finally, a TCP packet header containing the destination port 139 and the out-of-band (OOB) flag can serve as an indication of a WinNuke attack.

[0054] According to Figure 3 , the traffic flow analysis system according to the present invention may further include additional functions, such as:

[0055] ● 30—Packet filtering;

[0056] ● 31—Routing;

[0057] · 32—Intrusion detection function (IDS based on industrial protocols);

[0058] · 33—Creation of a fault-tolerant cluster;

[0059] · 34—Streaming media antivirus software;

[0060] · 35—Management and configuration of network interfaces;

[0061] ● 36—Virtual private network (VPN);

[0062] ● 37—Integration with Active Directory;

[0063] · 38—Domain Name System (DNS);

[0064] ·39 - Web server function.

Claims

1. A traffic flow analysis system, comprising a network traffic receiving and preliminary analysis unit, a new device detection unit, a deep packet inspection (DPI) device, a signature unit and a memory unit, characterized in that: The receiving and preliminary analysis unit is connected to the deep packet inspection (DPI) device, and the receiving and preliminary analysis unit is configured to perform preliminary analysis on the incoming traffic flow, wherein, when the deep packet inspection (DPI) device identifies that the traffic belongs to an established connection, which means that such a data packet with synchronization (SYN) and confirmation (ACK) flags has passed through the control unit, the data packet is allowed to pass without filtering.

2. The system of claim 1, further implementing a graphical user interface.

3. The system according to claim 1, wherein: In the event that the data packet is identified as belonging to a new connection, the data packet is forwarded to the DPI device for packet parsing, and the DPI device parses the data packet according to Internet protocol specifications to retrieve data at the link layer, network layer, transport layer, and application layer.

4. The system according to claim 1, wherein: The DPI device is able to retrieve known used application protocols, tunnel IP addresses, usernames, file names, HTTP links, request URLs, and server return codes.

5. The system according to claim 1, wherein: The network traffic receiving and preliminary analyzing unit of the traffic flow analyzing system is configured to be capable of performing data buffering for temporary storage of the data.

6. A traffic flow analysis method implemented using a network traffic receiving and preliminary analysis unit, the network traffic receiving and preliminary analysis unit receives incoming traffic and analyzes the incoming traffic, the network traffic receiving and preliminary analysis unit is connected to a new device detection unit, the new device detection unit detects and controls network devices that perform data exchange and data transmission on the network, the network traffic receiving and preliminary analysis unit is connected to a deep packet inspection (DPI) device, the deep packet inspection device compares samples from a database with the analyzed traffic and identifies the application or protocol, the method also uses a signature unit and a memory unit to sequentially check the received decoded, decompressed and normalized protocol fields to detect whether there is a network attack or malicious activity in the network flow, characterized in that The receiving and preliminary analysis unit is connected to the deep packet inspection (DPI) device, and the receiving and preliminary analysis unit is implemented to be able to perform preliminary analysis on the incoming traffic flow, wherein, when the deep packet inspection (DPI) device identifies that the traffic belongs to an established connection, which means that such a data packet with synchronization (SYN) and confirmation (ACK) flags has passed through the control unit, the data packet is allowed to pass without filtering.

7. The method according to claim 6, wherein: The results of the work are sent to the graphical user interface.

8. The method according to claim 6, wherein: In case the data packet belongs to a new connection, the data packet is sent to the DPI device for packet parsing, and the DPI device parses the data packet according to the Internet Protocol specification to retrieve data at the link layer, network layer, transport layer, and application layer.

9. The method according to claim 6, wherein: The DPI device retrieves known application protocols, tunnel IP addresses, usernames, file names, HTTP links, request URLs, and server return codes.

10. The method according to claim 6, wherein: The network traffic receiving and preliminary analyzing unit of the traffic flow analyzing system performs data buffering for temporary storage of the data.

Images