Non-depth flow analysis and streaming matching search analysis method

Abstract

The invention provides a non-deep traffic analysis streaming matching search analysis method, which comprises the following steps of: constructing a deep data packet detection architecture on the basis of a data platform development kit (DPDK); aiming at the non-encrypted traffic, establishing a multi-mode recognition mechanism combining a regular expression and a feature bit stream mode; aiming at the encrypted traffic, constructing an encrypted traffic feature library according to the statistical features, the protocol features and the behavior features; real-time risk detection of network traffic is realized by adopting a streaming matching search technology; and constructing an intelligent decision and response mechanism, and integrating flow analysis and identification results. According to the non-deep traffic analysis streaming matching search analysis method provided by the invention, real-time analysis, risk identification and supervision of network non-encrypted traffic and encrypted traffic are realized by taking a data platform development kit (DPDK) as a basis and combining a high-performance streaming regular expression engine and a finite-state machine principle; the method can be widely applied to scenes of network communication supervision, data security protection, malicious traffic monitoring and the like.

Description

Technical Field

[0001] This invention relates to the field of network data security supervision technology, specifically to a non-deep traffic parsing streaming matching search analysis method. Background Technology

[0002] In the field of network data security supervision, with the explosive growth of network traffic and the widespread adoption of data transmission encryption technologies, traditional IP packet inspection technologies are no longer sufficient to meet the demands and cannot penetrate deep into the application layer to detect potential risks. While deep packet inspection (DPI) technology has become a research hotspot, it still faces many challenges in practical applications.

[0003] For unencrypted traffic identification, existing methods for extracting rules using regular expressions and feature bitstream patterns have significant shortcomings. While regular expressions offer strong descriptive power, they suffer from low matching efficiency when dealing with large-scale network traffic, failing to meet real-time monitoring requirements. Although feature bitstream patterns are more efficient, their descriptive power is poor, making it difficult to accurately define complex traffic characteristics, which can easily lead to missed detections or misjudgments, thus failing to provide reliable traffic analysis results for regulatory authorities.

[0004] In terms of encrypted traffic processing, since most data is transmitted in encrypted form, existing technologies lack effective means to identify encrypted traffic, cannot parse the encrypted data content, and find it difficult to detect violations in encrypted transmission, which seriously restricts the regulatory authorities' comprehensive control over network data security.

[0005] Furthermore, in network traffic risk event detection scenarios, traditional matching methods require loading the entire text stream into memory before matching can be performed. This not only consumes a large amount of storage resources but also fails to output risk events in real time, making it difficult to meet regulatory requirements for timeliness. Common matching algorithms suffer from high computational load and low matching efficiency when handling large-scale network traffic, and cannot adapt to the high-speed and dynamically changing characteristics of network traffic.

[0006] Application number CN201810048761.2 discloses a multi-keyword search method applied to streaming data, comprising: reading characters of a target string in the current state of a preset finite state automaton, searching for characters in the read target string that correspond to the multi-keyword to be searched, and obtaining search results; wherein the preset finite state automaton is a finite state automaton constructed based on the multi-keyword; if the search fails, the backtracking state of the current state is repeatedly searched using preset rules until the backtracking state and the characters of the target string are used to obtain search results corresponding to the characters of the multi-keyword.

[0007] Application number CN202110329891.5 discloses a testing method, apparatus, device, and storage medium based on industrial control protocol traffic. The testing device, loaded with the Data Plane Development Kit (DPDK) program, acquires the source industrial control protocol traffic packets corresponding to the target industrial control protocol, modifies the target message fields in the source industrial control protocol traffic packets to generate at least one industrial control protocol traffic packet to be tested, then sends at least one test data packet corresponding to each industrial control protocol traffic packet to the industrial control network device under test, and receives at least one feedback data packet returned by the industrial control network device under test. Finally, based on the sending information of at least one test data packet and the receiving information of at least one feedback data packet, the test result of the industrial control network device under test is determined. Summary of the Invention

[0008] To address the aforementioned issues, this invention provides a non-deep traffic parsing streaming matching search analysis method. Based on the Data Platform Development Kit (DPDK), it combines a high-performance streaming regular expression engine with the principle of finite state machines to achieve real-time parsing, risk identification, and monitoring of unencrypted and encrypted network traffic. This method can be widely applied to scenarios such as network communication supervision, data security protection, and malicious traffic monitoring.

[0009] This invention provides a non-deep traffic parsing streaming matching search analysis method, including:

[0010] A deep packet inspection architecture is built based on the Data Platform Development Kit (DPDK).

[0011] For unencrypted traffic, a multi-mode recognition mechanism combining regular expressions and characteristic bitstream patterns is established;

[0012] For encrypted traffic, an encrypted traffic feature library is constructed based on statistical characteristics, protocol characteristics, and behavioral characteristics;

[0013] Real-time risk detection of network traffic is achieved using streaming matching search technology;

[0014] Build an intelligent decision-making and response mechanism that integrates traffic analysis and identification results.

[0015] The non-deep traffic parsing streaming matching search analysis method of the present invention, as a preferred embodiment, includes a deep packet detection architecture comprising:

[0016] Deploy data acquisition modules at key network nodes to acquire network interface card traffic data in real time based on eBPF technology, and perform full-level parsing of data packets from the link layer to the application layer; eBPF (extended Berkeley Packet Filter) technology is a revolutionary Linux kernel technology that allows users to run sandbox programs securely and efficiently without modifying the kernel source code or loading kernel modules, and is widely used in system observation, network optimization, security monitoring and other fields.

[0017] By leveraging DPDK's data processing capabilities, data processing latency can be reduced, enabling rapid capture and preliminary analysis of network traffic.

[0018] DPDK's efficient data processing capabilities provide foundational data for subsequent traffic identification and risk detection.

[0019] The non-deep traffic parsing streaming matching search analysis method of this invention, as a preferred embodiment, includes the establishment of a multimodal recognition mechanism comprising:

[0020] Accurate description of complex traffic characteristics using regular expressions;

[0021] Achieve rapid matching of massive amounts of data through feature bitstream patterns;

[0022] Through an intelligent rule scheduling algorithm, regular expressions, feature bitstream patterns, or a combination of both are selected based on traffic feature complexity, data scale, and real-time processing performance requirements. Intelligent rule scheduling ensures both the accuracy of rule descriptions and improves matching efficiency, effectively solving the problem of balancing efficiency and accuracy in unencrypted traffic identification using traditional single-mode methods. The multi-mode identification mechanism dynamically allocates regular expressions and feature bitstream patterns according to traffic characteristics. When processing traffic with complex business rules, regular expressions ensure accurate matching, such as identifying sensitive data transmissions in specific formats; while feature bitstream patterns quickly filter massive amounts of regular traffic.

[0023] The non-deep traffic parsing streaming matching search analysis method described in this invention, as a preferred embodiment, includes an intelligent rule scheduling algorithm comprising:

[0024] Real-time monitoring of network traffic load, rule matching time, and matching accuracy;

[0025] Each rule is assigned a priority based on the frequency and importance of traffic features; rules with priority I or II are matched using feature bitstream patterns, while rules with priority III or IV are matched using regular expressions.

[0026] The priority of a rule is determined based on the priority parameter F:

[0027] F=α×f+β×m

[0028] Where f is the frequency parameter of the flow characteristics, m is the importance parameter of the flow characteristics, and α and β are constants, α∈(0,1) and β∈(0,1);

[0029] When the system load exceeds the set threshold, the matching of the lowest priority rule is suspended.

[0030] The intelligent rule scheduling algorithm monitors metrics such as traffic load and feature complexity in real time and intelligently allocates computing resources. For example, during peak network usage periods, it prioritizes assigning fast matching modes to frequently occurring simple traffic features to ensure rapid processing of core business traffic; for a small number of complex security rule matches, it allocates more computing resources to ensure accuracy. This mechanism effectively balances resource consumption and processing efficiency, enabling the system to operate stably under high load.

[0031] The non-deep traffic parsing streaming matching search analysis method described in this invention, as a preferred method, includes the construction of an encrypted traffic feature database comprising:

[0032] By analyzing encrypted traffic data and combining it with machine learning algorithms, we can uncover the characteristic patterns of encrypted traffic. The data includes packet size distribution, transmission time interval, protocol type, and version information.

[0033] Encrypted traffic is classified and identified by its characteristic patterns to determine whether it poses a security risk.

[0034] The non-deep traffic parsing streaming matching search analysis method described in this invention, as a preferred embodiment, includes the following streaming matching search technology:

[0035] Based on the principles of automata and finite state machines, this algorithm utilizes AC automata algorithms, Rabin-Karp algorithms, Shift-And algorithms, and other technologies, combined with the Hyperscan high-performance streaming regular expression engine, to perform real-time pattern matching during data streaming.

[0036] The non-deep traffic parsing streaming matching search analysis method of the present invention, as a preferred embodiment, includes an apparatus comprising:

[0037] Data acquisition module: Installed on the data acquisition device, used to capture network data packets and transmit the network data packets to the data parsing module. The network data packets include link layer data packets and application layer data packets.

[0038] Data parsing module: Located on the data acquisition device, it is used to receive network data packets transmitted by the data acquisition module, parse the network data packets to obtain parsed data packets, and transmit the parsed data packets to the data preprocessing module.

[0039] Data preprocessing module: Used to receive the parsed data packets transmitted by the data parsing module, remove redundant fields, fill missing values ​​and unify the data format of the parsed data packets to obtain the processed data packets, and transmit the processed data packets to the data transmission module;

[0040] Data transmission module: used to receive the processed data packets transmitted by the data preprocessing module, encapsulate the processed data packets into a message format according to a preset protocol specification, and transmit the message format data packets to the traffic analysis service module via the network;

[0041] Traffic Analysis Service Module: This module receives message format data packets from the data transmission module, performs real-time pattern matching using the Hyperscan engine and finite state machine based on the message format data packets, performs risk detection on the traffic characteristics of the message format data packets, and triggers alarm mechanisms and corresponding strategies when the traffic characteristics meet the risk rules.

[0042] The non-deep traffic parsing streaming matching search analysis method described in this invention, as a preferred embodiment, involves data parsing of network data packets, including:

[0043] Parse data packets at the link layer, including identifying the source MAC address, destination MAC address, and protocol type fields in the Ethernet frame header;

[0044] If the protocol type is IPv4 or IPv6, then the network layer packet header is parsed, including extracting the source IP address, destination IP address, IP protocol version, header length, and type of service;

[0045] Parsing transport layer protocols, including parsing port numbers, sequence numbers, and acknowledgment numbers;

[0046] Application layer data parsing includes application protocol identification and data content extraction based on protocol format specifications.

[0047] The non-deep traffic parsing streaming matching search analysis method described in this invention, as a preferred embodiment, includes the Hyperscan engine configuration as follows:

[0048] Select the Hyperscan working mode according to system requirements. For batch data processing with high non-real-time requirements, use block mode; for real-time network traffic monitoring, use streaming mode.

[0049] In streaming mode, a segment of memory is pre-allocated to store matching status information;

[0050] The regular expression rule set is compiled into a database file by Hyperscan's compiler, and Hyperscan's matching engine is called at runtime to perform pattern matching on the input traffic data.

[0051] The matching process is accelerated by utilizing the SIMD instructions of Intel processors;

[0052] Register a callback function that, when a match is successful, executes actions such as recording the matching result and / or triggering an alarm.

[0053] The non-deep traffic parsing streaming matching search analysis method of this invention, as a preferred embodiment, includes the design of a finite state machine comprising:

[0054] Define a set of states, which includes an initial state, intermediate states, and a final state;

[0055] Design a transition function to determine the transition rules between states under different input conditions;

[0056] The output function outputs the corresponding result based on the current state.

[0057] The present invention has the following advantages:

[0058] (1) Significantly improves the efficiency of unencrypted traffic monitoring: The multi-pattern recognition mechanism dynamically allocates regular expressions and feature bitstream modes based on traffic characteristics. When processing traffic with complex business rules, regular expressions ensure accurate matching, such as identifying sensitive data transmissions in specific formats. Faced with massive amounts of regular traffic, feature bitstream modes quickly filter, improving efficiency by 20%. The intelligent rule scheduling algorithm monitors indicators such as traffic load and feature complexity in real time and intelligently allocates computing resources. During peak network usage periods, it prioritizes allocating fast matching modes for frequently occurring simple traffic features to ensure rapid processing of core business traffic. For a small number of complex security rule matches, it invests more computing resources to ensure accuracy. This mechanism effectively balances resource consumption and processing efficiency, enabling the system to operate stably under high load, improving resource utilization by 20%.

[0059] (2) Real-time and efficient traffic risk monitoring: Streaming matching search technology is based on the principles of automata and finite state machines. It performs real-time matching during data streaming. Taking Hyperscan high-performance streaming regular expression engine as an example, its streaming mode can save the data matching state, adapting to the situation where data is scattered across multiple packets in the network. It can output risk events in real time without waiting for all data to be received. When processing large-scale network traffic, the risk event response time is shortened by 10%, and memory usage is reduced by 10%, enabling rapid capture and early warning of network risks. Streaming matching search technology significantly reduces the computational load by optimizing algorithms and memory management. For example, in the AC automaton algorithm, it reduces the number of state transitions and improves matching efficiency. Hyperscan compresses and stores the matching state, reducing memory overhead. This allows the system to maintain high efficiency under high traffic surges, increasing processing capacity by 2 times, ensuring the stability and reliability of the monitoring system, and safeguarding network data security. Attached Figure Description

[0060] Figure 1 Flowchart of a non-deep traffic parsing streaming matching search analysis method;

[0061] Figure 2 A flowchart of the architecture for deep packet detection, a non-deep traffic parsing streaming matching search analysis method;

[0062] Figure 3 A flowchart is established for the multimodal recognition mechanism of a non-deep traffic parsing streaming matching search analysis method;

[0063] Figure 4 Flowchart of an intelligent rule scheduling algorithm for a non-deep traffic parsing streaming matching search analysis method;

[0064] Figure 5 This is a flowchart illustrating the construction process of an encrypted traffic feature library for a non-deep traffic parsing streaming matching search analysis method. Detailed Implementation

[0065] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0066] Example 1

[0067] like Figure 1 As shown, a non-deep traffic parsing streaming matching search analysis method includes:

[0068] S1. Based on the Data Platform Development Kit (DPDK), construct a deep packet inspection architecture; such as... Figure 2 As shown, the deep packet inspection architecture includes:

[0069] S11. Deploy data acquisition modules at key network nodes to acquire network interface card (NIC) traffic data in real time and perform full-layer parsing of data packets from the link layer to the application layer. Network data packet parsing includes parsing link layer packets, including identifying the source MAC address, destination MAC address, and protocol type field in the Ethernet frame header; if the protocol type is IPv4 or IPv6, then parsing the network layer header, including extracting the source IP address, destination IP address, IP protocol version, header length, and service type; parsing the transport layer protocol, including parsing the port number, sequence number, and acknowledgment number; and parsing application layer data, including identifying the application protocol and extracting data content according to the protocol's format specifications; and identifying the application protocol and extracting data content according to the format specifications of different protocols (such as HTTP, FTP, SMTP, etc.). For example, for HTTP protocol data packets, parsing out the request method, URL, HTTP version, header fields, and request / response body, etc.

[0070] S12. By leveraging DPDK's data processing capabilities, data processing latency is reduced, enabling rapid capture and preliminary analysis of network traffic.

[0071] S2. For unencrypted traffic, establish a multi-mode recognition mechanism that combines regular expressions with characteristic bitstream patterns; such as... Figure 3 As shown, the establishment of a multimodal recognition mechanism includes:

[0072] S21. Accurately describe complex traffic characteristics using regular expressions;

[0073] S22. Achieve rapid matching of massive amounts of data through feature bitstream patterns;

[0074] S23. Using an intelligent rule-based scheduling algorithm, select a regular expression or a feature bitstream mode, or a combination of both, based on the complexity of traffic characteristics, data scale, and real-time processing performance requirements; for example... Figure 4 As shown, the intelligent rule scheduling algorithm includes:

[0075] S231. Real-time monitoring of network traffic load, rule matching time, and matching accuracy;

[0076] S232. Assign priority to each rule based on the frequency and importance of traffic features; use feature bitstream pattern matching for rules with priority I or II, and use regular expression matching for rules with priority III or IV.

[0077] The priority of a rule is determined based on the priority parameter F:

[0078] F=α×f+β×m

[0079] Where f is the frequency parameter of the flow characteristics, m is the importance parameter of the flow characteristics, and α and β are constants, α∈(0,1) and β∈(0,1);

[0080] For example: f takes values ​​of 1 / 2 / 3 / 4 according to the frequency of occurrence of the flow characteristics; m takes values ​​of 1 / 2 / 3 / 4 according to the importance of the flow characteristics; α = 0.2, β = 0.8;

[0081] When 0 < F ≤ 1, the priority is I;

[0082] When 1 < F ≤ 2, the priority is II;

[0083] When 2 < F ≤ 3, the priority is III;

[0084] When 4 < F ≤ 4, the priority is IV;

[0085] For example, if the common flow characteristics that occur frequently are f=4 and m=1, then F=1.6, and the priority is II;

[0086] For complex rules involving sensitive data, if f=1 and m=4, then F=3.4, and the priority is IV;

[0087] S233. When the system load exceeds the set threshold, suspend the matching of the lowest priority rule;

[0088] S3. For encrypted traffic, construct an encrypted traffic feature library based on statistical characteristics, protocol characteristics, and behavioral characteristics; such as... Figure 5 As shown, the construction of the encrypted traffic signature database includes:

[0089] S31. By analyzing the data information of encrypted traffic and combining it with machine learning algorithms, we can discover the characteristic patterns of encrypted traffic. The data information includes data packet size distribution, transmission time interval, protocol type and version information.

[0090] S32. Classify and identify encrypted traffic based on its characteristic patterns to determine whether the encrypted traffic poses a security risk;

[0091] S4. Real-time risk detection of network traffic is achieved by using streaming matching search technology. Streaming matching search technology includes the use of automata and finite state machine principles, AC automata algorithm, Rabin-Karp algorithm, Shift-And algorithm, etc., combined with Hyperscan high-performance streaming regular expression engine to perform pattern matching in real time during data streaming transmission.

[0092] S5. Construct an intelligent decision-making and response mechanism, and integrate traffic analysis and identification results.

[0093] A non-deep traffic parsing streaming matching search analysis device includes:

[0094] Data acquisition module: Installed on the data acquisition device, it captures network data packets based on eBPF technology and transmits the network data packets to the data parsing module. The network data packets include link layer data packets and application layer data packets.

[0095] Data parsing module: Located on the data acquisition device, it is used to receive network data packets transmitted by the data acquisition module, parse the network data packets to obtain parsed data packets, and transmit the parsed data packets to the data preprocessing module.

[0096] Data preprocessing module: Used to receive the parsed data packets transmitted by the data parsing module, remove redundant fields, fill missing values ​​and unify the data format of the parsed data packets to obtain the processed data packets, and transmit the processed data packets to the data transmission module;

[0097] Data transmission module: used to receive processed data packets transmitted by the data preprocessing module, encapsulate the processed data packets into message format according to a preset protocol specification, and transmit the message format data packets to the traffic analysis service module through a high-speed network (such as InfiniBand or 10G Ethernet); during the transmission process, a checksum mechanism (such as CRC check) is used to ensure the integrity and accuracy of the data;

[0098] Traffic Analysis Service Module: This module receives message format data packets from the data transmission module, performs real-time pattern matching using the Hyperscan engine and finite state machine based on the message format data packets, and performs risk detection on the traffic characteristics of the message format data packets. When the traffic characteristics match risk rules, it triggers an alarm mechanism and corresponding strategies. This includes:

[0099] Rule base construction and management: An unencrypted traffic rule base is established, divided into regular expression rule sets and feature bitstream pattern rule sets. Regular expression rule sets are stored in text file format, with each rule corresponding to a traffic feature pattern; for example, the rule for recognizing ID card numbers is (^\d{15}$)|(^\d{18}$)|(^\d{17}(\d|X|x)$). Feature bitstream pattern rule sets are stored in binary file format, with rule definitions in hexadecimal representation; for example, defining the feature bitstream pattern of a specific protocol header as 0x12345678. The rule base management system provides a visual interface, supporting supervisors in adding, deleting, and modifying rules, and also features rule version management, rule validity verification, and rule conflict detection functions.

[0100] Multi-pattern matching engine implementation: A multi-pattern matching engine is built based on the Hyperscan high-performance streaming regular expression engine and a custom feature bitstream matching module. During system initialization, the regular expression rule set is compiled into a Hyperscan executable database file, and the feature bitstream pattern rules are loaded into memory. When unencrypted traffic data is input, the multi-pattern matching engine first determines the data type and characteristics. If the data contains text content and the rules are complex, the Hyperscan engine is called for regular expression matching. If the data is in binary format and the characteristics are relatively simple, the feature bitstream matching module is used for fast matching. The matching process adopts multi-threaded parallel processing to fully utilize the server's multi-core CPU resources and improve matching efficiency.

[0101] Intelligent Rule Scheduling Algorithm: An intelligent rule scheduling algorithm is designed that monitors network traffic load, rule matching time, and matching accuracy in real time. The algorithm employs a dynamic priority strategy, assigning priority to each rule based on the frequency and importance of traffic characteristics. For example, rules with frequently occurring common traffic characteristics are given lower priority and matched using a feature bitstream pattern; complex rules involving sensitive data transmission are given higher priority and regular expression matching is prioritized. When the system load is too high, the algorithm automatically adjusts the rule matching strategy, suspending the matching of some low-priority rules to ensure rapid processing of core business traffic.

[0102] Encrypted Traffic Feature Database Construction: Collect encrypted traffic data from various sources, including traffic samples from protocols such as HTTPS, SSH, and SSL / TLS; extract features from these samples, calculating statistical characteristics such as packet count, byte count, average packet size, packet size distribution, and transmission time interval; extracting protocol characteristics such as protocol type, protocol version, random numbers during the handshake process, and certificate information; analyzing behavioral characteristics such as traffic transmission direction, duration, and frequency of source / destination IP address pair changes; storing the extracted features in a relational database (such as MySQL) to build an encrypted traffic feature database, and regularly updating and expanding the database content.

[0103] Encrypted traffic identification model training: Machine learning algorithms (such as Convolutional Neural Networks (CNN) and Long Short-Term Memory Networks (LSTM)) are used to construct the encrypted traffic identification model. Data from the encrypted traffic feature database is divided into training and test sets in an 8:2 ratio. During training, the training set data is input into the model, and the model parameters are adjusted using backpropagation to minimize the loss function value on the training set. The trained model is evaluated using the test set, and metrics such as accuracy, recall, and F1 score are calculated. The model structure and parameters are optimized based on the evaluation results. The trained model is then deployed on a traffic analysis server for real-time encrypted traffic identification.

[0104] Application of Encrypted Traffic Classification and Regulatory Strategies: When encrypted traffic enters the system, the encrypted traffic identification model first determines whether it is abnormal. If it is identified as abnormal traffic, the classification model further categorizes it into different types, such as encrypted malicious software communication, illegal data leakage, and normal business transmission. Regulatory authorities take corresponding measures for different types of encrypted traffic according to preset regulatory strategies. For example, for encrypted malicious software communication traffic, its transmission is immediately blocked, and relevant source / destination IP addresses, timestamps, and other information are recorded. For normal business transmission traffic, traffic monitoring and logging are performed to ensure the compliance of data transmission.

[0105] Hyperscan Engine Configuration: Select the Hyperscan working mode according to system requirements. For batch data processing with high non-real-time requirements, use block mode; for real-time network traffic monitoring, use streaming mode. In streaming mode, pre-allocate a segment of memory to store matching status information. Compile the regular expression rule set into a database file using Hyperscan's compiler. During runtime, call Hyperscan's matching engine (such as NFA or DFA engine) to perform pattern matching on the input traffic data. Utilize Intel processor's SIMD instructions to accelerate the matching process. Register callback functions to execute operations such as recording matching results and / or triggering alarms when a match is successful.

[0106] Finite state machine design: Define a set of states, including an initial state, intermediate states, and a terminal state; design a transition function to determine the transition rules between states under different input conditions; for example, when matching a specific protocol header format, if the input character matches the format of the first byte of the protocol header, the system transitions from the initial state to the first intermediate state; if subsequent characters also match the protocol format requirements, the system continues to transition to the next intermediate state until the matching is complete and the system enters the terminal state; the output function outputs the corresponding result based on the current state, such as outputting an error message in the matching failure state and outputting the matched flow characteristics in the matching success state; the finite state machine is implemented in C / C++ and works in conjunction with the Hyperscan engine to improve the accuracy and efficiency of streaming matching search;

[0107] Real-time risk monitoring and response: Streaming matching search technology is integrated into the network traffic monitoring system. Upon entering the system, traffic data is immediately matched in real-time using the Hyperscan engine and finite state machine. Once traffic characteristics matching risk rules are detected, the system immediately triggers an alarm mechanism, notifying regulatory personnel via sound, email, SMS, etc. Simultaneously, based on preset response policies, corresponding operations are automatically executed, such as blocking source IP addresses, blocking traffic transmission, and recording detailed logs, achieving real-time monitoring and rapid response to network risks.

[0108] The above description is illustrative only and not restrictive of the present invention. Those skilled in the art will understand that any modifications, variations or equivalents that can be made without departing from the spirit and scope defined by the claims will fall within the protection scope of the present invention.

Claims

1. A non-deep traffic parsing streaming matching search analysis method, characterized in that: include: A deep packet inspection architecture is built based on the Data Platform Development Kit (DPDK). For unencrypted traffic, a multi-mode recognition mechanism combining regular expressions and characteristic bitstream patterns is established; For encrypted traffic, an encrypted traffic feature library is constructed based on statistical characteristics, protocol characteristics, and behavioral characteristics; Real-time risk detection of network traffic is achieved using streaming matching search technology; Build an intelligent decision-making and response mechanism that integrates traffic analysis and identification results.

2. The non-deep traffic parsing streaming matching search analysis method according to claim 1, characterized in that: The deep packet inspection architecture includes: Deploy data acquisition modules at key network nodes to acquire network interface card traffic data in real time based on eBPF technology, and perform full-level parsing of data packets from the link layer to the application layer; By leveraging DPDK's data processing capabilities, data processing latency can be reduced, enabling rapid capture and preliminary analysis of network traffic.

3. The non-deep traffic parsing streaming matching search analysis method according to claim 1, characterized in that: The establishment of the multimodal recognition mechanism includes: The regular expression is used to accurately describe complex traffic characteristics; The aforementioned characteristic bitstream pattern enables rapid matching of massive amounts of data; The intelligent rule scheduling algorithm selects the regular expression, the feature bitstream mode, or a combination of the two modes based on the complexity of traffic characteristics, data scale, and real-time processing performance requirements.

4. The non-deep traffic parsing streaming matching search analysis method according to claim 3, characterized in that: The intelligent rule scheduling algorithm includes: Real-time monitoring of network traffic load, rule matching time, and matching accuracy; Each rule is assigned a priority based on the frequency and importance of traffic features; rules with priority I or II are matched using feature bitstream patterns, while rules with priority III or IV are matched using regular expressions. When the system load exceeds the set threshold, the matching of the lowest priority rule is suspended.

5. The non-deep traffic parsing streaming matching search analysis method according to claim 1, characterized in that: The construction of the encrypted traffic signature database includes: By analyzing the data information of the encrypted traffic and combining it with machine learning algorithms, the characteristic patterns of the encrypted traffic are mined. The data information includes data packet size distribution, transmission time interval, protocol type, and version information. The encrypted traffic is classified and identified based on the aforementioned characteristics to determine whether the encrypted traffic poses a security risk.

6. The non-deep traffic parsing streaming matching search analysis method according to claim 1, characterized in that: The streaming matching search technology includes: Based on the principles of automata and finite state machines, this algorithm utilizes AC automata algorithms, Rabin-Karp algorithms, Shift-And algorithms, and other technologies, combined with the Hyperscan high-performance streaming regular expression engine, to perform real-time pattern matching during data streaming.

7. A non-deep traffic parsing streaming matching search analysis method according to any one of claims 1 to 6, characterized in that: The apparatus for implementing the method includes: Data acquisition module: installed on the data acquisition device, used to capture network data packets and transmit the network data packets to the data parsing module. The network data packets include link layer data packets and application layer data packets. Data parsing module: installed on the data acquisition device, used to receive the network data packets transmitted by the data acquisition module, to parse the network data packets to obtain parsed data packets, and to transmit the parsed data packets to the data preprocessing module; Data preprocessing module: used to receive the parsed data packet transmitted by the data parsing module, used to remove redundant fields, fill missing values ​​and unify the data format of the parsed data packet to obtain a processed data packet, used to transmit the processed data packet to the data transmission module; Data transmission module: used to receive the processed data packet transmitted by the data preprocessing module, to encapsulate the processed data packet into a message format according to a preset protocol specification, and to transmit the message format data packet to the traffic analysis service module through the network; Traffic analysis service module: used to receive the message format data packets transmitted by the data transmission module, to perform real-time pattern matching based on the message format data packets using the Hyperscan engine and finite state machine, to perform risk detection on the traffic characteristics of the message format data packets, and to trigger an alarm mechanism and corresponding strategy when the traffic characteristics meet the risk rules.

8. The non-deep traffic parsing streaming matching search analysis method according to claim 7, characterized in that: The data parsing of the network data packets includes: The link layer data packets are parsed, including identifying the source MAC address, destination MAC address, and protocol type field in the Ethernet frame header; If the protocol type is IPv4 or IPv6, then the network layer packet header is parsed, including extracting the source IP address, destination IP address, IP protocol version, header length, and type of service; Parsing transport layer protocols, including parsing port numbers, sequence numbers, and acknowledgment numbers; Application layer data parsing includes application protocol identification and data content extraction based on protocol format specifications.

9. The non-deep traffic parsing streaming matching search analysis method according to claim 7, characterized in that: The Hyperscan engine configuration includes: Select the Hyperscan working mode according to system requirements. For batch data processing with high non-real-time requirements, use block mode; for real-time network traffic monitoring, use streaming mode. In streaming mode, a segment of memory is pre-allocated to store matching status information; The regular expression rule set is compiled into a database file by Hyperscan's compiler, and Hyperscan's matching engine is called at runtime to perform pattern matching on the input traffic data. The matching process is accelerated by utilizing the SIMD instructions of Intel processors; Register a callback function that, when a match is successful, executes actions such as recording the matching result and / or triggering an alarm.

10. The non-deep traffic parsing streaming matching search analysis method according to claim 7, characterized in that: The design of the finite state machine includes: Define a set of states, which includes an initial state, intermediate states, and a final state; Design a transition function to determine the transition rules between states under different input conditions; The output function outputs the corresponding result based on the current state.

Images