Fixed network traffic screening and multi-dimensional monitoring method and system based on instruction association

Abstract

The application discloses a kind of fixed network traffic screening and multidimensional monitoring method, system based on instruction association, and screening instruction and monitoring instruction are screened by screening instruction ID and user ID association, and the traffic screened is directly sent into traffic monitoring engine by associated user ID output channel, the problem of screening and monitoring is solved and is split;The traffic monitoring engine of the application integrates NetFlow sub-monitoring engine, protocol metadata sub-monitoring engine, malicious message sub-monitoring engine, malicious resource sub-monitoring engine, malicious file sub-monitoring engine, encrypted traffic sub-monitoring engine, DNS sub-monitoring engine, and realizes the monitoring of full scene by one instruction platform.Compared with prior art, the application solves the encrypted traffic, DNS special monitoring capability, also supports PCAP index and on-demand query backtracking, can provide high-quality, backtracking traffic data for advanced threat analysis, attack tracing and emergency disposal.

Description

Technical Field

[0001] This invention relates to the field of network security screening and monitoring technology, and is particularly applicable to fixed network traffic screening and multi-dimensional monitoring methods and systems based on command association. Background Technology

[0002] Cyberattacks are becoming increasingly complex and covert, particularly with the emergence of new threats such as encrypted traffic attacks, DNS tunneling attacks, and FastFlux. Traditional fixed-line security monitoring methods are no longer sufficient for real-world applications. In accordance with national-level construction requirements, it is necessary to achieve full traffic coverage of existing data collection nodes in key provinces and cities, building upon existing fixed-line network coverage in provincial / metropolitan area networks. This requires enhancing the ability to dynamically filter traffic based on IP addresses, domain names, and attack characteristics, ensuring that the filtered traffic accounts for no less than 20% of the total bandwidth of the provincial / metropolitan area network. Furthermore, it necessitates refined monitoring and analysis of the filtered traffic, supporting the on-demand transmission of PCAP files back to the Ministry.

[0003] Existing technical solutions, such as the DPI-based full-traffic splitting acquisition and analysis scheme, can acquire complete data packets by deploying splitters at key network nodes and copying all traffic to a bypass analysis cluster. However, this requires huge equipment investment, consumes a lot of computing and storage resources, has low analysis efficiency, and cannot guide and filter traffic for specific targets in real time, dynamically, and accurately according to security requirements. It also lacks flexibility and efficient resource utilization.

[0004] Traffic sampling and statistical analysis solutions based on NetFlow / IPFIX monitor flow metadata generated by network device sampling. Although they are simple to deploy and have low load, the sampling ratio is too low, resulting in insufficient traffic coverage. Attacks are easily missed, and the data information is too limited to conduct effective in-depth attack analysis and forensic tracing.

[0005] The high-level traffic dynamic tracking and acquisition technology for key targets uses the BGP FlowSpec protocol to dynamically track high-level traffic for key targets and leverages hardware acceleration to complete line-rate matching of complex rules. However, it has significant shortcomings in terms of refined monitoring, collaborative mechanisms, and multi-scenario adaptation. For example, it lacks refined monitoring commands and there is no correlation mechanism between filtering and monitoring; it only supports basic traffic packet feature matching and lacks the ability to specifically monitor encrypted traffic and DNS traffic; PCAP only supports real-time acquisition and has no backtracking function; it does not explicitly support large-scale concurrent commands; and the filtered traffic is directly output, which cannot adapt to the differentiated needs of multi-user scenarios. Summary of the Invention

[0006] The purpose of this invention is to provide a fixed network traffic filtering and multi-dimensional monitoring method and system based on instruction association, which can solve the problems of fragmented traffic filtering and monitoring mechanisms, lack of dedicated monitoring capabilities for encrypted traffic and DNS traffic, and inability to trace back PCAP files as needed in the current fixed network traffic security monitoring, resulting in a single overall monitoring dimension.

[0007] To achieve the above objectives, the fixed network traffic fine-grained filtering and multi-dimensional monitoring method based on command association described in this invention includes the following steps: S1, associates filtering instructions and monitoring instructions by filtering instruction ID and user ID; S2, based on the monitoring instructions, determines the corresponding filtering instruction ID and user ID; and distributes the filtered traffic that meets the corresponding filtering instruction ID to the traffic monitoring engine in real time through the output channel of the user ID. S3, save the filtered traffic as a PCAP file, and simultaneously extract the metadata of the filtered traffic to build an index; the metadata includes the filtering instruction ID, user ID, 5-tuple, and timestamp; S4 quickly retrieves the index based on the PCAP query command, extracts the corresponding PCAP file, and reports it via SFTP.

[0008] Furthermore, the traffic monitoring engine includes a NetFlow sub-monitoring engine, a protocol metadata sub-monitoring engine, a malicious packet sub-monitoring engine, a malicious resource sub-monitoring engine, a malicious file sub-monitoring engine, an encrypted traffic sub-monitoring engine, and a DNS sub-monitoring engine.

[0009] Furthermore, the monitoring instruction includes a monitoring type field, which is used to determine the sub-monitoring engine that executes the monitoring instruction.

[0010] Furthermore, the monitoring type field supports multiple items in parallel, simultaneously triggering multiple sub-monitoring engines to execute monitoring instructions.

[0011] Furthermore, the encrypted traffic sub-monitoring engine extracts encrypted traffic features, including JA3, JA3S, JA4, or JA4S fingerprints. After the encrypted traffic features are matched with the monitoring instructions, encrypted traffic security monitoring reporting information is generated.

[0012] Furthermore, the encrypted traffic monitoring engine also includes parsing the X.509 encryption protocol certificate, calculating the certificate MD5, and extracting the certificate subject, issuer, and validity period.

[0013] Furthermore, the encrypted traffic monitoring engine also includes extracting the server's public key and calculating its MD5 hash for SSH traffic.

[0014] Furthermore, the DNS sub-monitoring engine supports the detection of DNS tunneling, Fast Flux, and DNS attacks involving domain name fronting; including parsing DNS query messages to extract the queried domain name; parsing DNS response messages to extract the IP address of the resolution result; identifying DNS tunneling characteristics; and identifying Fast Flux characteristics.

[0015] The present invention discloses a fixed network traffic fine-grained filtering and multi-dimensional monitoring system based on command association, comprising a command platform, an interface server, a traffic filtering and monitoring module, a PCAP index backtracking module, and a PCAP storage server; The command platform is used for human-computer interaction and visualization of command filtering, command monitoring, PCAP query commands, and execution results. The interface server is used to parse filtering instructions, monitoring instructions, and PCAP query instructions, establish association mappings between instructions according to preset instruction association rules, provide PCAP file query and index management functions, and standardize and encapsulate feedback results and report them on demand. The traffic filtering and monitoring module includes a BGP routing module, a traffic filtering module, and a traffic monitoring engine. The BGP routing module enables dynamic routing of fixed network traffic, and the traffic filtering module mirrors and replicates the routed traffic. Line-speed matching is completed based on the filtering rules of the filtering instructions. Traffic that matches the filtering rules is allocated to the corresponding output channel according to the user ID and distributed to the traffic monitoring engine in real time to complete monitoring and analysis. The PCAP index backtracking module names PCAP files according to preset rules, establishes a PCAP file index, responds to PCAP query commands, and extracts the corresponding PCAP files.

[0016] The advantage of this invention lies in associating filtering and monitoring commands by using a filtering command ID and a user ID. The filtered traffic is then directly sent to the traffic monitoring engine via the associated user ID output channel, solving the problem of separation between filtering and monitoring. The traffic monitoring engine of this invention integrates a NetFlow sub-monitoring engine, a protocol metadata sub-monitoring engine, a malicious packet sub-monitoring engine, a malicious resource sub-monitoring engine, a malicious file sub-monitoring engine, an encrypted traffic sub-monitoring engine, and a DNS sub-monitoring engine, enabling full-scenario monitoring through a single command platform. Compared with existing technologies, this invention solves the problems of encrypted traffic and DNS-specific monitoring capabilities, and also supports PCAP indexing and on-demand query backtracking, providing high-quality, traceable traffic data to support advanced threat analysis, attack tracing, and emergency response. Attached Figure Description

[0017] Figure 1 This is the flowchart of the fixed network traffic fine-grained screening and multi-dimensional monitoring method based on instruction association described in this invention.

[0018] Figure 2 This is a diagram of the fixed network traffic fine-grained filtering and multi-dimensional monitoring system based on instruction association described in this invention. Detailed Implementation

[0019] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] The present invention describes a method for refined filtering and multi-dimensional monitoring of fixed network traffic based on instruction association, such as... Figure 1 As shown, it includes the following steps: S1 associates filtering instructions and monitoring instructions with filtering instruction ID and user ID.

[0021] Construct a filtering instruction table and a monitoring instruction association table, as shown in Table 1 and Table 2: Table 1 Filtering Instructions Table 2 Monitoring Command Association Table Upon receiving a monitoring instruction, the system first queries the associated filtering instruction ID and associated user ID within the monitoring instruction to determine the validity of the association between the filtering instruction and the monitoring instruction. Then, it establishes and saves the mapping relationship between the filtering instruction and the monitoring instruction, thereby achieving automatic binding between traffic filtering instructions and refined monitoring instructions without manual intervention.

[0022] S2, according to the monitoring instructions, distributes the filtered traffic that meets the corresponding filtering instruction ID to the traffic monitoring engine in real time through the user ID output channel, so that the filtered traffic automatically and accurately enters the preset monitoring task, avoiding the complexity and error rate of manual configuration, and realizing a closed loop of screening as needed and measurement as required.

[0023] The traffic monitoring engine includes the NetFlow sub-monitoring engine, protocol metadata sub-monitoring engine, malicious packet sub-monitoring engine, malicious resource sub-monitoring engine, malicious file sub-monitoring engine, encrypted traffic sub-monitoring engine, and DNS sub-monitoring engine.

[0024] The monitoring instruction includes a monitoring type field. The value of the monitoring type field determines the sub-monitoring engine that will execute the monitoring instruction. The monitoring type field supports multiple entries, which can be separated by commas to trigger multiple sub-monitoring engines to execute the monitoring instruction simultaneously.

[0025] NetFlow monitoring generates NetFlow logs according to the configured sampling ratio (1:1 to 10000:1), and supports 1:1 full-flow sampling.

[0026] Protocol metadata monitoring extracts protocol metadata based on the protocol type (HTTP, DNS, TLS, etc.) specified by the outputType field.

[0027] Malicious packet monitoring performs in-depth inspection of traffic packets by loading snortRule or mtxRule.

[0028] Malicious resource monitoring detects malicious network resources based on conditions such as URL, domain name, IP, port, and protocol type.

[0029] Malicious file detection is based on malwareMd5List or yaraDescription to detect malicious files, and supports sample extraction and reporting.

[0030] Encrypted traffic monitoring extracts encrypted traffic characteristics, including JA3, JA3S, JA4, or JA4S fingerprints. Upon a match between these characteristics and monitoring commands, it generates encrypted traffic security monitoring reports. Simultaneously, it parses X.509 encryption protocol certificates, calculates their MD5 hashes, and extracts the certificate subject, issuer, and validity period. For SSH traffic, it extracts the server's public key and calculates its MD5 hash to identify malicious encrypted communication. By combining encrypted traffic characteristics with filtering rules, this invention effectively identifies malicious encrypted communication (such as C2 backlinks and encrypted tunnels), filling a gap in existing technologies for encrypted traffic monitoring.

[0031] DNS monitoring detects abnormal DNS behavior based on dnsIp (DNS server address) and dnsHost (DNS domain name). It extracts the queried domain name by parsing DNS query messages; extracts the IP address of the resolution result by parsing DNS response messages; identifies DNS tunnel characteristics; and identifies Fast Flux characteristics, supporting the detection of DNS tunneling, Fast Flux, and DNS attacks involving domain name fronting. This invention performs deep inspection only on filtered DNS traffic, balancing detection depth and processing performance.

[0032] The monitoring results of each monitoring sub-engine are uniformly packaged and reported to the command platform via HTTPS channel.

[0033] This invention unifies six monitoring capabilities—NetFlow, protocol metadata, malicious packets, malicious files, encrypted traffic, and DNS monitoring—under a single command framework. By flexibly combining monitoring type fields, it enables one-time filtering, multi-dimensional monitoring, and integrated output, significantly improving monitoring efficiency and coverage.

[0034] S3 saves the filtered traffic as a PCAP file, named according to the rule "ruleID_operator code_province_device number_start time_end time_auto-incrementing serial number". It also simultaneously extracts metadata from the filtered traffic, including key information such as the filtering instruction ID, user ID, 5-tuple, and timestamp, to build an index. This solves the problem of real-time data collection but difficulty in post-event investigation in traditional solutions, providing data support for in-depth forensics of security incidents.

[0035] S4, when receiving a PCAP query command, quickly retrieves the index according to the PCAP query command, extracts the corresponding PCAP file, compresses it into tar.gz format, and reports it via SFTP.

[0036] The present invention describes a fixed network traffic fine-grained filtering and multi-dimensional monitoring system based on command association, such as... Figure 2 As shown, it includes an instruction platform, an interface server, a traffic filtering and monitoring module, a PCAP index backtracking module, and a PCAP storage server; The command platform is used for human-computer interaction and visualization of command filtering, command monitoring, PCAP query commands, and execution results. The interface server runs on a general-purpose CPU and is used to parse filtering instructions, monitoring instructions, and PCAP query instructions; establish association mappings between instructions according to preset instruction association rules; provide PCAP file query and index management functions; and standardize and encapsulate feedback results and report them on demand. The traffic filtering and monitoring module includes a BGP routing module, a traffic filtering module, and a traffic monitoring engine. The BGP routing module uses a programmable interactive chip to realize dynamic routing, route announcement, and lossless re-injection of fixed network traffic. The traffic filtering module relies on DPU / FPGA hardware acceleration to mirror and copy the traffic after routing, and completes line-speed matching based on the filtering rules of the filtering instructions. The traffic that matches the filtering rules is allocated to the corresponding output channel according to the user ID and distributed to the traffic monitoring engine in real time to complete monitoring and analysis. The PCAP index backtracking module names PCAP files according to preset rules, establishes a PCAP file index, responds to PCAP query commands, and extracts the corresponding PCAP files.

[0037] The fixed-line traffic fine-grained filtering and multi-dimensional monitoring system described in this invention can be deployed in national-level regulatory agencies to provide collaborative fixed-line security monitoring capabilities. It enhances the ability to dynamically filter traffic based on IP addresses, domain names, and attack characteristics, and performs fine-grained monitoring and analysis. It can also transmit PCAP files back as needed, providing source data for tracing. Furthermore, it can be applied to specific enterprises / organizations (such as telecom operators, large internet companies, and financial institutions) to identify network security threats to key targets, such as filtering traffic from key targets, identifying and analyzing potential risks in hardware and software assets, important systems, and network services within the traffic, detecting encrypted traffic threats, identifying DNS attacks, and extracting malicious files. Simultaneously, for end users (such as enterprise employees and individual consumers) or specific business applications, this invention can prevent communication network fraud, trace network attacks, and meet the needs of network security supervision, source tracing, and risk prevention.

Claims

1. A method for refined filtering and multi-dimensional monitoring of fixed network traffic based on command association, characterized in that, Includes the following steps: S1, associates filtering instructions and monitoring instructions by filtering instruction ID and user ID; S2, based on the monitoring instructions, determines the corresponding filtering instruction ID and user ID; and distributes the filtered traffic that meets the corresponding filtering instruction ID to the traffic monitoring engine in real time through the output channel of the user ID. S3, save the filtered traffic as a PCAP file, and simultaneously extract the metadata of the filtered traffic to build an index; the metadata includes the filtering instruction ID, user ID, 5-tuple, and timestamp; S4 quickly retrieves the index based on the PCAP query command, extracts the corresponding PCAP file, and reports it via SFTP.

2. The method for refined filtering and multi-dimensional monitoring of fixed network traffic based on command association as described in claim 1, characterized in that: The traffic monitoring engine includes a NetFlow sub-monitoring engine, a protocol metadata sub-monitoring engine, a malicious packet sub-monitoring engine, a malicious resource sub-monitoring engine, a malicious file sub-monitoring engine, an encrypted traffic sub-monitoring engine, and a DNS sub-monitoring engine.

3. The method for refined filtering and multi-dimensional monitoring of fixed network traffic based on instruction association as described in claim 2, characterized in that: The monitoring instruction includes a monitoring type field, which is used to determine the sub-monitoring engine that executes the monitoring instruction.

4. The method for refined filtering and multi-dimensional monitoring of fixed network traffic based on instruction association as described in claim 3, characterized in that: The monitoring type field supports multiple items in parallel, which can trigger multiple sub-monitoring engines to execute monitoring commands simultaneously.

5. The method for refined filtering and multi-dimensional monitoring of fixed network traffic based on instruction association as described in claim 2, characterized in that: The encrypted traffic monitoring engine extracts encrypted traffic features, including JA3, JA3S, JA4 or JA4S fingerprints. After the encrypted traffic features are matched with the monitoring command, encrypted traffic security monitoring reporting information is generated.

6. The method for refined filtering and multi-dimensional monitoring of fixed network traffic based on instruction association as described in claim 5, characterized in that: The encrypted traffic monitoring engine also includes parsing X.509 encryption protocol certificates, calculating the certificate MD5, and extracting the certificate subject, issuer, and validity period.

7. The method for refined filtering and multi-dimensional monitoring of fixed network traffic based on instruction association as described in claim 5, characterized in that: The encrypted traffic monitoring engine also includes extracting the server's public key and calculating its MD5 hash for SSH traffic.

8. The method for refined filtering and multi-dimensional monitoring of fixed network traffic based on instruction association as described in claim 5, characterized in that: The DNS sub-monitoring engine supports the detection of DNS tunneling, Fast Flux, and DNS attacks involving domain name fronting. This includes parsing DNS query messages to extract the queried domain name; parsing DNS response messages to extract the IP address of the resolution result; and identifying DNS tunnel characteristics. Identify Fast Flux features.

9. A system for refined filtering and multi-dimensional monitoring of fixed network traffic based on command association, characterized in that: It includes a command platform, an interface server, a traffic filtering and monitoring module, a PCAP index backtracking module, and a PCAP storage server; The command platform is used for human-computer interaction and visualization of command filtering, command monitoring, PCAP query commands, and execution results. The interface server is used to parse filtering instructions, monitoring instructions, and PCAP query instructions, establish association mappings between instructions according to preset instruction association rules, provide PCAP file query and index management functions, and standardize and encapsulate feedback results and report them on demand. The traffic filtering and monitoring module includes a BGP routing module, a traffic filtering module, and a traffic monitoring engine. The BGP routing module enables dynamic routing of fixed network traffic, and the traffic filtering module mirrors and replicates the routed traffic. Line-speed matching is completed based on the filtering rules of the filtering instructions. Traffic that matches the filtering rules is allocated to the corresponding output channel according to the user ID and distributed to the traffic monitoring engine in real time to complete monitoring and analysis. The PCAP index backtracking module names PCAP files according to preset rules, establishes a PCAP file index, responds to PCAP query commands, and extracts the corresponding PCAP files.

Images