Malicious encrypted traffic detection system based on network traffic acquisition equipment

Abstract

The invention discloses a malicious encrypted traffic detection system based on network traffic acquisition equipment, and aims to solve the problems that existing commercial detection equipment is high in cost, poor in customization and incomplete in function when an open source tool is independently used. The system adopts a five-layer architecture: collecting, filtering and slicing TLS traffic by using dawn SUNA-R10000, and analyzing a multi-dimensional TLS field through a Tshark custom Lua plug-in; a malicious feature library is constructed through a Redis cluster to realize static matching, and dynamic anomaly detection is completed by means of an Flink deployment fusion model; and finally, fusing double detection results by 7: 3 weighted voting, judging malicious traffic and triggering blocking and alarming. The method integrates hardware and an open source tool chain, is low in cost and customizable, supports real-time detection of a 10Gbps link, gives consideration to known and unknown threats, and meets encrypted traffic defense requirements of small and medium-sized enterprises.

Description

Technical Field

[0001] This invention belongs to the field of encrypted traffic detection, specifically a malicious encrypted traffic detection system based on network traffic collection equipment. Background Technology

[0002] With the rapid development of network technology, encrypted communication is not only a crucial safeguard for enterprise data transmission security, but it is also being exploited by malicious attackers. Malicious encrypted traffic has become a major threat to enterprise network security. Current methods for detecting malicious encrypted traffic largely rely on commercial security equipment such as next-generation firewalls (NGFWs) and intrusion detection systems (IDS), which, while possessing some detection capabilities, have significant limitations.

[0003] First, the high cost, including hardware, software licensing, and maintenance, makes it unaffordable for small and medium-sized enterprises (SMEs), preventing them from deploying effective detection equipment. Second, the difficulty in customization, with closed detection rules and algorithms, prevents users from flexibly adjusting them according to their network environment and business needs, resulting in delayed adaptation and poor detection performance when facing special threats or new attacks. Third, poor compatibility with the open-source ecosystem, making it impossible to fully utilize the rich resources of open-source tools such as Tshark and Suricata, limiting the improvement of detection capabilities. Fourth, insufficient detection of unknown threats, relying heavily on the detection of known malicious signatures, makes it difficult to cope with the ever-emerging new malicious encrypted traffic, leading to vulnerabilities in protection.

[0004] The Sugon SUNA-R10000, a high-performance network traffic acquisition device, boasts powerful packet capture and processing capabilities, enabling real-time collection and filtering of large-scale network traffic. However, this device focuses on traffic acquisition and preprocessing, lacking fine-grained TLS field parsing and intelligent malicious encrypted traffic detection capabilities, thus failing to directly meet enterprise detection needs. Tshark, an open-source command-line protocol analysis tool, is often used in conjunction with Redis—Redis can store malicious IPs, domain names, and other feature libraries, providing fast feature matching—but it only enables static storage and querying, unable to analyze dynamic traffic or detect anomalies in real time. Flink, an open-source distributed stream processing framework, can perform dynamic anomaly detection operations such as preprocessing and model inference on real-time TLS traffic, but it lacks its own traffic acquisition and protocol parsing capabilities, requiring collaboration with other devices and tools to build a complete malicious encrypted traffic detection system.

[0005] In summary, existing technologies often lack intelligent detection capabilities for dedicated traffic acquisition devices. While open-source tools have advantages in their respective fields, their individual use cannot meet the comprehensive needs of enterprises for detecting malicious encrypted traffic. Therefore, this solution deeply integrates the Sugon SUNA-R10000 high-performance traffic acquisition device with open-source toolchains such as Tshark, Redis, and Flink, leveraging their respective strengths to build a low-cost, customizable, efficient, and accurate malicious encrypted traffic detection solution. This fills the technological gap for small and medium-sized enterprises in encrypted traffic defense, providing them with reliable network security protection. Summary of the Invention

[0006] The purpose of this invention is to provide a malicious encrypted traffic detection solution based on Sugon SUNA-R10000, Tshark, Redis, and Flink:

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A malicious encrypted traffic detection system based on network traffic collection devices includes:

[0009] Traffic Acquisition Layer: The Sugon SUNA-R10000 device is used to collect enterprise Internet traffic via bypass mirroring, which is then used to filter and output TLS packet slices.

[0010] Field parsing layer: Deploy the Tshark tool, which contains a custom Lua plugin; the custom Lua plugin is used to parse the TLS packet slices to extract multi-dimensional TLS fields;

[0011] Static detection layer: A malicious feature library is built based on a Redis cluster, and the Suricata rule engine is used to perform feature matching on the multi-dimensional TLS fields to obtain static matching results;

[0012] The features to be matched include JA3 fingerprints and malicious domain names;

[0013] Dynamic detection layer: Deploy an isolated forest and autoencoder fusion model using the Apache Flink stream processing framework to calculate the anomaly degree of the multi-dimensional TLS fields and obtain dynamic detection results;

[0014] Result fusion layer: Summarizes the static matching results and the dynamic detection results, and outputs the final judgment conclusion using a weighted voting mechanism.

[0015] Furthermore, the configuration of the Sugon SUNA-R10000 device includes:

[0016] Enable the TLS protocol recognition rule: to filter out the traffic of target ports including 443, 853, and 8443;

[0017] Enable the packet slicing function: slice the TLS protocol packets and only retain the first 1024 bytes of the ClientHello and ServerHello messages in the TLS handshake phase;

[0018] Traffic load balancing distribution: Through the built-in intelligent load balancing algorithm, according to the load conditions of the Tshark parsing nodes, evenly distribute the sliced TLS traffic to multiple Tshark parsing nodes.

[0019] Furthermore, the parsing process of the custom Lua plugin for slicing the TLS packets includes:

[0020] Locate the ClientHello message and extract the cliVer field and cliCipSui field; the cliVer field is used to identify the client TLS version; the cliVer field is used to identify the client cipher suite;

[0021] Parse to obtain the cliExt field; the cliExt field is used to record the extension list in the TLS handshake message;

[0022] Extract the srvCertHashes field and cliRand, and call the OpenSSL library to calculate the entropy values of srvCertHashes and cliRand; the srvCertHashes is used to identify the SHA-256 fingerprint of the server certificate; the cliRand is the random number generated and exchanged by the client and the server respectively in the TLS handshake phase;

[0023] Locate the ServerHello message and extract the srvEllCur field and srvCipSui field; the srvEllCur field is used to identify the server elliptic curve, and the srvCipSui field is used to identify the server cipher suite.

[0024] Furthermore, the malicious feature library includes a malicious JA3 fingerprint set, a malicious domain name set, and weak cipher suite rules;

[0025] The malicious JA3 fingerprint set stores known malicious cliJA3 and srvJA3 fingerprints in the Redis string type, and the storage adopts the key-value format of <ja3: malicious fingerprint name>;

[0026] The malicious domain name set stores malicious domain names in the Redis set type and matches malicious domain names through the SMISMEMBER command;

[0027] The weak cipher suite rules use a Redis hash type to store the abnormal mapping relationship between the TLS version and the cipher suite.

[0028] Furthermore, the operators used in the dynamic detection process of the Apache Flink stream processing framework include stream processing operators, feature engineering operators, and model inference operators;

[0029] The stream processing operator is used to clean the multi-dimensional TLS field, and the cleaning operation includes filtering null values ​​and format conversion;

[0030] The feature engineering operator is used to calculate the Z-score value of the cliExt field and the cosine similarity between the cliCipSui field and the normal baseline.

[0031] The model inference operator is used to load a pre-trained isolated forest model and output anomaly scores.

[0032] A method for implementing a detection system includes the following steps:

[0033] S1: SUNA-R10000 collects and filters TLS traffic, and pushes the data packets to the Tshark resolution node via TCP protocol;

[0034] S2: Tshark parses 23 target TLS fields and outputs them to the Kafka message queue in JSON format;

[0035] S3: The Redis client consumes data from Kafka, queries the malicious signature database, and returns static matching results;

[0036] S4: Flink reads field data from Kafka, calculates anomaly scores using a dynamic model, and generates dynamic detection results;

[0037] S5: Flink scores the static matching results and dynamic detection results separately, and the scores are combined with a weight of 7:3. If the score is ≥0.6, it is judged as malicious traffic.

[0038] Furthermore, the JSON format fields mentioned in step S2 include basic fields and TLS feature fields;

[0039] The basic fields include: timestamp, destination IP, and source IP;

[0040] The TLS feature fields include: cliVer, cliCipSui, srvName, cliRand, cliExt, and srvCertHashes and srvName; where srvName represents the server domain name.

[0041] Furthermore, the anomaly calculation in step S4 includes:

[0042] Calculate the deviation rate of the cliExt field from the mean of the normal baseline. When the deviation rate is greater than 200%, it is considered an anomaly.

[0043] The 23 target TLS fields after one-hot encoding are input into the isolated forest model and rule engine to output anomaly probabilities. When the probability is greater than 0.8, it is recorded as high risk.

[0044] Furthermore, it also includes the following malicious signature database update steps:

[0045] Malicious JA3 fingerprints are added daily from the MISP threat intelligence platform and written to the Redis cluster in batches via Redis Pipeline;

[0046] The normal baseline is updated weekly using Flink's window computation.

[0047] Furthermore, step S5 also includes outputting the result:

[0048] For traffic deemed malicious, a HTTP POST request is used to notify the firewall to block the source IP.

[0049] Record static matching results and dynamic detection results, and visualize them.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] Cost advantage: Replacing commercial detection engines with open-source toolchains reduces hardware costs by more than 60%;

[0052] Highly flexible: Tshark parsing logic can be extended via Lua scripts to adapt to new TLS fields;

[0053] Controllable performance: The high-speed acquisition of SUNA-R10000 and the parallel computing of Flink support real-time detection of 10Gbps links.

[0054] Scalability: Supports horizontal scaling of Tshark parsing nodes and Flink TaskManager to adapt to traffic growth. Attached Figure Description

[0055] Figure 1 This is a system hierarchy diagram of the present invention;

[0056] Figure 2 This is a system workflow diagram of the present invention;

[0057] Figure 3 This is a diagram showing the system detection results of the present invention; Detailed Implementation

[0058] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] A malicious encrypted traffic detection system based on network traffic collection devices includes:

[0060] Traffic Acquisition Layer: The Sugon SUNA-R10000 device is used to collect enterprise Internet traffic via bypass mirroring, which is then used to filter and output TLS packet slices.

[0061] Field parsing layer: Deploy the Tshark tool, which contains a custom Lua plugin; the custom Lua plugin is used to parse the TLS packet slices to extract multi-dimensional TLS fields;

[0062] Static detection layer: A malicious feature library is built based on a Redis cluster, and the Suricata rule engine is used to perform feature matching on the multi-dimensional TLS fields to obtain static matching results;

[0063] The features to be matched include JA3 fingerprints and malicious domain names;

[0064] Dynamic detection layer: Deploy an isolated forest and autoencoder fusion model using the Apache Flink stream processing framework to calculate the anomaly degree of the multi-dimensional TLS fields and obtain dynamic detection results;

[0065] Result fusion layer: Summarizes the static matching results and the dynamic detection results, and outputs the final judgment conclusion using a weighted voting mechanism.

[0066] I. System Architecture

[0067] (a) Traffic Acquisition Layer

[0068] (1) Hardware configuration

[0069] The Sugon SUNA-R10000 device is used as the core data acquisition hardware. Its specific configuration and features are as follows:

[0070] Port configuration: Equipped with four 10Gbps optical ports to meet the needs of large-scale network traffic collection.

[0071] Protection features: Supports Bypass function, automatically switches to Bypass mode in case of device failure or power failure, ensuring uninterrupted enterprise network communication and guaranteeing business continuity.

[0072] (2) Core functions

[0073] Traffic Acquisition: By using port mirroring, bidirectional traffic (inflow and outflow from the enterprise network) of the enterprise's egress gateway is mirrored to the acquisition port of the Sugon SUNA-R10000 device without affecting normal network communication, and all traffic data can be obtained comprehensively and accurately.

[0074] L7 Protocol Identification and Filtering: Enable the device's built-in L7 protocol identification function, analyze traffic based on deep packet inspection (DPI) technology, and accurately identify TLS protocol traffic; perform dual filtering by combining port and protocol characteristics, first filtering through common TLS ports (443, 853, 8443), and then filtering through TLS handshake message format, field structure, etc., to retain only real TLS protocol traffic.

[0075] Packet slicing: Only the first 1024 bytes of the filtered TLS packets are retained. This part contains key negotiation information from the ClientHello and ServerHello messages during the TLS handshake phase, such as the TLS version, cipher suite, and extended fields. This can meet the needs of subsequent parsing while reducing data transmission and storage, and reducing the pressure on subsequent modules.

[0076] Traffic load balancing distribution: Through the device's built-in intelligent load balancing algorithm, the sliced ​​TLS traffic is evenly distributed to the three Tshark resolution nodes according to the load of the Tshark resolution nodes, avoiding excessive load on a single node and improving the overall processing capacity and stability of the system.

[0077] (ii) Field parsing layer

[0078] (1) Software configuration

[0079] Core parsing tool: Deploy Tshark version 3.6.0.

[0080] Feature extension: Develop and deploy a custom Lua plugin (tls_parser.lua) to extend Tshark parsing capabilities.

[0081] Supported library: Configures OpenSSL version 1.1.1 for deep parsing and cryptographic calculation of TLS certificates.

[0082] (2) Core functions

[0083] Packet reception and parsing command execution: The Tshark parsing node receives sliced ​​TLS packets pushed by the Sugon SUNA-R10000 device in real time and immediately executes the following parsing commands:

[0084] tshark -r - -T json -X lua_script:tls_parser.lua

[0085] Here, -r - indicates reading data packets from standard input, -T json indicates outputting the parsed results in JSON format, and -X lua_script:tls_parser.lua indicates loading a custom Lua plugin.

[0086] Lua plugin parsing includes the following functions:

[0087] First, extracting ClientHello message fields. Locate the ClientHello message and parse the tls.handshake.version field to extract the cliVer field, which identifies the client's TLS version and is formatted using standardization. Parse the tls.handshake.cipher_suites field to extract the cliCipSui field, formatted as an integer array, which identifies the client's cipher suite.

[0088] Secondly, extended field parsing and statistics. The `tls.handshake.extensions` field is parsed, the total number of extended fields `cliExt_count` is counted, and key extension types are identified and recorded, such as SNI extension type 0, ALPN extension type 16, signature algorithm extension type 13, etc.

[0089] Third, cliRand entropy calculation. The OpenSSL library's RAND_check function is called to perform randomness checks on the cliRand field and calculate the entropy value (cli_rand_entropy), recording it as a floating-point number as a dynamic detection feature. The cliRand field is parsed from tls.handshake.random.

[0090] Fourth, extract the ServerHello message fields. Extract srvEllCur, which identifies the server's elliptic curve and corresponds to the tls.handshake.elliptic_curve field; extract srvCipSui, which identifies the cipher suite selected by the server and corresponds to the tls.handshake.cipher_suite field, with a single value type.

[0091] Fifth, JSON encapsulation and parsing results: All extracted and calculated fields are integrated and encapsulated into a JSON format containing 23 fields, which correspond one-to-one with the multi-dimensional TLS fields to ensure data integrity and accuracy.

[0092] (III) Static Detection Layer

[0093] (1) Software configuration

[0094] Redis Cluster: Deploy Redis version 6.2.0, using a 3-master 3-slave architecture. The master nodes are responsible for data writing and querying, while the slave nodes synchronize data in real time and support failover, ensuring high availability and data reliability.

[0095] Rule engine: Deploys Suricata version 6.0.9, an open-source, high-performance intrusion detection and prevention system that supports multi-threaded processing and provides rule support.

[0096] (2) Core functions

[0097] The Redis cluster includes a malicious signature database, which is designed as follows:

[0098] Firstly, a malicious JA3 fingerprint set. This is stored using a Redis string type, and the command format used for storage is as follows:

[0099] SET ja3:malicious fingerprint name "JA3 fingerprint value"

[0100] For example, SET ja3:emotet "abc123...".

[0101] The query command is as follows:

[0102] GET ja3:malicious fingerprint name

[0103] It's important to note that `cliJA3` is a command-line tool used to calculate or parse JA3 fingerprints. JA3 is an algorithm for identifying TLS client fingerprints. It extracts key fields from the TLS client handshake phase (ClientHello message) to generate a unique hash value, which is used to identify the client's TLS behavioral characteristics, such as browser type, operating system version, and unique TLS configurations of malware.

[0104] Secondly, a malicious domain set. This is stored using a Redis Set type, and the storage command is:

[0105] SADD malicious_domains "malicious domains"

[0106] For example, SADD malicious_domains "xn--80ak6aa92e.com".

[0107] The command to use is:

[0108] SMISMEMBER malicious_domains "domains to be detected"

[0109] If the command returns 1, the domain name to be detected is determined to be a malicious domain name.

[0110] Third, weak password suite rules. Use Redis hash type for storage; the storage command format is:

[0111] HSET weak_ciphers "cipherset identifier" "cipherset name"

[0112] For example, HSET weak_ciphers "0x0005" "RC4-MD5".

[0113] The command to use is:

[0114] HGET weak_ciphers "cipher suite identifiers"

[0115] Use commands to check if it is a weak cipher suite.

[0116] The Redis cluster includes the Suricata rule; an example of this rule is given below:

[0117] Example 1: Malicious JA3 fingerprint matching rules

[0118] The `alert tls any any -> any any (msg:"malicious JA3 fingerprint matching"; tls.client_ja3:abc123...; sid:100001; rev:1;)` command is used to alert tls any any -> any any (msg:"malicious JA3 fingerprint matching"; tls.client_ja3:abc123...; sid:100001; rev:1;).

[0119] Meaning: Detect all TLS traffic. If the client's JA3 fingerprint is equal to "abc123...", trigger an alarm and record the rule ID (sid:100001) and version (rev:1).

[0120] Example 2: Missing SNI extended detection rules:

[0121] The `alert tls any any -> any any (msg:"Missing SNI extension"; tls.handshake.extension.type != 0; sid:100002; rev:1;)` command alerts the user to any any of the SNI extensions.

[0122] Meaning: Detect all TLS traffic. If the TLS handshake extension field type is not equal to 0 (0 corresponds to the SNI extension type), trigger a "Missing SNI extension" alarm and record the rule ID and version.

[0123] (iv) Dynamic Detection Layer

[0124] (1) Software configuration

[0125] Stream processing framework: Deploy Apache Flink version 1.15.0, responsible for real-time TLS traffic data preprocessing, feature engineering, and model inference.

[0126] Message queue: Deploy Kafka version 3.2.0 to receive JSON parsing results from Tshark, providing stable data input for Flink.

[0127] Machine learning framework: Deploy TensorFlow version 2.9.0 and train a fusion model of isolated forest and autoencoder as the core of dynamic detection.

[0128] (2) Core functions

[0129] The dynamic detection layer includes data access, feature preprocessing, and model inference functions.

[0130] The data access process is as follows:

[0131] Flink consumes JSON-formatted TLS field data from Kafka topics such as tls_features through a Kafka connector; it sets up parallel consumption instances based on the number of topic partitions, supports exactly-Once semantics, and ensures real-time data acquisition and reliable consumption.

[0132] Feature preprocessing includes the following operations:

[0133] First, invalid data filtering. Data with empty cliVer, empty cliCipSui, or incorrect src_ip / dst_ip formats is filtered to avoid affecting the accuracy of subsequent analysis. src_ip / dst_ip represents the source IP / destination IP.

[0134] Secondly, cliCipSui one-hot encoding. Common enterprise cipher suites are statistically analyzed to determine the encoding dimensions. The cliCipSui array is then one-hot encoded, with the dimension corresponding to the included cipher suite marked as 1 and the rest as 0, thus converting discrete information into continuous numerical features.

[0135] Third, the edit distance calculation between cliExt and the normal baseline: Treat cliExt as a string (e.g., "0,16,13"), establish a normal baseline based on normal TLS traffic, and calculate the edit distance between the cliExt of the traffic to be detected and the baseline. The larger the distance, the greater the difference, which may indicate an anomaly. cliExt represents a list of client extended field types.

[0136] The model inference process is as follows:

[0137] First, outlier detection is performed using an isolated forest model. High-dimensional features such as the Z-score of the one-hot encoded cliCipSui and cliExt_count, cli_rand_entropy, and cliExt edit distance are input. Data is isolated using multiple isolated trees to identify outliers that significantly differ from normal traffic; these outliers are considered abnormal traffic.

[0138] Then, a rule engine is used for anomaly detection. Rules are set based on the experience of security experts and attack cases, such as cli_rand_entropy < 4.0 indicating weak randomness of random numbers, srvCertLen < 2048 bits indicating low certificate encryption strength, and TLS version 1.0 / 1.1 indicating the existence of security vulnerabilities, to quickly identify obvious abnormal characteristics.

[0139] Finally, a dynamic anomaly score is output. Combining the detection results from the Isolation Forest model with the judgment results from the rule engine, a dynamic anomaly score of 0-1 is generated. A higher score indicates a higher degree of anomaly, potentially indicating malicious traffic.

[0140] (v) Results Fusion Layer

[0141] (1) Fusion Logic

[0142] First, calculate the static score. A binary scoring method is used: 1 point is awarded for a successful match between the TLS fields of the traffic to be detected (cliJA3 fingerprint, srvName, cliCipSui, etc.) and Redis malicious features, and 0 points are awarded for a non-match, reflecting the degree of matching with known malicious features.

[0143] Then, the dynamic score is calculated. A segmented scoring method is used: a dynamic anomaly score ≥0.8 receives 1 point (extremely high degree of anomaly), between 0.5 and 0.8 receives 0.5 points (some anomaly exists), and <0.5 receives 0 points (basically consistent with normal traffic).

[0144] Finally, calculate the overall score. A weighted sum is applied, with the static score having a weight of 0.7 (for high accuracy and reliability) and the dynamic score having a weight of 0.3 (to compensate for deficiencies in static detection). The formula is: Overall Score = Static Score × 0.7 + Dynamic Score × 0.3.

[0145] (2) Handling procedures

[0146] Malicious traffic handling: Traffic with a comprehensive score of ≥0.6 is identified as malicious traffic. An HTTP POST request is immediately sent to the firewall to block the traffic (including source IP, destination IP, TLS port, etc.), and an ACL denial rule is added to the firewall. At the same time, detailed information about the malicious traffic (collection time, IP address, TLS version, detection results, etc.) is recorded in the security log for auditing, tracing, and evidence collection.

[0147] Suspicious traffic handling: Traffic with a comprehensive score of 0.4-0.6 is judged as suspicious traffic, an alarm log (including suspicious information and characteristic description) is generated and sent to the security management platform to remind administrators; administrators can view details and analyze historical communication records. If malicious is confirmed, manual blocking is triggered. If normal is confirmed, it is marked as normal to avoid false alarms.

[0148] Normal traffic handling: Traffic with a comprehensive score <0.4 is considered normal traffic. Only statistical information (source IP, destination IP, TLS version, traffic size, collection time, etc.) is recorded for enterprise network traffic analysis and optimization, providing a reference for network planning and security policy formulation.

[0149] II. Key Technical Details

[0150] (a) TLS field mapping relationship

[0151] field name Tshark field path Storage format cliVer tls.handshake.version Strings (such as "TLS 1.2") cliCipSui tls.handshake.cipher_suites Integer arrays (e.g., [0x1301, 0x1302]) srvName tls.handshake.extensions.server_name Strings (such as "example.com") cliRand tls.handshake.random 32-byte binary data → entropy value (floating-point type)

[0152] (II) Redis Cluster Optimization

[0153] Hash slot partitioning is used to allocate malicious domain sets to fixed slots, thereby improving query efficiency;

[0154] Enable persistence (RDB+AOF) to prevent the feature library from being lost due to node failure;

[0155] Configure the maximum memory policy as allkeys-lru to automatically eliminate older features that are accessed infrequently.

[0156] (III) Flink performance tuning

[0157] Parallelism setting: Same as the number of Kafka partitions (e.g., 8 parallel instances);

[0158] State backend: RocksDB is used, which supports large state storage (such as normal baseline statistics).

[0159] Checkpoint interval: 5 minutes, balancing fault tolerance and performance.

[0160] III. Examples

[0161] Scenario: Detecting encrypted communication traffic of Emotet malware within an enterprise's intranet. The detection and response process is as follows:

[0162] 1. The SUNA-R10000 collected TLS traffic from source IP 192.168.1.100, filtered it, and pushed it to Tshark;

[0163] 2. Tshark parsed the following: cliVer=TLSv1.0, cliCipSui=[0x0005] (RC4-MD5), cliJA3=abc123 (matches Emotet fingerprint), srvName=malicious.example.com;

[0164] 3. Redis queries revealed that both cliJA3 and srvName matched malicious characteristics, with a static score of 1.

[0165] 4. Flink calculated the dynamic anomaly score to be 0.9 (due to the use of weak packages and an older version).

[0166] 5. Overall score = 1 × 0.7 + 0.9 × 0.3 = 0.97 ≥ 0.6, therefore it is determined to be malicious traffic;

[0167] 6. The system notifies the firewall to block 192.168.1.100 and generates an alert in Grafana.

[0168] This solution fully verifies the collaborative capabilities of the Sugon SUNA-R10000 and the open-source toolchain, comprehensively covers the core aspects of the detection method described in the patent, and provides enterprises with a cost-effective solution for defending against malicious encrypted traffic.

[0169] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A malicious encrypted traffic detection system based on network traffic collection equipment, characterized in that, include: Traffic Acquisition Layer: The Sugon SUNA-R10000 device is used to collect enterprise Internet traffic via bypass mirroring, which is then used to filter and output TLS packet slices. Field parsing layer: Deploy the Tshark tool, which contains a custom Lua plugin; the custom Lua plugin is used to parse the TLS packet slices to extract multi-dimensional TLS fields; Static detection layer: A malicious feature library is built based on a Redis cluster, and the Suricata rule engine is used to perform feature matching on the multi-dimensional TLS fields to obtain static matching results; The features to be matched include JA3 fingerprints and malicious domain names; Dynamic detection layer: Deploy an isolated forest and autoencoder fusion model using the Apache Flink stream processing framework to calculate the anomaly degree of the multi-dimensional TLS fields and obtain dynamic detection results; Result fusion layer: Summarizes the static matching results and the dynamic detection results, and outputs the final judgment conclusion using a weighted voting mechanism.

2. The system according to claim 1, characterized in that, The configuration of the Dawning SUNA-R10000 device includes: Enable TLS protocol identification rules to filter out traffic to target ports including 443, 853, and 8443; Enable packet slicing: Slice TLS protocol packets, retaining only the first 1024 bytes of the ClientHello and ServerHello messages during the TLS handshake phase; Traffic load balancing distribution: Through the built-in intelligent load balancing algorithm, the sliced ​​TLS traffic is evenly distributed to multiple Tshark resolution nodes according to the load of the Tshark resolution nodes.

3. The system according to claim 1, characterized in that, The parsing process of the TLS packet slice by the custom Lua plugin includes: Locate the ClientHello message and extract the cliVer and cliCipSui fields; the cliVer field is used to identify the client's TLS version; the cliVer field is used to identify the client's cipher suite. The cliExt field is obtained through parsing; the cliExt field is used to record the extended list in the TLS handshake message; Extract the srvCertHashes field and cliRand, and call the OpenSSL library to calculate the entropy values ​​of srvCertHashes and cliRand; srvCertHashes is used to identify the SHA-256 fingerprint of the server certificate; cliRand is a random number generated and exchanged by the client and server during the TLS handshake phase. Locate the ServerHello message and extract the srvEllCur and srvCipSui fields; the srvEllCur field is used to identify the server elliptic curve, and the srvCipSui field is used to identify the server cipher suite.

4. The system according to claim 1, characterized in that, The malicious feature library includes a malicious JA3 fingerprint set, a malicious domain name set, and weak password suite rules; The malicious JA3 fingerprint set stores known malicious cliJA3 and srvJA3 fingerprints in the Redis string type, and the storage uses a key-value format of <ja3: malicious fingerprint name>; The malicious domain name set stores malicious domain names in the Redis set type, and matches malicious domain names through the SMISMEMBER command; The weak password suite rules store the abnormal mapping relationship between the TLS version and the password suite in the Redis hash type.

5. The system according to claim 1, wherein the operators used in the dynamic detection process of the Apache Flink stream processing framework include stream processing operators, feature engineering operators, and model inference operators; The stream processing operator is used to clean the multi-dimensional TLS fields, and the cleaning operations include filtering null values and format conversion; The feature engineering operator is used to calculate the Z-score value of the cliExt field and the cosine similarity between the cliCipSui field and the normal baseline; The model inference operator is used to load a pre-trained isolation forest model and output an anomaly score.

6. A method for implementing the system according to claims 1-5, characterized in that, It includes the following steps: S1: SUNA-R10000 collects and filters TLS traffic, and pushes the data packets to the Tshark parsing node through the TCP protocol; S2: Tshark extracts 23 target TLS fields and outputs them to the Kafka message queue in JSON format; S3: The Redis client consumes data from Kafka, queries the malicious feature library, and returns static matching results; [[ID=一三]]S4: Flink reads the field data from Kafka, calculates the anomaly degree through a dynamic model, and generates a dynamic detection result; S5: Flink scores the static matching result and the dynamic detection result respectively, and the scoring results are fused according to a 7:3 weight. If the score ≥ 0.6, it is determined as malicious traffic.

7. The implementation method according to claim 6, wherein the JSON format fields in step S2 include basic fields and TLS feature fields; The basic fields include: timestamp, destination IP, and source IP; The TLS feature fields include: cliVer, cliCipSui, srvName, cliRand, cliExt, srvCertHashes, and srvName; where srvName represents the server domain name.

8. The method according to claim 6, characterized in that, The anomaly degree calculation in step S4 includes: Calculating the deviation rate of the cliExt field from the normal baseline mean, and recording it as an anomaly when the deviation rate > 200%; Inputting the one-hot encoded 23 target TLS field values into the isolation forest model and the rule engine to output an anomaly probability, and recording it as a high risk when the probability > 0.

8.

9. The method according to claim 6, characterized in that, It also includes the following steps for updating the malicious feature library: Synchronize newly added malicious JA3 fingerprints from the MISP threat intelligence platform every day, and write them into the Redis cluster in batches through Redis Pipeline; Update the normal baseline based on the window calculation of Flink every week.

10. The method according to claim 6, characterized in that, Step S5 also includes outputting the results: For traffic deemed malicious, a HTTP POST request is used to notify the firewall to block the source IP. Record static matching results and dynamic detection results, and visualize them.

Images