A method for detecting encrypted communication behavior based on extended JA3 fingerprints

Abstract

The application relates to an encrypted communication behavior detection method based on an extended JA3 fingerprint, and belongs to the field of malicious encrypted traffic identification.The method comprises the following steps: obtaining the client JA3 fingerprint, the Application Data length sequence and the server JA3S fingerprint of the TLS encrypted traffic to be detected in a traffic table, and the client JA3 fingerprint, the Application Data length sequence and the server JA3S fingerprint in a detection rule table; when the client JA3 fingerprint in the detection rule table and the client JA3 fingerprint in the traffic table are successfully matched, judging whether the Application Data length sequence contains an encrypted payload interaction behavior feature string; when the Application Data length sequence contains the encrypted payload interaction behavior feature string, and the server JA3S fingerprint in the traffic table and the server JA3S fingerprint in the detection rule table are successfully matched, issuing an alarm.The application combines the joint verification of the client JA3, the server JA3S double fingerprint and the application data length sequence payload behavior feature, can greatly reduce the false positives and false negatives of encrypted traffic compared with single fingerprint identification, accurately identifies malicious encrypted communication and timely issues an alarm.

Description

Technical Field

[0001] This invention relates to the field of malicious encrypted traffic identification technology, specifically to a method for detecting encrypted communication behavior based on extended JA3 fingerprints. Background Technology

[0002] In recent years, with the escalation of security technology countermeasures, more and more attack traffic is becoming encrypted. Network security traffic detection can be divided into two main categories based on traffic direction. The first category is attack traffic targeting specific applications, which actively attacks applications from the external internet, such as vulnerability exploitation scans. The traffic generated depends on the protocols exposed by the application, typically TLS. The second category is network attack traffic generated by attackers targeting the target system. This traffic, primarily from Trojan viruses actively connecting to the attacker's C&C server from the victim's host within the internal network, depends on the communication protocol used by the attacker when writing the Trojan, and mostly tends to use TLS. Based on current network traffic threat detection levels, network intrusion detection systems mainly handle plaintext HTTP protocol detection; they are not very effective at detecting encrypted data, especially encrypted data using the TLS protocol.

[0003] Currently, traffic security detection vendors primarily focus on using TLS fingerprinting to identify maliciously encrypted HTTPS traffic. This includes client-side fingerprinting (JA3) during the handshake phase, server-side fingerprinting (JA3S), and certificate fingerprinting. This type of fingerprinting technology has become an industry consensus and has accumulated considerable expertise. However, according to feedback from most traffic security products, fingerprinting technology has significant limitations in identifying TLS threat traffic, resulting in a large number of false positives and collisions, rendering it ineffective. Summary of the Invention

[0004] To address the aforementioned problems, the purpose of this invention is to provide a method for detecting encrypted communication behavior based on extended JA3 fingerprints.

[0005] A method for detecting encrypted communication behavior based on extended JA3 fingerprints, comprising:

[0006] Step 1: Obtain the client JA3 fingerprint, ApplicationData length sequence, and server JA3S fingerprint of the TLS encrypted traffic to be inspected in the traffic table;

[0007] Step 2: Use the preset detection rule table to detect the TLS encrypted traffic to be detected and obtain the client JA3 fingerprint, Application Data length sequence and server JA3S fingerprint from the detection rule table;

[0008] Step 3: When the client JA3 fingerprint in the detection rule table and the client JA3 fingerprint in the traffic table match successfully, determine whether the Application Data length sequence contains the encrypted payload interaction behavior feature string;

[0009] Step 4: When the Application Data length sequence contains the encrypted payload interaction behavior feature string, and the server-side JA3S fingerprint in the traffic table and the server-side JA3S fingerprint in the detection rule table match successfully, an alarm is issued.

[0010] Preferably, in step 3, when the TLS encrypted traffic to be detected belongs to single-stream behavioral interaction detection, the Application Data length sequence in the traffic table matches the encrypted payload length list in the detection rule table, and the frequency of the target payload occurrence meets the frequency set in the detection rule table, it is determined whether the Application Data length sequence contains an encrypted payload interaction behavior feature string.

[0011] Preferably, in step 3, the Application Data length sequence in the traffic table is formatted into a uniform string. When the string contains the encrypted payload interaction behavior feature string in the detection rules, an alarm for suspicious single-stream behavior is generated. In the string, positive values ​​representing uplink data are represented by 1, and negative values ​​representing downlink payloads are represented by 0.

[0012] Preferably, in step 3, when the TLS encrypted traffic to be detected belongs to multi-stream behavior interaction detection, five streams that simultaneously satisfy the source IP, destination IP, and destination port of the current data stream are found, and the Application Data length sequences of these five streams are combined together. When the combined Application Data length sequence successfully matches the payload sequence in the detection rule table, and the frequency of the target payload occurrence meets the frequency set in the detection rule table, it is determined whether the combined Application Data length sequence contains encrypted payload interaction behavior feature strings.

[0013] Preferably, in step 3, the combined Application Data length sequence is formatted to form a connection string. When the connection string contains the encrypted payload interaction behavior feature string in the detection rule, an alarm for suspicious multi-stream behavior is generated. In the connection string, positive values ​​representing uplink data are represented by 1, and negative values ​​representing downlink payloads are represented by 0, forming the string of each stream, and the streams are connected by the symbol "-".

[0014] The present invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, characterized in that the computer program, when executed by the processor, implements the steps in the above-described method for detecting encrypted communication behavior based on extended JA3 fingerprints.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps in the above-described method for detecting encrypted communication behavior based on extended JA3 fingerprints.

[0016] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0017] This invention relates to a method for detecting encrypted communication behavior based on extended JA3 fingerprints. Compared with existing technologies, this invention integrates client JA3 and server JA3S dual fingerprints with application data length sequence payload behavior features for joint verification. Compared with single fingerprint recognition, it can significantly reduce false positives and false negatives of encrypted traffic, accurately identify malicious encrypted communication and issue timely alerts.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of an encrypted communication behavior detection method based on extended JA3 fingerprint provided by the present invention;

[0021] Figure 2 This is a schematic diagram of the JA3 fingerprint matching process provided by the present invention;

[0022] Figure 3 A schematic diagram illustrating the detection process of encrypted data length and frequency provided by the present invention;

[0023] Figure 4 This is a schematic diagram of the behavioral interaction data detection process provided by the present invention;

[0024] Figure 5This is a schematic diagram of the JA3S fingerprint detection process provided by the present invention. Detailed Implementation

[0025] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0026] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0028] Please see Figure 1 This invention provides a method for detecting encrypted communication behavior based on extended JA3 fingerprints, comprising:

[0029] Step 1: Obtain the client JA3 fingerprint, ApplicationData length sequence, and server JA3S fingerprint of the TLS encrypted traffic to be inspected in the traffic table;

[0030] Step 2: Use the preset detection rule table to detect the TLS encrypted traffic to be detected and obtain the client JA3 fingerprint, Application Data length sequence and server JA3S fingerprint from the detection rule table;

[0031] Step 3: When the client JA3 fingerprint in the detection rule table and the client JA3 fingerprint in the traffic table match successfully, determine whether the Application Data length sequence contains the encrypted payload interaction behavior feature string;

[0032] In step 3, when the TLS encrypted traffic to be detected belongs to single-stream behavioral interaction detection, the Application Data length sequence in the traffic table matches the encrypted payload length list in the detection rule table, and the frequency of the target payload occurrence meets the frequency set in the detection rule table, the Application Data length sequence in the traffic table is formatted into a uniform string. When the string contains the encrypted payload interaction behavior feature string in the detection rule, an alarm for suspicious single-stream behavior is generated. In the string, positive values ​​representing uplink data are represented by 1, and negative values ​​representing downlink payload are represented by 0.

[0033] When the TLS encrypted traffic to be detected belongs to multi-stream behavior interaction detection, five flows that simultaneously satisfy the source IP, destination IP, and destination port of the current data flow are found. The application data length sequences of these five flows are combined together. When the combined application data length sequence successfully matches the payload sequence in the detection rule table, and the frequency of the target payload appears meets the frequency set in the detection rule table, the combined application data length sequence is formatted to form a connection string. When the connection string contains the encrypted payload interaction behavior feature string in the detection rules, a suspicious multi-stream behavior alarm is generated. In the connection string, positive values ​​representing uplink data are represented by 1, and negative values ​​representing downlink payloads are represented by 0, forming the string of each flow, and the flows are connected by the symbol "-".

[0034] Step 4: When the Application Data length sequence contains the encrypted payload interaction behavior feature string, and the server-side JA3S fingerprint in the traffic table and the server-side JA3S fingerprint in the detection rule table match successfully, an alarm is issued.

[0035] The overall processing flow of the present invention will be described below with reference to specific embodiments:

[0036] (1) The premise of this scheme is that the TLS encrypted traffic to be detected has been successfully parsed and all the necessary information has been entered into the database table, including but not limited to the TLS client JA3 fingerprint, Application Data length sequence, and server fingerprint used in this scheme. The first step of detection is to match the client JA3 in the detection rule table with the client fingerprint in the TLS traffic table to be detected. If the match is successful, proceed to the next step of processing.

[0037] like Figure 2As shown, the attributes crucial for generating the JA3 fingerprint in the ClientHello message are the SSL version, password, extension, elliptic curve, and elliptic curve dot format. The password, extension, and elliptic curve help identify the client. After extracting each value, they are concatenated with commas to separate each field. The long string is converted from a decimal value to an MD5 hash value to create a simple 32-character fingerprint, the JA3 fingerprint. The fingerprint detection matching process involves determining whether the JA3 fingerprint in the TLS table matches the one in the detection rule table.

[0038] (2) The next step is to determine whether the detection rule table is for single-stream detection or multi-stream detection. In the case of single-stream detection, the application data length sequence in the TLS table is matched with the specific encrypted payload length list in the detection rule table. If the match is successful, the frequency of these payloads must also meet the frequency requirements in the detection rule table. If both conditions are met, proceed to the next step of single-stream behavior interaction detection. For multi-stream TLS payload detection, it is necessary to find 5 streams that simultaneously meet the requirements based on the source IP, destination IP, and destination port of the current data stream. The application data of these 5 streams are combined together and matched with the specific payload sequence and frequency in the detection rule table. If both conditions are met, proceed to the next step of multi-stream behavior interaction detection.

[0039] like Figure 3 As shown, the length of the Application Data message in the TLS protocol varies. Specific tools may generate specific encrypted data lengths, which can be one or more. Furthermore, these messages often appear more than once in a single communication; the minimum number of occurrences is the frequency of that tool's specific encrypted payload. For example, in the heartbeat behavior of a certain version of the CobaltStrike attack tool, the uplink payload Application Data length for a single stream is 308 bytes, and the downlink payload is 423 bytes. Therefore, the tool's specific payload can be defined as [308, -423], with an occurrence count of 1. In practical applications, simply checking whether the TLS payload sequence contains these two payload lengths and whether the occurrence count is greater than or equal to 1 is sufficient to determine if it meets the requirements for this round of detection.

[0040] (3) Single-stream behavior interaction detection: After passing the single-stream detection in step two, the Application Data length sequence in the TLS table to be detected needs to be formatted into a uniform string. Positive values ​​represent uplink data (represented by 1), and negative values ​​represent downlink payload (represented by 0), forming a string. Determine whether this string contains the encrypted payload interaction behavior feature string in the detection rules. If it matches, generate an alarm for suspicious single-stream behavior. At the same time, proceed to the judgment in step five of the next stage.

[0041] (4) Multi-stream behavior interaction detection: After the multi-stream behavior detection in the second step, the five payloads are formatted. Positive values ​​representing uplink data are represented by 1, and negative values ​​representing downlink payloads are represented by 0. These are combined into a string, and the streams are connected by the symbol "-". It is determined whether this connected string contains the encrypted payload interaction behavior feature string in the detection rules. If it matches, the TLS stream is detected, and an alarm for suspicious multi-stream behavior is generated.

[0042] like Figure 4 As shown, uplink and downlink behavior interaction detection targets the Application Data payload in the TLS protocol. It primarily identifies the interaction logic between uplink and downlink, using numbers 0 and 1 for labeling, a method known as the 01 labeling method. The payload's positive and negative values ​​in the data table are formatted into a uniform string, and then pattern matching is performed. If the formatted string of the payload list in the TLS table contains the uplink and downlink interaction behavior strings provided in the detection rule table, then it meets the conditions and conforms to the uplink and downlink interaction behavior. Specifically, positive numbers are formatted as 1, and negative numbers as 0; in the case of multiple streams, they are connected with "-". Examples are as follows:

[0043] The TLS payload sequence to be detected is [23,324,12,43,-12,-45,-78,-23,1,34,67,-23,345], and the formatted interaction behavior identifier is "1111000011101". If the string of the uplink and downlink interaction behavior in the detection rule table is "1100", then it meets the detection criteria for this round; similarly, in the multi-stream detection, the formatted string is "11100-11100-111000-111100", and if the value in the rule detection table is "11100", then it meets the detection criteria for this round.

[0044] (5) Server fingerprint detection: Match the server JA3S fingerprint of the TLS stream to be detected with the server fingerprint in the detection rule table. If the match is satisfied, it indicates that the TLS stream has been detected and is a high-confidence alarm.

[0045] like Figure 5 As shown, the Server Hello packet contains the decimal byte values ​​of the following fields: Version, AcceptedCipher, and List of Extensions. These values ​​are then concatenated sequentially, using commas to separate each field and hyphens to separate each value within each field. These strings are then MD5 hashed to generate a simple and shareable 32-character fingerprint. The fingerprint matching process involves determining whether the JA3S fingerprint in the TLS table matches the one in the detection rule table.

[0046] It should be noted that the detection rule table needs to be developed by analyzing attack traffic and extracting detection rules; these detection rules need to include the following key data:

[0047] (1) Detection tool name, i.e. family information, tool version.

[0048] (2) Classification of tools: This test currently only has two types of remote control Trojans: C&C (command and control) and webshell.

[0049] (3) JA3 fingerprint obtained by attacking tools from TLS traffic calculation.

[0050] (4) JA3S fingerprint obtained by TLS traffic calculation of the attack tool.

[0051] (5) JA4 fingerprint obtained by attacking tools from TLS traffic calculation.

[0052] (6) A list of specific Application Data payload lengths in the TLS traffic of the attack tool.

[0053] (7) Limit on the number of Application Data per stream.

[0054] (8) Single-stream and multi-stream detection markers, which are also the detection frequencies of a list consisting of a specific Application Data payload length.

[0055] (9) Uplink and downlink behavior pattern matching string, Application Data uplink is 1 and downlink is 0; multi-stream payload lengths are detected after being connected together with "-" symbol.

[0056] (10) Description information, which needs to explain what tool is used, what type of behavior is detected, and under what operating system the behavior is generated.

[0057] Note 1: When it is single-stream detection, the length and frequency of Application Data match. Single-stream detection only matches single-stream data; under multi-stream detection, the Application Data detection is a combination of the Application Data of 5 single streams with the same source IP, destination IP, and destination port.

[0058] Explanation 2: The behavior sequence consists of up and down lines, with Application Data having 1s for the up line and 0s for the down line. For example, if the Application Data length interaction is [23,-23,54,12,343,1212], then the behavior interaction sequence is represented as {101111}. For multi-stream detection, 5 streams are required to be connected by the character '-'; for example, {1100-1100-1100-111100-110}.

[0059] Table 1. Description of Fields in the Rule Detection Table

[0060] Fields type illustrate id UInt64 Default ID Family_Name String family Category String Categorization: C&C, webshell, etc. C_JA3_FP String Client-side JA3 fingerprint S_JA3S_FP String Server-side JA3S fingerprinting C_JA4 String Client JA4 fingerprint ENC_Data_Len_list Array(UInt16) List of matching encrypted payload sizes ENC_Data_Count UInt32 Encrypted payload number limit; when detecting single-stream behavior, this value represents the minimum number of encrypted payloads that must be detected in a single stream; when detecting multi-stream behavior, this value represents the maximum number of encrypted payloads that must be detected in a single stream. S_flowFrequency UInt32 Single-stream encrypted payload matching count Muti_flowFrequency UInt32 Number of times the encrypted payload is matched across multiple streams; matching of the merged Application Data from multiple streams; Behavior String Uplink and downlink behavior pattern matching strings; uplink is represented by 1; downlink is represented by 0. Multiple streams are connected by "-". Content`String String Description information create_time DateTime Creation time

[0061] This invention integrates client-side JA3 and server-side JA3S dual fingerprints with application data length sequence payload behavior features for joint verification. Compared with single fingerprint recognition, it can significantly reduce false alarms and missed alarms in encrypted traffic, accurately identify malicious encrypted communication and issue timely alerts.

[0062] The present invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. The computer program, when executed by the processor, implements the steps in the aforementioned method for detecting encrypted communication behavior based on extended JA3 fingerprints. Compared with the prior art, the beneficial effects of the electronic device provided by the present invention are the same as those of the aforementioned method for detecting encrypted communication behavior based on extended JA3 fingerprints, and will not be elaborated upon here.

[0063] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps in the above-described method for detecting encrypted communication behavior based on extended JA3 fingerprints. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present invention are the same as the beneficial effects of the above-described method for detecting encrypted communication behavior based on extended JA3 fingerprints, and will not be elaborated here.

[0064] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting encrypted communication behavior based on extended JA3 fingerprints, characterized in that, include: Step 1: Obtain the client JA3 fingerprint, Application Data length sequence, and server JA3S fingerprint of the TLS encrypted traffic to be inspected in the traffic table; Step 2: Use the preset detection rule table to detect the TLS encrypted traffic to be detected and obtain the client JA3 fingerprint, Application Data length sequence and server JA3S fingerprint from the detection rule table; Step 3: When the client JA3 fingerprint in the detection rule table and the client JA3 fingerprint in the traffic table match successfully, determine whether the Application Data length sequence contains the encrypted payload interaction behavior feature string; Step 4: When the Application Data length sequence contains the encrypted payload interaction behavior feature string, and the server-side JA3S fingerprint in the traffic table and the server-side JA3S fingerprint in the detection rule table match successfully, an alarm is issued.

2. The encrypted communication behavior detection method based on extended JA3 fingerprint as described in claim 1, characterized in that, In step 3, when the TLS encrypted traffic to be detected belongs to single-stream behavioral interaction detection, the Application Data length sequence in the traffic table matches the encrypted payload length list in the detection rule table, and the frequency of the target payload occurrence meets the frequency set in the detection rule table, it is determined whether the Application Data length sequence contains the encrypted payload interaction behavior feature string.

3. The encrypted communication behavior detection method based on extended JA3 fingerprints according to claim 2, characterized in that, In step 3, the Application Data length sequence in the traffic table is formatted into a uniform string. When the string contains the encrypted payload interaction behavior feature string in the detection rules, an alarm for suspicious single-stream behavior is generated. In the string, positive values ​​representing uplink data are represented by 1, and negative values ​​representing downlink payloads are represented by 0.

4. The encrypted communication behavior detection method based on extended JA3 fingerprint as described in claim 3, characterized in that, In step 3, when the TLS encrypted traffic to be detected belongs to multi-stream behavior interaction detection, five streams that simultaneously satisfy the source IP, destination IP, and destination port of the current data stream are found. The application data length sequences of these five streams are combined together. When the combined application data length sequence successfully matches the payload sequence in the detection rule table, and the frequency of the target payload occurrence meets the frequency set in the detection rule table, it is determined whether the combined application data length sequence contains encrypted payload interaction behavior feature strings.

5. The encrypted communication behavior detection method based on extended JA3 fingerprint as described in claim 4, characterized in that, In step 3, the combined Application Data length sequence is formatted to form a connection string. When the connection string contains the encrypted payload interaction behavior feature string in the detection rule, an alarm for suspicious multi-stream behavior is generated. In the connection string, positive values ​​representing uplink data are represented by 1, and negative values ​​representing downlink payloads are represented by 0, forming the string of each stream, and the streams are connected by the symbol "-".

6. An electronic device comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, characterized in that, When the computer program is executed by the processor, it implements the steps in the encrypted communication behavior detection method based on extended JA3 fingerprint as described in any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the encrypted communication behavior detection method based on extended JA3 fingerprint as described in any one of claims 1-5.

Images