HTTPS attack traffic restoration method and system

Abstract

The invention discloses an HTTPS attack traffic restoration method and system, and belongs to the field of computer network security and traffic analysis. The method comprises the following steps: establishing an attacker behavior knowledge base, wherein the attacker behavior knowledge base is used for describing a system calling behavior event which is necessarily executed by an attacker to realize an attack purpose; constructing a behavior event tree of each HTTPS request in the HTTPS flow; wherein the behavior event tree comprises an HTTPS request event, a system call behavior event and an HTTPS response event, and data of the HTTPS request event and data of the HTTPS response event are associated with plaintext data of an HTTPS request and plaintext data of an HTTPS response respectively; and matching the system call behavior event in the attacker behavior knowledge base, and performing HTTPS flow reduction of the HTTPS request based on the HTTPS request and the plaintext data of the HTTPS response under the condition of successful matching. According to the method, the attack traffic can be restored after the attack traffic is associated with the attack behavior.

Description

Technical Field

[0001] The present invention relates to the fields of computer network security and traffic analysis, and in particular to a HTTPS attack traffic restoration method and system. Background Art

[0002] According to statistics from w3techs, the proportion of websites using HTTPS as the default protocol has increased from 22.5% to 80% in the five years from 2017 to 2022. As of November 2023, 84.6% of all websites in the world use https as their default protocol. With the popularization of awareness of network security and privacy protection, the number of websites using HTTPS as the default protocol has been increasing year by year. However, while HTTPS protects user privacy, it also affects the defender's review of traffic content. In the process of attack and defense, HTTPS has become a serious blind spot for traffic analysis equipment. Traffic auditing equipment deployed in a network environment is in the same situation as the attacker. It is unable to recover data information and can only obtain encrypted ciphertext from the traffic. Thanks to the security of modern encryption algorithms, it is difficult for listeners to recover sufficiently effective information from ciphertext.

[0003] In addition, thousands or even tens of thousands of new web vulnerabilities are exposed every year. Conventional traffic analysis is mostly based on known rules and can only analyze known attacks. It has poor detection effects on obfuscated or deformed attack payloads and has no detection capabilities for unknown 0day vulnerabilities. Therefore, being able to decrypt HTTPS traffic and associate traffic detection with attack behaviors to detect known attacks and discover unknown attacks is an important direction of current network security research. Summary of the invention

[0004] In view of the above problems, the present invention proposes a HTTPS attack traffic restoration method and system, which inserts the HTTPS server through the eBPF technology, decrypts the HTTPS traffic and analyzes the web application behavior, realizes the decryption of the HTTPS traffic, and restores the attack traffic after associating the attack traffic with the attack behavior.

[0005] In order to achieve the above-mentioned purpose, the specific technical solutions adopted by the present invention include the following contents.

[0006] A method for restoring HTTPS attack traffic, the method comprising:

[0007] Establishing an attacker behavior knowledge base, wherein the attacker behavior knowledge base is used to describe system call behavior events that the attacker must execute to achieve the attack purpose;

[0008] Constructing a behavior event tree for each HTTPS request in HTTPS traffic; wherein the behavior event tree includes: HTTPS request events, system call behavior events, and HTTPS response events, and data of the HTTPS request events and the HTTPS response events are respectively associated with plaintext data of the HTTPS request and the HTTPS response;

[0009] The system call behavior events are matched in the attacker behavior knowledge base, and if the match is successful, the HTTPS traffic of the HTTPS request is restored based on the plaintext data of the HTTPS request and the HTTPS response.

[0010] Furthermore, a behavior event tree of each HTTPS request in the HTTPS traffic is constructed, including:

[0011] Use ebpf technology to instrument key functions for HTTPS traffic encryption and decryption;

[0012] Use ebpf technology to instrument key system calls of the operating system kernel;

[0013] When the web server receives an HTTPS request, it creates a Span root node and marks the Span root node as active;

[0014] When the web server reads the HTTPS request, based on the relevant hook generated by the ebpf technology, an HTTPS request event is created, and the plaintext data of the HTTPS request is obtained; wherein the HTTPS request event, the system call behavior event and the HTTPS response event are described by metadata and attributes, and the metadata includes: time, thread ID, belonging Span node and event name, and the attributes are corresponding data obtained from the kernel key system call, and the time, thread ID and event name are obtained according to the relevant hook generated by the ebpf technology;

[0015] Using the time and thread ID in the HTTPS request event as indexes, searching for the currently active Span root node, associating the HTTPS request event with the found Span root node, and then updating the corresponding Span node in the HTTPS request event based on the Span root node;

[0016] When the web server processes an HTTPS request, it creates a system call behavior event and a Span node based on the related hooks generated in the eBPF technology based on the system call behavior, and marks the Span node as active;

[0017] Using the time, thread ID and attribute in the system call behavior event as indexes, searching for the Span node to which the system call behavior event belongs, and after associating the system call behavior event with the found Span node, updating the Span node in the HTTPS request event based on the Span node;

[0018] For the Span node associated with the system call behavior event, the time and thread ID in the system call behavior event are used as indexes to find the Span root node or Span node to generate the parent-child relationship in the behavior event tree;

[0019] When the web server processes the HTTPS request, it creates an HTTPS response event based on the related hooks generated in the eBPF technology and obtains the plaintext data of the HTTPS response.

[0020] Using the time and thread ID in the HTTPS response event as indexes, searching for the currently active Span root node, associating the HTTPS response event with the found Span root node, and then updating the corresponding Span node in the HTTPS response event based on the Span root node;

[0021] After the HTTPS request is completed, the span root node and all span nodes under the span root node are marked as inactive.

[0022] Furthermore, the time, thread ID and attribute in the system call behavior event are used as indexes to find the Span node to which the system call behavior event belongs, including:

[0023] Use the time, thread ID and attributes in the system call behavior event as indexes to search for Span nodes in active Span nodes;

[0024] If the corresponding Span node is found in the active Span nodes, the Span node is used as the Span node to which the system call behavior event belongs;

[0025] If the corresponding Span node is not found in the active Span nodes, a new Span node is created and used as the Span node to which the system call behavior event belongs.

[0026] Furthermore, matching of system call behavior events is performed in the attacker behavior knowledge base, including:

[0027] Obtain a list of rules for system call behavior events that the attacker must execute to achieve the attack purpose;

[0028] The attributes of the system call behavior event are matched in the rule list.

[0029] Furthermore, in the case of a successful match, the HTTPS traffic of the HTTPS request is restored based on the plaintext data of the HTTPS request and the HTTPS response, including:

[0030] Get the Span node to which the system call behavior event belongs;

[0031] According to the behavior event tree, obtain the Span root node corresponding to the Span node to which it belongs;

[0032] Get the plaintext data of the HTTPS request and HTTPS response corresponding to the span root node;

[0033] The HTTPS traffic of the HTTPS request is restored based on the plaintext data of the HTTPS request and the HTTPS response.

[0034] A HTTPS attack traffic restoration system, the system comprising:

[0035] A knowledge base construction module is used to establish an attacker behavior knowledge base, wherein the attacker behavior knowledge base is used to describe the system call behavior events that the attacker must execute to achieve the attack purpose;

[0036] A behavior event tree construction module, used to construct a behavior event tree for each HTTPS request in HTTPS traffic; wherein the behavior event tree includes: HTTPS request events, system call behavior events and HTTPS response events, and the data of the HTTPS request events and the HTTPS response events are respectively associated with the plaintext data of the HTTPS request and the HTTPS response;

[0037] The HTTPS traffic restoration module is used to match system call behavior events in the attacker behavior knowledge base, and if the match is successful, the HTTPS traffic restoration of the HTTPS request is performed based on the plaintext data of the HTTPS request and the HTTPS response.

[0038] An electronic device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the HTTPS attack traffic restoration method described in any one of the above items is implemented.

[0039] A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement any of the above-mentioned HTTPS attack traffic restoration methods.

[0040] A computer program product, characterized in that when the computer program product is run on a computer device, the computer device executes any one of the above-mentioned HTTPS attack traffic restoration methods.

[0041] Compared with the prior art, the present invention has at least the following beneficial effects.

[0042] Due to the popularity of HTTPS, defenders in the network cannot effectively analyze the attack load in the encrypted traffic, and a large number of HTTPS devices have become blind spots in defense. Conventional attack detection based on traffic rules can only deal with known risks and cannot effectively prevent attacks that have not yet been disclosed. In response to this problem, the present invention proposes a method for restoring HTTPS attack traffic to achieve decryption of HTTPS traffic and behavioral detection of unknown network attacks. The method can effectively restore the plaintext of HTTPS encrypted traffic and discover attack behaviors in web services, thereby improving the discovery and response speed of network attacks in the HTTPS environment and achieving the purpose of detecting HTTPS attack traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a system overall architecture diagram of the solution of the present invention.

[0044] Figure 2 This is an algorithm flow chart that uses Span / Event to describe HTTPS requests.

[0045] Figure 3 It is a flow chart of the algorithm for associating Span with Event.

[0046] Figure 4 It is a flowchart of the algorithm for associating attack behavior with attack traffic.

[0047] In the figure, 210, 220, 230, 240, 250, 260, 270, 280, 290, 310, 320, 330, 340, 410, 420, 430, and 440 are steps. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention and to make the objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0049] The present invention provides a HTTPS attack traffic restoration system, the system architecture is as follows Figure 1As shown in the figure, it is mainly divided into three modules: knowledge base construction module, behavior event tree construction module and HTTPS traffic restoration module. These three modules involve four parts: user-mode handler, kernel-mode instrumentation program, attacker behavior knowledge base and behavior event database.

[0050] The attacker behavior knowledge base records the attacker behavior in real attack scenarios summarized by experts. It is used to describe the corresponding actions that the attacker will take when implementing the attack behavior and to characterize the attack from a behavioral level.

[0051] The kernel instrumentation program is responsible for the instrumentation of openssl and kernel system calls, and sends the events generated by the instrumentation to the user-mode handler for processing through ebpfmap, so as to observe the events during the HTTPS traffic processing.

[0052] The user-mode program is responsible for processing the data output from the kernel mode, realizing the association between HTTPS traffic and web application behavior, and storing the associated data in the behavior event database. Specifically, the user-mode event handler can realize the analysis of ebpf instrumentation events and realize the association between HTTPS traffic and web application behavior. The program is responsible for loading the ebpf kernel program, configuring the ebpf program using ebpfmap, and accepting events output by the ebpf instrumentation from the kernel. The events output by the ebpf instrumentation are standardized and converted to obtain Events; Events are arranged according to time and thread ID as indexes to obtain Span; Spans are arranged according to time and thread ID as indexes to establish parent-child relationships between Spans; the correspondence between HTTPS traffic and web application behavior is obtained based on the relationship between Spans; the correspondence between HTTPS traffic and web application behavior is matched with the attack behavior knowledge base to obtain the plaintext of HTTPS attack traffic and attack behavior.

[0053] The event database records the Span and Event data in the above observation process, as well as the associated data, and compares it with the attacker behavior knowledge base to discover attack behaviors.

[0054] The HTTPS attack traffic restoration method disclosed in the present invention includes the following steps 1 to 3.

[0055] Step 1: Build a knowledge base of attacker behavior.

[0056] Although the attacker uses different attack payloads for different vulnerability types, the attacker's attack purpose is always the same, so by analyzing the attacker's purpose and describing the attacker's behavior, the attack can be effectively detected. Based on the above, the present invention can model the attacker's intention based on the real environment and establish an attacker behavior knowledge base, which is used to describe the system call behavior events that the attacker must execute to achieve the attack purpose.

[0057] Step 2: Construct a behavior event tree for each HTTPS request in HTTPS traffic.

[0058] The process of building the behavior event tree of HTTPS requests is mainly to first instrument the openssl library through the ebpf technology, obtain the corresponding plaintext data by instrumenting the key functions of HTTPS traffic encryption and decryption, and realize the decryption of HTTPS traffic; then, instrument the key system calls of the operating system kernel through the ebpf technology to realize the behavior monitoring of the web service from the aspects of network, file, process, IO, etc.; then, realize the communication between the ebpf instrumentation and the user state through ebpfmap, and the ebpf instrumentation reports the instrumentation-triggered data to the user state through ebpfmap, and the user state program dynamically sends the configuration of the ebpf instrumentation through ebpfmap; finally, the user state program constructs the behavior event tree of each HTTPS request by receiving the events generated by the ebpf instrumentation program, thereby obtaining the behavior sequence of the web service, and establishing the behavior description of the web service process by defining Span and Event.

[0059] Among them, in the behavior event tree, Event is defined as an event, which is triggered by the ebpf plug-in and is used to describe an action that occurs in a certain behavior of the web service process, such as opening a file, creating a process, initiating a network connection, etc. Event is described by metadata and attributes. Metadata is composed of a four-tuple of time, thread ID, span, and event name, which is used to describe the common attributes of Event. Attributes describe independent parameters under different Event behaviors.

[0060] Attribute is defined as a property, and its value is the corresponding data obtained from the kernel when the ebpf stub is triggered. The attributes of different events are different. The unified storage of attributes is achieved by serializing the attributes into json format.

[0061] Span is defined as the time span of behavior, describing a time period for a web service process to perform a certain type of behavior, such as: a remote network communication, the creation and destruction of a child process, the time period for opening a file for reading, etc. Span is composed of Events, and a Span can contain one or more Events. There is a parent-child relationship between spans, and the child span is used to describe the subsidiary behaviors generated during the parent span behavior process. By associating spans, a complete description of the web service process behavior is achieved.

[0062] In one embodiment, the user-mode program constructs a behavior event tree for each HTTPS request by receiving events generated by the ebpf instrumentation program, such as Figure 2 As shown, the steps include steps 210 to 290.

[0063] Step 210: When the web server receives an HTTPS request, it creates a root span and marks it as active. The root span is used to describe the entire behavior flow of the web server from accepting the current HTTPS request to completing the response. In the process of request processing to response, the spans used to describe the remaining subsidiary behaviors caused by the request are all child spans of the root span.

[0064] Step 220: The web server reads the HTTPS request, triggers the HTTPS traffic decryption-related hook in ebpf, obtains the HTTPS request plaintext traffic, and creates an HTTPS request Event.

[0065] Step 230: Obtain the time and thread ID from the HTTPS request Event metadata as indexes, find the currently active root Span, and associate the HTTPS request Event with the found Span.

[0066] Step 240: When processing an HTTPS request, the web server performs a series of actions, triggering other ebpfhooks, generating corresponding sub-spans and marking them as active.

[0067] Step 250: Similar to step 240, the creation of Span also corresponds to the generation of related Event.

[0068] Step 260: For different Span types, the Span to which the Event belongs is searched using the time, thread ID in the Event metadata, and additional attributes in the Attribute as indexes, and the Span is associated with the current Event.

[0069] Step 270: The Span generated in the HTTPS request is associated with the root Span using the time and thread ID as indexes. In this step, the current Span may have a lower-level child Span, and the child Span can be associated with the parent Span using the method of step 260 or step 270 according to the type.

[0070] Step 280: After the HTTPS request is processed, the web server writes the HTTPS response, triggering ebpf The hook obtains the response plaintext traffic and creates an HTTPS response Event, which is associated with the root Span in the same way as step 230.

[0071] Step 290: After the HTTPS request is completed, it means that the processing of the current HTTPS request is completed. The current root span and all its child spans are removed from the active span, and the root span is used as a complete description of the behavior during the HTTPS request.

[0072] Among them, when steps 230, 260, and 280 involve the association between Span and Event, you can apply Figure 3 It is an association algorithm between Span and Event, and specifically includes the following steps 310 to 340.

[0073] Step 310: When ebpfhook is triggered, a corresponding event (Event) is generated according to the hook type. The event attributes vary according to different hooks, but all events contain metadata consisting of a four-tuple of time, thread ID, span, and event name.

[0074] Step 320: Generate a corresponding index based on the metadata and attributes of the event. Figure 2 Step 230, step 260, and step 280 all generate indexes to find Span through this step.

[0075] Step 330: Use the generated index to find out whether there is a corresponding active Span. If the search is successful, use the existing Span; otherwise, create a corresponding Span and mark it as active.

[0076] Step 340: Update the Span field in the metadata of the current event to complete adding the event to the subsequent attack behavior and attack traffic matching algorithm.

[0077] Step 3: Match the system call behavior event in the attacker behavior knowledge base, and if the match is successful, restore the HTTPS traffic of the HTTPS request based on the plaintext data of the HTTPS request and HTTPS response.

[0078] This step first uses the attack behavior proposed in the present invention to match the attack traffic matching algorithm, and then restores the HTTPS attack traffic based on the plaintext data corresponding to the matching result.

[0079] in, Figure 4 It is the attack behavior and attack traffic matching algorithm, which is described as follows:

[0080] Step 410: For each event generated by each behavior in the corresponding web service, compare it with the attack behavior in the knowledge base to determine whether it is an attack behavior. If it hits the attack entry, it will be processed later, otherwise it is considered to be a normal behavior and ignored. It should be noted that the knowledge base not only has rule matching for a single event, but also can distinguish event groups composed of multiple events in a time series.

[0081] Step 420: Search upward through the corresponding Span in the Event metadata until the root Span describing the entire HTTPS request is found.

[0082] Step 430: Search for HTTPS request events and HTTPS response events from the root Span, and obtain the plaintext traffic of the HTTPS request and response from the events.

[0083] Step 440: The attack behavior hit by the knowledge base is associated with the specific plaintext traffic, and the attack behavior and attack traffic matching algorithm ends.

[0084] Finally, it should be noted that the above implementation cases are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention is described in detail using examples, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for restoring HTTPS attack traffic, characterized in that: The method comprises: Establishing an attacker behavior knowledge base, wherein the attacker behavior knowledge base is used to describe system call behavior events that the attacker must execute to achieve the attack purpose; Constructing a behavior event tree for each HTTPS request in HTTPS traffic; wherein the behavior event tree includes: HTTPS request events, system call behavior events, and HTTPS response events, and data of the HTTPS request events and the HTTPS response events are respectively associated with plaintext data of the HTTPS request and the HTTPS response; The system call behavior events are matched in the attacker behavior knowledge base, and if the match is successful, the HTTPS traffic of the HTTPS request is restored based on the plaintext data of the HTTPS request and the HTTPS response.

2. The method according to claim 1, characterized in that Construct a behavior event tree for each HTTPS request in HTTPS traffic, including: Use ebpf technology to instrument key functions for HTTPS traffic encryption and decryption; Use ebpf technology to instrument key system calls of the operating system kernel; When the web server receives an HTTPS request, it creates a Span root node and marks the Span root node as active; When the web server reads the HTTPS request, based on the relevant hook generated by the ebpf technology, an HTTPS request event is created, and the plaintext data of the HTTPS request is obtained; wherein the HTTPS request event, the system call behavior event and the HTTPS response event are described by metadata and attributes, and the metadata includes: time, thread ID, belonging Span node and event name, and the attributes are corresponding data obtained from the kernel key system call, and the time, thread ID and event name are obtained according to the relevant hook generated by the ebpf technology; Using the time and thread ID in the HTTPS request event as indexes, searching for the currently active Span root node, associating the HTTPS request event with the found Span root node, and then updating the corresponding Span node in the HTTPS request event based on the Span root node; When the web server processes an HTTPS request, it creates a system call behavior event and a Span node based on the related hooks generated in the eBPF technology based on the system call behavior, and marks the Span node as active. Using the time, thread ID and attribute in the system call behavior event as indexes, searching for the Span node to which the system call behavior event belongs, and after associating the system call behavior event with the found Span node, updating the Span node in the HTTPS request event based on the Span node; For the Span node associated with the system call behavior event, the time and thread ID in the system call behavior event are used as indexes to find the Span root node or Span node to generate the parent-child relationship in the behavior event tree; When the web server processes the HTTPS request, it creates an HTTPS response event based on the related hooks generated in the eBPF technology and obtains the plaintext data of the HTTPS response. Using the time and thread ID in the HTTPS response event as indexes, searching for the currently active Span root node, associating the HTTPS response event with the found Span root node, and then updating the corresponding Span node in the HTTPS response event based on the Span root node; After the HTTPS request is completed, the span root node and all span nodes under the span root node are marked as inactive.

3. The method according to claim 2, characterized in that Use the time, thread ID, and attributes in the system call behavior event as indexes to find the Span node to which the system call behavior event belongs, including: Use the time, thread ID and attributes in the system call behavior event as indexes to search for Span nodes in active Span nodes; If the corresponding Span node is found in the active Span nodes, the Span node is used as the Span node to which the system call behavior event belongs; If the corresponding Span node is not found in the active Span nodes, a new Span node is created and used as the Span node to which the system call behavior event belongs.

4. The method according to claim 2, characterized in that: Matching system call behavior events in the attacker behavior knowledge base includes: Obtain a list of rules for system call behavior events that the attacker must execute to achieve the attack purpose; The attributes of the system call behavior event are matched in the rule list.

5. The method according to claim 2, characterized in that: If the match is successful, the HTTPS traffic of the HTTPS request is restored based on the plaintext data of the HTTPS request and HTTPS response, including: Get the Span node to which the system call behavior event belongs; According to the behavior event tree, obtain the Span root node corresponding to the Span node to which it belongs; Get the plaintext data of the HTTPS request and HTTPS response corresponding to the span root node; The HTTPS traffic of the HTTPS request is restored based on the plaintext data of the HTTPS request and the HTTPS response.

6. A HTTPS attack traffic restoration system, characterized in that: The system comprises: A knowledge base construction module is used to establish an attacker behavior knowledge base, wherein the attacker behavior knowledge base is used to describe the system call behavior events that the attacker must execute to achieve the attack purpose; A behavior event tree construction module, used to construct a behavior event tree for each HTTPS request in HTTPS traffic; wherein the behavior event tree includes: HTTPS request events, system call behavior events and HTTPS response events, and the data of the HTTPS request events and the HTTPS response events are respectively associated with the plaintext data of the HTTPS request and the HTTPS response; The HTTPS traffic restoration module is used to match system call behavior events in the attacker behavior knowledge base, and if the match is successful, the HTTPS traffic restoration of the HTTPS request is performed based on the plaintext data of the HTTPS request and the HTTPS response.

7. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the HTTPS attack traffic restoration method as described in any one of claims 1-5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the HTTPS attack traffic restoration method according to any one of claims 1 to 5 is implemented.

9. A computer program product, characterized in that When the computer program product runs on a computer device, the computer device executes the HTTPS attack traffic restoration method according to any one of claims 1 to 5.

Images