A rule tree-based HTTPS traffic monitoring method and device

Abstract

The present application relates to a kind of HTTPS traffic monitoring method and device based on rule tree, the method includes: based on machine learning, matching rule base is constructed;From the matching rule base, balanced binary rule tree is constructed;Through the balanced binary rule tree, according to the matching rule base, service conversion is carried out to HTTPS service;After service conversion, HTTPS service is monitored to HTTP traffic.The present application is converted to HTTP service after corresponding rule matching based on rule tree on agent, realizes real-time HTTPS service traffic degradation, so as to the real-time monitoring of HTTP traffic after degradation is realized to the real-time monitoring of HTTPS traffic, improves the monitoring accuracy.

Description

Technical Field

[0001] This invention belongs to the field of network traffic management, and particularly relates to a method and apparatus for HTTPS traffic monitoring based on rule trees. Background Technology

[0002] Because the traditional HTTP protocol transmits data in plaintext, data is easily stolen or tampered with during transmission. To avoid these problems, application service providers usually use the HTTPS protocol for data transmission. For example, when accessing a website, even if you use the HTTP protocol, you will be forced to access the website using the HTTPS protocol.

[0003] However, this also presents challenges for network traffic monitoring, as traditional traffic monitoring methods (such as DPI) cannot directly decrypt and analyze HTTPS traffic. Therefore, how to effectively monitor HTTPS traffic has become an urgent problem to be solved.

[0004] Current client-side HTTPS traffic monitoring typically employs machine learning to analyze bypass traffic. For example, it uses information such as packet length, packet interval, and TCP sequence number to train a C4.5 decision tree model, which is then used to classify bypass traffic. This classification method has a certain number of false positives and false negatives, resulting in low accuracy. Other methods for handling client-side HTTPS traffic involve decrypting the traffic using fake certificates or illegally obtained official certificate private keys before detection or analysis. For instance, using a forged server certificate to encrypt and decrypt traffic for detection requires installing a custom root certificate on the client beforehand, enabling the proxy server to establish HTTPS connections with both the client and server to provide normal service. In addition to the fake certificate, two SSL / TLS encryption / decryption operations are required on the HTTPS proxy server, significantly increasing the computational burden and resulting in low performance. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of the invention is to provide a method and device for HTTPS traffic monitoring based on rule trees.

[0006] The first aspect of this invention provides a rule tree-based HTTPS traffic monitoring method, comprising:

[0007] S1: Building a matching rule base based on machine learning;

[0008] S2: Construct a balanced binary rule tree from the matching rule base;

[0009] S3: Using the balanced binary rule tree, perform service conversion on the HTTPS service according to the matching rule base;

[0010] S4: Perform HTTP traffic monitoring on the HTTPS service after service conversion.

[0011] According to the rule tree-based HTTPS traffic monitoring method provided by the present invention, step S1 further includes:

[0012] S11: Configure transparent proxy for the client, and select two groups of websites for proxying respectively;

[0013] S12: Collect and obtain HTTP communication information between the client and the agent respectively, the HTTP communication information including first HTTP communication information and second HTTP communication information;

[0014] S13: Manually label the first HTTP communication information with errors, and fill the first HTTP communication information and the modification rules corresponding to the error labels into the feature library to obtain the first dataset; perform machine learning error labeling on the first HTTP communication information, and fill the second HTTP communication information and the modification rules corresponding to the error labels into the feature library to obtain the second dataset;

[0015] S14: Construct a matching rule base using the first dataset and the second dataset.

[0016] According to the rule tree-based HTTPS traffic monitoring method provided by the present invention, the HTTP communication information in step S12 specifically includes: message information, status code, HTTP request header, HTTP response header, and control exception information.

[0017] According to the HTTPS traffic monitoring method based on rule tree provided by the present invention, the matching rule base in step S1 specifically includes: an attribute type table, a feature attribute and type relationship table, a feature table, and a matching rule table.

[0018] According to the HTTPS traffic monitoring method based on rule trees provided by the present invention, the balanced binary rule tree in step S2 specifically includes: a balanced binary rule tree for request messages and a balanced binary rule tree for response messages.

[0019] According to the HTTPS traffic monitoring method based on rule tree provided by the present invention, in step S3, when the proxy receives client data, it performs matching rule base matching through the balanced binary rule tree of the request message, and performs service conversion according to the matched rule; when the proxy receives server data, it performs matching rule base matching through the balanced binary rule tree of the request message, and performs service conversion according to the matched rule.

[0020] According to the rule tree-based HTTPS traffic monitoring method provided by the present invention, step S3 further includes:

[0021] S31: The proxy receives the client's input HTTPS service request and redirects the input HTTPS service request to an HTTP service request;

[0022] S32: The agent performs general rule matching and specified rule matching on the HTTP service request according to the matching rule base to obtain the output HTTPS service request;

[0023] S33: The server receives the output HTTPS service request and returns a response message;

[0024] S34: The agent performs general rule matching and specified rule matching on the response message according to the matching rule base to obtain the modified message;

[0025] S35: The agent returns the modified message to the client via the HTTP protocol.

[0026] A second aspect of the present invention also provides an HTTPS traffic monitoring device based on a rule tree, comprising:

[0027] Rule base building module: used to build a matching rule base based on machine learning;

[0028] Rule tree construction module: used to construct a balanced binary rule tree from the matching rule library;

[0029] Conversion module: used to convert HTTPS services according to the matching rule base using the balanced binary rule tree;

[0030] Monitoring module: Used to monitor HTTP traffic for the converted HTTPS service.

[0031] A third aspect of the present invention also provides an HTTPS traffic monitoring device based on a rule tree, comprising:

[0032] A memory and at least one processor, wherein the memory stores instructions;

[0033] At least one of the processors invokes the instructions in the memory to cause the rule-tree-based HTTPS traffic monitoring device to perform the rule-tree-based HTTPS traffic monitoring method as described in any of the preceding embodiments.

[0034] A fourth aspect of the present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the rule tree-based HTTPS traffic monitoring method as described in any of the preceding claims.

[0035] The beneficial effects of this invention are as follows:

[0036] This invention provides a rule-tree-based HTTPS traffic monitoring method, apparatus, device, and storage medium. It adds a forward HTTPS proxy between the client browser and the HTTPS server. The forward HTTPS proxy establishes a normal HTTPS connection with the real HTTPS server. Based on the rule tree, the proxy performs rule matching to convert the HTTPS service to an HTTP service, enabling the client browser to establish an HTTP connection with the HTTPS proxy server. This achieves real-time HTTPS service traffic degradation, allowing real-time monitoring of HTTPS traffic through real-time monitoring of the degraded HTTP traffic.

[0037] This invention eliminates the need for separate HTTPS connections between the proxy server and the client / server, improving communication efficiency. Furthermore, when building the rule base, a semi-supervised machine learning method is employed to automatically label and fill in rules for abnormal messages, automatically adapting the matching rules for abnormal messages, thus improving adaptation efficiency. A balanced binary rule tree is used during matching to increase the speed at which the proxy server processes messages. Finally, the message sent to the client after matching based on the rule tree is an HTTP message. Combined with an HTTP traffic monitoring device, HTTPS traffic monitoring is completed. Monitoring does not require decryption using fake certificates or other methods for analysis, ensuring the security and integrity of user data between the client and the proxy server, while also improving monitoring performance, reducing false positives and false negatives, and increasing accuracy. Attached Figure Description

[0038] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. It is obvious that the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.

[0039] Figure 1 This is a schematic diagram of a rule tree-based HTTPS traffic monitoring method provided in an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of the structure of an HTTPS traffic monitoring device based on a rule tree, provided in an embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram illustrating the service process principle based on forward proxy provided in an embodiment of the present invention.

[0042] Figure label:

[0043] 100. Rule base construction module; 200. Rule tree construction module; 300. Conversion module; 400. Monitoring module. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0045] Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts disclosed in this invention.

[0046] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The terms "installed," "connected," and "linked" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0047] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with some aspects of the invention as detailed in the appended claims.

[0048] To better understand this invention, the terms used in the embodiments of this invention will be explained below.

[0049] HTTP (Hypertext Transfer Protocol) is a simple request-response protocol. It typically runs on top of TCP and specifies what messages a client might send to a server and what responses it might receive. The main function of the HTTP protocol is to define the communication standards between WWW servers and browsers; it is a protocol that both parties adhere to. It allows communication and data transmission between clients and servers and is one of the most widely used network protocols on the Internet.

[0050] TLS (Transport Layer Security) is a transport layer security protocol, and its predecessor, Secure Sockets Layer (SSL), is a security protocol designed to provide security and data integrity for internet communications.

[0051] HTTPS (Hypertext Transfer Protocol Secure) is an HTTP channel designed for security. Building upon HTTP, it ensures security during transmission through encryption and authentication. HTTPS adds SSL / TLS to HTTP; the security foundation of HTTPS is SSL / TLS, and the details of encryption are handled by SSL / TLS. HTTPS uses a different default port than HTTP and includes an encryption / authentication layer (between HTTP and TCP). This system provides authentication and encrypted communication methods and is widely used for security-sensitive communications on the World Wide Web.

[0052] To better understand this invention, the research background of this invention will be explained below.

[0053] Because the traditional HTTP protocol transmits data in plaintext, the data is easily stolen or tampered with during transmission. To avoid these problems, application service providers usually use the HTTPS protocol for data transmission, which employs encryption algorithms.

[0054] The specific process is as follows: The user initiates an HTTP service request by entering http: / / www.xxx1.com in the browser. The Upgrade-Insecure-Requests header is set to 1, indicating a request to upgrade to a secure connection. The xxx1 server returns a 30x status code response and redirects to https: / / www.xxx1.com, forcing the user to access the service via HTTPS. The browser then initiates a request to the server at https: / / www.xxx1.com. The xxx1 server returns a response message based on the HTTPS protocol.

[0055] HTTPS traffic monitoring is divided into server-side monitoring and client-side monitoring. Server-side HTTPS services typically employ a combination of HTTPS reverse proxy services and regular HTTP application services. The reverse proxy server possesses the service's CA certificate, enabling it to encrypt and decrypt traffic. Traffic between the reverse proxy server and the application server is HTTP traffic, which can be monitored along this link, thus monitoring all traffic accessing that server. Methods for server-side HTTPS traffic monitoring are relatively mature; Intrusion Detection System (IDS) detection utilizes these methods.

[0056] Client-side HTTPS traffic monitoring typically employs machine learning to analyze bypass traffic. For example, it uses packet length, packet interval, and TCP sequence number information to train a C4.5 decision tree model, which is then used to classify bypass traffic. This method has a certain number of false positives and false negatives, resulting in low accuracy. Another approach to handling client-side HTTPS traffic involves decrypting the traffic using fake certificates or illegally obtained official certificate private keys before detection or analysis. For instance, it uses forged server certificates to encrypt and decrypt traffic for detection. This method requires installing a custom root certificate on the client beforehand, enabling the proxy server to establish HTTPS connections with both the client and server to provide normal service. In addition to the fake certificate, it also requires two SSL / TLS encryption / decryption processes on the HTTPS proxy server, significantly increasing the computational burden and resulting in low performance.

[0057] To address the issue of how to perform real-time HTTPS traffic monitoring and ensure normal and efficient communication with clients when a proxy server can only establish an HTTPS connection with the server but not a normal connection with the client without a real server certificate, this invention proposes an HTTPS traffic monitoring method and apparatus based on rule tree matching.

[0058] The embodiments of the present invention are described below with reference to the figures.

[0059] like Figure 1 As shown, the first aspect of the present invention provides a method for HTTPS traffic monitoring based on a rule tree, comprising:

[0060] S1: Build a matching rule base based on machine learning.

[0061] Specifically, the matching rule base in step S1 includes: an attribute type table, a feature attribute and type relationship table, a feature table, and a matching rule table.

[0062] Furthermore, the rule base is used to store a list of message modifications required to convert HTTPS to HTTP service protocol on the forward proxy. The rule base of this invention is designed with four tables: attribute type table, feature attribute and type relationship table, feature table, and matching rule table. The feature table contains feature attribute information from the machine learning feature library, and feature attributes can be added or removed according to different analysis scenarios.

[0063] Step S1 further includes:

[0064] S11: Configure transparent proxy for the client, and select two groups of websites for proxying respectively.

[0065] S12: Collect the HTTP communication information between the client and the agent respectively, the HTTP communication information including first HTTP communication information and second HTTP communication information.

[0066] Specifically, the HTTP communication information in step S12 includes: message information, status code, HTTP request header, HTTP response header, and control exception information.

[0067] S13: Manually label the first HTTP communication information with errors, and fill the first HTTP communication information and the modification rules corresponding to the error labels into the feature library to obtain the first dataset; perform machine learning error labeling on the first HTTP communication information, and fill the second HTTP communication information and the modification rules corresponding to the error labels into the feature library to obtain the second dataset.

[0068] S14: Construct a matching rule base using the first dataset and the second dataset.

[0069] In steps S11 to S14, a transparent proxy is first configured for the client, at which point the rules in the proxy are empty. Several websites are selected, and the HTTP communication message information, status codes, HTTP request headers, response headers, and console exception information between the client and the proxy are manually obtained and filled into the corresponding attribute fields of the feature table. Then, an automated tool is used to access other websites, and the message information, status codes, HTTP request headers, response headers, and console exception information between the abnormal client and the proxy are extracted and filled into a new feature table that is isomorphic to the feature table. A semi-supervised machine learning method is used to label the error types of the data accessed from other websites and fill in the modification rules. After verification, the data is merged into the feature library. A matching rule library is constructed based on the feature library.

[0070] In semi-supervised learning, a dataset needs to be constructed. The specific steps are to merge the first and second datasets, assign different weights to each attribute according to the attribute type, normalize the features of the datasets, and then use the K-Means algorithm to perform K-classification on the merged dataset. The specific clustering steps are as follows.

[0071] First, the value of K is determined. The parameter K in the K-Means algorithm represents the number of clusters generated; therefore, the algorithm involves selecting the value of K. Verification based on actual data shows that the sum of intra-cluster distances is inversely proportional to the number of clusters. When K is less than the actual number of clusters, the sum of cluster distances decreases sharply as K increases. When K is equal to or greater than the actual number of clusters, the decreasing trend of the intra-cluster sum is very slow. The intra-cluster distance sums after K-Means clustering were calculated for K = 1, 2, 3, 4, ..., n. When the intra-cluster distance sum reaches a relatively stable value, the determined K value is closer to the actual value.

[0072] After selecting the K value, K sample points are randomly selected from the sample as the initial cluster centers. The distance between each sample point and the K cluster centers is calculated to obtain the cluster center closest to the sample point. The sample point is then assigned to the cluster. The center of each cluster is represented by the center of all sample points in each cluster.

[0073] Repeat the above steps until the cluster center remains unchanged or is less than a certain threshold, at which point K-Means clustering ends.

[0074] For each class, iterate through it, calculate the number of C records labeled in the cluster, and the label types. Calculate the label types with the highest proportion, and then use the rules of the label type to process the abnormal messages in the category. After modification, verify whether it is normal. If it is normal, move the attribute features, labels and rules to the feature table and end. If it is abnormal, repeat the above steps. If it is still abnormal after three iterations, mark it as unprocessable and then handle it manually.

[0075] If there is no labeled data in a cluster, a few abnormal data points in that cluster need to be manually processed and labeled. Subsequent iterations can then process them automatically. Thus, through the above steps, only a small portion of the data needs to be manually calibrated. The feature table and rules are constructed through iterative execution of the above algorithm, greatly improving the efficiency of manual calibration.

[0076] S2: Construct a balanced binary rule tree from the matching rule base.

[0077] As the number of compatible websites or services increases, the number of feature attributes in the feature table will continue to grow. To perform rule matching more quickly or efficiently, the feature table is converted into a matching rule table of the following type: rules contained in the Rule field of the feature table are extracted into general rules, while other non-general parts are retained. Furthermore, one feature vector corresponds to one rule in the matching rule table, and the attrType field in the Rule field is extracted as the rule type.

[0078] The following is an example of the structure of the matching rules table. The core fields include the following:

[0079] Rule UUID: A unique ID describing the rule;

[0080] Domain name: Describes the scope of application of this rule, usually represented by a domain name. A domain name with an asterisk (*) indicates a general rule that all requests need to match.

[0081] Subdomains: Describes the subdomains under the main domain to which this rule applies;

[0082] Path: Describes the specific path of the request or response to which this rule applies;

[0083] Rule types: Rule types mainly include General, Request Header, Request Content, Response Header, Response Content, and Custom. General corresponds to the general information of the request, Request Header corresponds to the request header information, Request Content corresponds to the request message content, Response Header corresponds to the response header information, Response Content corresponds to the response message content, and Custom corresponds to custom code, which usually solves more complex rule matching.

[0084] Rule content: The rule content describes the specific rules to be executed.

[0085] Specifically, the balanced binary rule tree in step S2 includes a request message balanced binary rule tree and a response message balanced binary rule tree.

[0086] When there are a large number of compatible websites, the rule table will grow dramatically, and real-time queries to match the rule table will take a long time, impacting query performance. To improve query performance, the matching rule table is converted into a balanced binary rule tree, increasing the time complexity from n to log n.

[0087] Based on the matching rule table, two balanced binary rule trees are constructed: a balanced binary rule tree for request messages and a balanced binary rule tree for response messages. The request message rule tree contains rules of types General, RequestHeader, and Request Content, while the response message rule tree contains rules of types Response Header, Response Content, and Custom. First, rules are categorized by rule type into request rules (Request) and response rules (Response).

[0088] The following example, using the Request type, illustrates the construction approach of a balanced binary rule tree.

[0089] First, determine the key-value pairs of the balanced binary tree nodes. The key is a string consisting of "domain name & subdomain name & path", and the value is a List of specific rules.

[0090] Traverse the rule table of type Request. If the domain name is *, add it to the list of approved rules. Otherwise, proceed to the process of constructing a balanced binary tree.

[0091] When constructing a balanced binary tree, if the Key value of the node to be inserted is the same as the Key value of a node in the balanced binary tree, it means that there are multiple specific rules corresponding to that Key. In this case, it is not necessary to insert the node into the balanced binary tree, but only to insert the corresponding rule into the specific rule list of the Value value. Otherwise, proceed to the next step.

[0092] If the key value of the node to be inserted is different from the key value of the node in the balanced binary tree, then the node is inserted into the binary tree, and the binary tree remains a balanced binary tree.

[0093] Once the balanced binary tree is constructed, a list of rules will be inserted into each node of the balanced binary tree to ensure that the general rules are matched.

[0094] By following the steps above, a balanced binary rule tree of type Request can be constructed. Similarly, a balanced binary rule tree of type Response can also be constructed.

[0095] S3: Using the balanced binary rule tree, perform service conversion on the HTTPS service according to the matching rule base.

[0096] In step S3, when the proxy receives client data, it performs matching rule base matching through the balanced binary rule tree of the request message and performs service conversion according to the matched rule; when the proxy receives server data, it performs matching rule base matching through the balanced binary rule tree of the request message and performs service conversion according to the matched rule.

[0097] Step S3 further includes:

[0098] S31: The proxy receives the client's input HTTPS service request and redirects the input HTTPS service request to an HTTP service request;

[0099] S32: The agent performs general rule matching and specified rule matching on the HTTP service request according to the matching rule base to obtain the output HTTPS service request;

[0100] S33: The server receives the output HTTPS service request and returns a response message;

[0101] S34: The agent performs general rule matching and specified rule matching on the response message according to the matching rule base to obtain the modified message;

[0102] S35: The agent returns the modified message to the client via the HTTP protocol.

[0103] In step S3, the high-speed HTTPS service protocol conversion is implemented through the matching rule base in step S1 and the balanced binary rule tree in step S2. The specific matching process is as follows.

[0104] When the agent receives a message, it first determines the message type, whether it is a Request or a Response; it assigns the ruleType value to the message to match the rule type; it traverses the balanced binary rule tree corresponding to the ruleType; it checks the number of matching nodes after traversing the balanced binary rule tree, and if it is equal to 1, it executes the subsequent steps, otherwise it ends; it executes the general rules and specific rule lists of the matching nodes, and at this point, the ruleType of a message has been executed.

[0105] Step S3 of this invention aims to add a rule tree-based forward proxy between the client browser and the HTTPS server. The proxy performs rule matching on HTTPS messages, and the matched messages are converted into HTTP messages. The following describes the implementation of HTTPS service protocol conversion in detail, taking access to xxx1 as an example, in conjunction with the description of steps S31 to S34 above.

[0106] like Figure 3 As shown, when a user initiates an HTTPS service request by entering https: / / www.xxx1.com in their browser, the proxy receives the HTTPS request and redirects it to http: / / www.xxx1.com, forcing the proxy to establish an HTTP connection with the client. Here, Browser represents the user's browser, and Server represents the server.

[0107] After the client browser is redirected, the HTTP service request first reaches the proxy. The proxy queries the balanced binary rule tree of the request message. After matching the binary tree node, it first performs general rule matching in the node, and then matches the specific rules of the www.xxx1.com service. After matching, http: / / www.xxx1.com is converted to https: / / www.xxx1.com, and the proxy sends the HTTPS-based request to the xxx1 server.

[0108] The specific matching rules are stored in the rule list of the binary tree node. Example of a rule is shown below.

[0109] Remove the cache policy parameters If-Modified-Since and Cache-Control; remove the security policy parameter Upgrade-Insecure-Requests for upgrading to HTTPS; replace http with https in the request URL and change the port to port 443.

[0110] After receiving a request from https: / / www.xxx1.com, the xxx1 server will return a response message. The proxy queries the balanced binary rule tree of the response message, matches the node, traverses the node's general rules and specific rules, and processes the message. An example of the rules is as follows.

[0111] Remove security-related parameters from the response header, such as Cross-Origin-Opener-Policy, Cross-Origin-Embedder-Policy, Cross-Origin-Resource-Policy, Access-Control-Allow-Origin, Cross-Origin-Opener-Policy-Report-Only, Strict-Transport-Security, X-Frame-Options, and Public-Key-Pins, to prevent the browser from forcibly converting HTTP to HTTPS. When a specific service sets a response strategy, the response header content also needs to be modified accordingly. For example, if Access-Control-Allow-Origin is required to be non-*, then Access-Control-Allow-Origin needs to be set to the value of the request header Origin or another value; change the HTTPS protocol in the URL, Refer, and Location headers of the response message to HTTP; change the links referenced in the response message content to HTTP protocol. The message type may be Document or Script. After modifying the message content, HTTPS request services will no longer be introduced; modify the attributes of specific cookies, remove the SameSite attribute, and remove cookies with specific tags.

[0112] The agent eventually returns a modified HTTP-based response message.

[0113] S4: Perform HTTP traffic monitoring on the HTTPS service after service conversion.

[0114] After steps S1 to S3, the process of adding a forward proxy between the user's browser and the xxx1 server has been completed. The proxy is then used to match rules for the accessed services based on a rule base. After the rules are matched, the HTTPS service can be downgraded to the HTTP service. At this point, the HTTPS traffic is converted into HTTP traffic, and the HTTPS traffic can be monitored using traditional traffic monitoring devices or software.

[0115] like Figure 2 As shown, the present invention also provides an HTTPS traffic monitoring device based on a rule tree, comprising:

[0116] Rule base building module 100: Used to build a matching rule base based on machine learning;

[0117] Rule tree construction module 200: used to construct a balanced binary rule tree from the matching rule library;

[0118] Conversion module 300: used to perform service conversion on HTTPS services according to the matching rule base through the balanced binary rule tree;

[0119] Monitoring module 400: Used to monitor HTTP traffic of the HTTPS service after service conversion.

[0120] This invention also provides an HTTPS traffic monitoring device based on a rule tree, comprising:

[0121] A memory and at least one processor, wherein the memory stores instructions;

[0122] At least one of the processors invokes the instructions in the memory to cause the rule-tree-based HTTPS traffic monitoring device to perform the rule-tree-based HTTPS traffic monitoring method as described in any of the preceding embodiments.

[0123] The present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the rule tree-based HTTPS traffic monitoring method as described in any of the preceding claims.

[0124] Furthermore, the rule-tree-based HTTPS traffic monitoring device provided by this invention can vary considerably depending on configuration or performance. It may include one or more central processing units (CPUs), for example, one or more processors and memory, and one or more storage media for storing applications or data, such as one or more mass storage devices. The memory and storage media may be temporary or persistent storage. The program stored in the storage media may include one or more modules, each module may include a series of instruction operations in the rule-tree-based HTTPS traffic monitoring device. Furthermore, the processor may be configured to communicate with the storage media and execute a series of instruction operations in the storage media on the rule-tree-based HTTPS traffic monitoring device.

[0125] It may also include one or more power supplies, one or more wired or wireless network interfaces, one or more input / output interfaces, and one or more operating systems, such as Windows Server, MacOSX, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that the rule-tree-based HTTPS traffic monitoring device structure provided by this invention does not constitute a limitation on the device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0126] This invention provides a rule-tree-based HTTPS traffic monitoring method, apparatus, device, and storage medium. By constructing a matching rule base and utilizing a balanced binary rule tree for service transformation, it can more accurately and quickly match and identify key information in HTTPS traffic, thereby improving the accuracy and efficiency of traffic monitoring. Compared to traditional monitoring methods, it can more effectively handle complex HTTPS communication data. In the process of constructing the matching rule base, this invention employs machine learning technology to automatically label errors in HTTP communication information and generate corresponding modification rules, thus greatly enhancing the automation and intelligence level of the system, reducing the need for manual intervention, and improving work efficiency. By configuring a proxy, this invention can achieve seamless transformation and monitoring of HTTPS traffic without requiring additional configuration or modification to the client, ensuring a normal user experience while achieving effective monitoring of HTTPS traffic. The matching rule base and balanced binary rule tree design of this invention have good scalability and flexibility, and can be dynamically adjusted and expanded according to actual needs, helping to adapt to the ever-changing network security environment and business requirements.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A rule-tree-based HTTPS traffic monitoring method, characterized in that, include: S1: Building a matching rule base based on machine learning; Step S1 further includes: S11: Configuring a transparent proxy for the client, selecting two groups of websites for proxying respectively; S12: Collecting HTTP communication information between the client and the proxy respectively, the HTTP communication information including first HTTP communication information and second HTTP communication information; S13: Manually labeling errors in the first HTTP communication information, and filling the first HTTP communication information and the modification rules corresponding to the error labels into the feature library to obtain a first dataset; performing machine learning error labeling on the first HTTP communication information, and filling the second HTTP communication information and the modification rules corresponding to the error labels into the feature library to obtain a second dataset; S14: Constructing a matching rule library using the first dataset and the second dataset; S2: Construct a balanced binary rule tree from the matching rule base; S3: Using the balanced binary rule tree, perform service conversion on the HTTPS service according to the matching rule base; S4: Perform HTTP traffic monitoring on the HTTPS service after service conversion.

2. The HTTPS traffic monitoring method based on rule trees according to claim 1, characterized in that, The HTTP communication information in step S12 specifically includes: message information, status code, HTTP request header, HTTP response header, and control exception information.

3. The HTTPS traffic monitoring method based on rule trees according to claim 1, characterized in that, The matching rule base in step S1 specifically includes: an attribute type table, a feature attribute and type relationship table, a feature table, and a matching rule table.

4. The HTTPS traffic monitoring method based on rule trees according to claim 1, characterized in that, The balanced binary rule tree in step S2 specifically includes: a balanced binary rule tree for request messages and a balanced binary rule tree for response messages.

5. The HTTPS traffic monitoring method based on rule trees according to claim 4, characterized in that, In step S3, when the proxy receives client data, it performs matching rule base matching through the balanced binary rule tree of the request message and performs service conversion according to the matched rule; when the proxy receives server data, it performs matching rule base matching through the balanced binary rule tree of the request message and performs service conversion according to the matched rule.

6. The HTTPS traffic monitoring method based on rule trees according to claim 1, characterized in that, Step S3 further includes: S31: The proxy receives the client's input HTTPS service request and redirects the input HTTPS service request to an HTTP service request; S32: The agent performs general rule matching and specified rule matching on the HTTP service request according to the matching rule base to obtain the output HTTPS service request; S33: The server receives the output HTTPS service request and returns a response message; S34: The agent performs general rule matching and specified rule matching on the response message according to the matching rule base to obtain the modified message; S35: The agent returns the modified message to the client via the HTTP protocol.

7. A rule-tree-based HTTPS traffic monitoring device, used to perform the method as described in any one of claims 1 to 6, characterized in that, include: Rule base building module: used to build a matching rule base based on machine learning; The rule base construction module is further used to: configure transparent proxies for clients and select two groups of websites for proxying respectively; HTTP communication information between the client and the agent is collected, including first HTTP communication information and second HTTP communication information. The first HTTP communication information is manually annotated with errors, and the first HTTP communication information and the corresponding modification rules are entered into a feature library to obtain a first dataset. The first HTTP communication information is then annotated with machine learning errors, and the second HTTP communication information and the corresponding modification rules are entered into the feature library to obtain a second dataset. A matching rule library is constructed using the first dataset and the second dataset. Rule tree construction module: used to construct a balanced binary rule tree from the matching rule library; Conversion module: used to convert HTTPS services according to the matching rule base using the balanced binary rule tree; Monitoring module: Used to monitor HTTP traffic for the converted HTTPS service.

8. A rule-tree-based HTTPS traffic monitoring device, characterized in that, include: A memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the rule-tree-based HTTPS traffic monitoring device to perform the rule-tree-based HTTPS traffic monitoring method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a processor, implement the rule-tree-based HTTPS traffic monitoring method as described in any one of claims 1 to 6.

Images