HTTPS encrypted traffic auditing method, device, system and medium

Abstract

The invention relates to an HTTPS encrypted traffic auditing method, device and system and a medium. The method comprises: receiving and processing a key packet from a client to obtain a pre-master key and TLS session metadata, the session metadata comprising a client random number, a server random number and a cipher suite identifier; performing key derivation processing on the pre-master key based on the TLS session metadata to obtain a key parameter required for decrypting the HTTPS flow; when the mirrored HTTPS flow is obtained, the secret key parameter is utilized to decrypt the HTTPS flow, and plaintext application layer data is obtained; and performing content auditing on the application layer data. According to the invention, the method effectively avoids the single-point fault and certificate trust risk of a man-in-the-middle agent, overcomes the ineffectiveness of a private key decryption scheme to a forward secrecy algorithm, avoids the system resource occupation and deployment complexity of a client agent, and achieves the efficient, compliant and bypass type auditing of the national secret and standard encrypted traffic including ECDHE and the like.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method, device, system and medium for HTTPS encrypted traffic auditing. Background Technology

[0002] With the widespread increase in internet security awareness, the HTTPS (Hypertext Transfer Protocol Secure) protocol has become a global standard for network communication. HTTPS achieves end-to-end communication encryption by introducing an SSL / TLS encryption layer on top of the HTTP protocol, effectively protecting user privacy and data transmission security.

[0003] However, the widespread adoption of HTTPS has also brought serious challenges to cybersecurity supervision and compliance auditing. In sectors with stringent data security requirements, such as finance, government, and enterprises, content auditing of network traffic is necessary to prevent security risks such as data leaks and unauthorized transmissions. For example, China's Cybersecurity Classified Protection 2.0 standard, Level 3, explicitly requires "auditing encrypted communication content," and Article 32 of the EU's GDPR also stipulates the need to monitor sensitive data transmissions. However, the encrypted nature of HTTPS renders traditional plaintext-based auditing methods completely ineffective, making encrypted traffic a regulatory blind spot.

[0004] Currently, the industry mainly uses the following technical solutions for auditing HTTPS encrypted traffic: 1. Man-in-the-Middle Proxy Solution This solution deploys a proxy server between the client and the server, with the proxy establishing independent TLS connections with both the client and the server. The proxy server forges the server certificate to communicate with the client while simultaneously establishing a connection with the real server, thereby enabling traffic decryption and auditing at the proxy node.

[0005] This method has the following drawbacks: 1) A proxy server needs to be deployed in series in the communication link, which poses a single point of failure risk; 2) The client is required to forcibly trust the proxy's CA certificate, which may trigger browser security warnings and affect user experience; 3) Unable to audit websites that have enabled HSTS (HTTP Strict Transport Security) for mandatory encryption; 4) It can only audit the current session and cannot perform retrospective analysis on historical encrypted traffic.

[0006] 2. Server private key decryption scheme This scheme uses the HTTPS server's private key to decrypt the pre-master secret in the TLS handshake, and then generates a session key to decrypt the traffic.

[0007] This method has the following drawbacks: 1) Only applicable to TLS connections using non-forward-secrecy algorithms such as RSA; 2) Connections using forward secrecy algorithms such as ECDHE (Elliptic Curve Diffie-Hellman Ephemeral) are completely ineffective because forward secrecy algorithms use ephemeral key negotiation, and the server's private key cannot decrypt historical sessions; 3) Centralized hosting of server private keys poses a risk of leakage, making security difficult to guarantee; 4) Poor compatibility with national cryptographic algorithms (such as SM2 and SM4).

[0008] 3. Client-side proxy solution Deploy a local proxy process on the client device to redirect all network traffic to the local proxy port, whereby the proxy handles the decryption, auditing, and re-encryption of HTTPS traffic.

[0009] This method has the following drawbacks: 1) Root certificates or proxy settings need to be manually installed on each client, resulting in high deployment and maintenance costs; 2) Insufficient cross-platform support; different implementations are required for different operating systems. 3) The agent process runs continuously, consuming system resources and affecting client performance; 4) It also cannot support historical traffic auditing.

[0010] Statistics show that most TLS 1.3 connections in modern HTTPS communication use forward secrecy algorithms such as ECDHE. These algorithms achieve "forward secrecy" (FS) by negotiating a temporary key for each session. This means that even if the server's private key is leaked, attackers cannot decrypt historical communications. This security feature protects users' long-term privacy while rendering traditional auditing schemes based on server private keys completely ineffective.

[0011] Therefore, the core technical challenge currently facing HTTPS traffic auditing is how to effectively audit and control encrypted traffic while supporting modern encryption technologies such as forward secrecy algorithms and national cryptographic algorithms. This need is particularly urgent in scenarios such as financial transaction monitoring, government data leakage prevention, and enterprise compliance auditing. Summary of the Invention

[0012] This disclosure provides an HTTPS encrypted traffic auditing method, device, system, and medium, which can effectively decrypt and audit various types of encrypted traffic, including forward secrecy algorithms, without interfering with normal HTTPS encrypted communication.

[0013] This disclosure provides a method for auditing HTTPS encrypted traffic. The method includes: receiving and processing a key packet from a client to obtain a pre-master key and TLS session metadata, wherein the session metadata includes a client random number, a server random number, and a cipher suite identifier; performing key derivation processing on the pre-master key based on the TLS session metadata to obtain key parameters required to decrypt HTTPS traffic; when mirrored HTTPS traffic is obtained, decrypting the HTTPS traffic using the key parameters to obtain plaintext application layer data; and performing content auditing on the application layer data.

[0014] As in this embodiment of the invention, receiving and processing a key packet from the client to obtain a pre-master key and TLS session metadata includes: receiving encrypted transmission data sent by the client through a Transport Layer Security (TLS) channel via a secure service port; decrypting the encrypted transmission data using the TLS layer to obtain a key packet, the key packet including: pre-master key ciphertext and TLS session metadata; and decrypting the pre-master key ciphertext using the private key of the auditing device to obtain the pre-master key.

[0015] As in this embodiment of the invention, the TLS session metadata further includes: a timestamp and a message authentication code; before decrypting the pre-master key ciphertext using the private key of the auditing device, the method further includes: comparing and verifying the recalculated message authentication value with the message authentication code; comparing the timestamp with the current time to verify whether it is within a preset valid time window; if the message authentication code verification passes and the timestamp verification is valid, then the decryption operation of the pre-master key ciphertext continues; if the message authentication code verification fails or the timestamp verification is invalid, then the key packet is discarded and the processing flow of the key packet is terminated.

[0016] As in this embodiment of the invention, the step of performing key derivation processing on the pre-master key based on the TLS session metadata to obtain the key parameters required for decrypting HTTPS traffic includes: determining the corresponding hash algorithm according to the cipher suite identifier; using the pre-master key as input key material, performing a key derivation function extraction operation based on the hash algorithm to obtain an intermediate key; and using the intermediate key, client random number, and server random number as input, performing a labeled key derivation function expansion operation to obtain the key parameters.

[0017] As in this embodiment of the invention, the key parameters include a client-side write key and a server-side write key; the step of decrypting the HTTPS traffic using the key parameters includes: identifying the transmission direction of the HTTPS traffic; if the transmission direction is from the client to the server, then the client-side write key is used for decryption; if the transmission direction is from the server to the client, then the server-side write key is used for decryption.

[0018] As in this embodiment of the invention, when the HTTPS traffic of the mirror is obtained, decrypting the HTTPS traffic using the key parameters includes: determining the corresponding key parameters for the HTTPS traffic based on the association between the HTTPS traffic and the key parameters; and decrypting the HTTPS traffic using the determined key parameters.

[0019] As in this embodiment of the invention, the content auditing of the application layer data includes: using a regular expression engine to match sensitive data in the application layer data, wherein the sensitive data includes at least one of credit card number and ID card number; when sensitive data is matched and meets the preset blocking policy, based on the connection 5-tuple of the HTTPS traffic, forging TCP RST packets pointing to the client and server respectively and injecting them into the network to forcibly terminate the corresponding HTTPS connection; wherein the connection 5-tuple includes source IP address, source port, destination IP address, destination port and protocol type.

[0020] According to one aspect of the embodiments of this application, an HTTPS encrypted traffic auditing device is disclosed, wherein the device includes: a receiving and processing module configured to receive and process a key packet from a client to obtain a pre-master key and TLS session metadata, the session metadata including: a client random number, a server random number, and a cipher suite identifier; a key derivation module configured to perform key derivation processing on the pre-master key based on the TLS session metadata to obtain key parameters required for decrypting HTTPS traffic; a traffic decryption module configured to decrypt the HTTPS traffic using the key parameters when mirrored HTTPS traffic is obtained to obtain plaintext application layer data; and a content auditing module configured to perform content auditing on the application layer data.

[0021] As in this embodiment of the invention, the receiving processing module is configured to receive encrypted transmission data sent by the client through a Transport Layer Security (TLS) channel via a secure service port; decrypt the encrypted transmission data using the TLS layer to obtain a key packet, the key packet including: a pre-master key ciphertext and TLS session metadata; and decrypt the pre-master key ciphertext using the private key of the auditing device to obtain the pre-master key.

[0022] As in this embodiment of the invention, the TLS session metadata further includes: a timestamp and a message authentication code; before decrypting the pre-master key ciphertext using the private key of the auditing device, the receiving processing module is further configured to compare and verify the recalculated message authentication value with the message authentication code; compare the timestamp with the current time to verify whether it is within a preset valid time window; if the message authentication code verification passes and the timestamp verification is valid, then the decryption operation of the pre-master key ciphertext continues; if the message authentication code verification fails or the timestamp verification is invalid, then the key packet is discarded and the processing flow of the key packet is terminated.

[0023] In this embodiment of the invention, the key derivation module is configured to: determine the corresponding hash algorithm based on the cryptographic suite identifier; use the pre-master key as input key material, perform a key derivation function extraction operation based on the hash algorithm to obtain an intermediate key; and use the intermediate key, client random number, and server random number as input, perform a labeled key derivation function expansion operation to obtain the key parameters.

[0024] As in this embodiment of the invention, the key parameters include a client-side write key and a server-side write key; the traffic decryption module is configured to identify the transmission direction of the HTTPS traffic; if the transmission direction is from the client to the server, the client-side write key is used for decryption; if the transmission direction is from the server to the client, the server-side write key is used for decryption.

[0025] As in this embodiment of the invention, the traffic decryption module is configured to: determine the corresponding key parameters for the HTTPS traffic based on the association between the HTTPS traffic and the key parameters; and decrypt the HTTPS traffic using the determined key parameters.

[0026] In this embodiment of the invention, the content auditing module is configured to use a regular expression engine to match sensitive data in the application layer data. The sensitive data includes at least one of credit card numbers and ID card numbers. When sensitive data is matched and meets the preset blocking policy, based on the connection 5-tuple of the HTTPS traffic, TCP RST packets pointing to the client and server are forged and injected into the network to forcibly terminate the corresponding HTTPS connection. The connection 5-tuple includes the source IP address, source port, destination IP address, destination port, and protocol type.

[0027] According to one aspect of the embodiments of this application, an electronic device is disclosed, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods provided in the various optional implementations described above.

[0028] According to one aspect of the embodiments of this application, a computer program medium is disclosed, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the methods provided in the various optional implementations described above.

[0029] In this embodiment of the invention, a key packet from a client is received and processed to obtain a pre-master key and TLS session metadata. The session metadata includes a client random number, a server random number, and a cipher suite identifier. Based on the TLS session metadata, the pre-master key is used for key derivation to obtain the key parameters required to decrypt HTTPS traffic. When the mirrored HTTPS traffic is obtained, the HTTPS traffic is decrypted using the key parameters to obtain plaintext application layer data. Content auditing is performed on the application layer data. By decrypting the received client key packet and reconstructing the key parameters, the auditing device can directly perform real-time decryption and content auditing of mirrored HTTPS traffic without intervening in the normal communication link, without obtaining the server's private key, and without relying on a specific encryption algorithm. This method effectively avoids the single point of failure and certificate trust risks of man-in-the-middle proxies, overcomes the ineffectiveness of private key decryption schemes for forward secrecy algorithms, and avoids the system resource consumption and deployment complexity of client proxies. It achieves efficient, compliant, and bypass auditing of traffic encrypted with national cryptographic standards such as ECDHE.

[0030] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0031] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0033] Figure 1 A flowchart of the HTTPS encrypted traffic auditing method provided in this application is shown.

[0034] Figure 2 A block diagram of the HTTPS encrypted traffic auditing apparatus provided in this application is shown.

[0035] Figure 3 A block diagram of the electronic device provided in this application is shown. Detailed Implementation

[0036] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0037] Furthermore, the accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0038] In this disclosure, at least one item can be described as one item or multiple items, and multiple items can be two, three, four, or more items, without limitation. The " / " sign can indicate that the related objects are in an "or" relationship; for example, A / B can mean A or B. "And / or" can be used to describe three relationships between related objects; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any design described as "exemplary" or "for example" should not be construed as superior or more advantageous than other design solutions. The use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0039] The HTTPS encrypted traffic auditing method proposed in this invention will be described in detail below with reference to specific embodiments.

[0040] First, this invention provides an HTTPS encrypted traffic auditing system based on bypass decryption and dynamic key reconstruction. This system adopts an architecture of bypass capture, terminal collaboration, and centralized auditing, achieving transparent auditing of HTTPS / TLS encrypted traffic without interrupting business operations or reducing security. The system mainly includes the following three core components: 1. Bypass Traffic Capture Module: This module is deployed on the mirrored port (SPAN) of the network switch to capture target HTTPS traffic without interference. This module can be a hardware or a combination of hardware and software, and its functions include: Traffic mirroring reception: Using the port mirroring function of a network switch, HTTPS bidirectional traffic passing through the target port is copied; Traffic preprocessing: Performing session reassembly, protocol identification (TLS 1.2 / 1.3), and metadata extraction on the captured raw packets; Traffic forwarding: The pre-processed encrypted traffic is forwarded to the audit processing engine through a secure tunnel, or written directly to local high-speed storage for subsequent processing.

[0041] The bypass traffic capture module can be integrated into a dedicated hardware probe, supporting line-rate capture of 10 Gigabit and higher speeds. This module can also run in software on servers equipped with multi-queue network cards and DPDK / PF_RING acceleration.

[0042] 2. Client Key Plugin: Deployed as a lightweight process on the terminal device, it dynamically extracts the TLS session key and securely uploads it. The client key plugin supports deployment on multiple platforms. Windows environment: Installed as a system service, the session key is extracted via TLS library hooking or memory scanning; Linux environment: Runs as a daemon process, using LD_PRELOAD or eBPF technology to extract keys; macOS environment: Key capture is implemented as a kernel extension (KEXT) or system extension.

[0043] After the TLS handshake is complete, the client key plugin extracts the master key or session key from the client process memory and uploads it to the key coordination subsystem of the audit processing engine via HTTPS or a dedicated encryption protocol. The plugin has self-protection, tamper-proof, and stealth operation capabilities, and is transparent to the user.

[0044] 3. Audit Processing Engine: The audit processing engine is the core processing unit of this invention, responsible for centrally completing the decryption, analysis, and auditing tasks of HTTPS encrypted traffic. This engine receives the TLS session key from the client key plugin and encrypted network traffic from the bypass traffic capture module, and achieves transparent auditing of encrypted traffic through dynamic key matching and hardware-accelerated decryption technology.

[0045] 3.1 Key Reception and Management The audit processing engine receives TLS session keys uploaded by the client key plugin through a secure encrypted channel, establishing and maintaining a mapping between traffic identifiers and keys. The engine verifies the legitimacy of the key source, manages the key's lifecycle, and provides accurate key matching support for the decryption process.

[0046] 3.2 Hardware-accelerated decryption The audit processing engine utilizes dedicated hardware acceleration to efficiently decrypt captured encrypted HTTPS traffic. By matching the corresponding session key, the engine converts the encrypted traffic into a plaintext data stream, providing readable network communication data for subsequent content analysis.

[0047] 3.3 Strategy Audit and Analysis The audit processing engine performs deep content inspection and security policy matching on the decrypted plaintext traffic. Based on preset security rules, the engine identifies security threats such as unauthorized access, data breaches, and malicious behavior, generating detailed audit reports and security incident alerts.

[0048] 3.4 Audit Result Output The audit processing engine outputs structured audit logs, compliance reports, and real-time security alerts, supporting integration with existing security management systems. Audit results meet cybersecurity regulatory requirements, providing enterprises with traceable HTTPS communication auditing capabilities.

[0049] The bypass traffic capture module, client key plugin, and audit processing engine communicate through a secure encrypted channel to ensure the confidentiality and integrity of keys and traffic data during transmission.

[0050] When deploying the above components, the bypass traffic capture module and audit processing engine can be integrated into the same hardware device. This device is equipped with a multi-port network interface card (NIC), with one port connected to the switch's SPAN port and the remaining ports used for management and data output. It is suitable for small to medium-sized network environments, offering simple deployment and centralized maintenance.

[0051] During deployment, the bypass traffic capture module can also be deployed as an independent probe in each network partition, while the audit processing engine can be deployed in a cluster in the data center. Key coordination, hardware decryption, and policy auditing can be deployed on different servers and work together with the load balancer through message queues, making it suitable for high-traffic scenarios such as large enterprises and carriers.

[0052] The following section uses audit devices equipped with audit processing engines as the execution entity, combined with the attached... Figure 1 This application provides a detailed description of the HTTPS encrypted traffic auditing method provided, such as... Figure 1 As shown, the method may include, but is not limited to, the following steps: Step S110: Receive and process the key packet from the client to obtain the pre-master key and TLS session metadata, wherein the session metadata includes: client random number, server random number and cipher suite identifier.

[0053] In this embodiment of the invention, the auditing device can receive encrypted transmission data sent by the client through the Transport Layer Security (TLS) channel via the secure service port, decrypt the encrypted transmission data using the TLS layer to obtain a key packet, the key packet including: pre-master key ciphertext and TLS session metadata; and decrypt the pre-master key ciphertext using the auditing device's private key to obtain the pre-master key.

[0054] It should be noted that the client has a client key plugin deployed. When the client accesses the HTTPS service, it completes a standard TLS handshake. Then, the key plugin extracts key parameters from memory using hooking technology. Pre-master key: The pre-master key is a temporary random key generated by the client during the SSL / TLS handshake process. It is used to negotiate the final master key with the server, and then derive the encryption keys required for the session (such as symmetric session keys).

[0055] Client-side random number (client_random): In the first step of the TLS handshake (ClientHello message), the client generates a 32-byte random number to participate in key calculations, ensuring the uniqueness and security of each session.

[0056] Server random number (server_random): In the second step of the TLS handshake (ServerHello message), the server generates a 32-byte random number (usually including a timestamp + cryptographically secure random number), which, together with the client random number (Client Random), participates in key calculation to ensure the security and uniqueness of the session.

[0057] Cipher suite identifier (cipher_suite): A string code in the SSL / TLS protocol used to uniquely identify a set of encryption algorithms. It defines the combination of key exchange algorithms, encryption algorithms, message authentication code (MAC) algorithms, and pseudo-random functions (PRFs) used in the handshake process and communication phases.

[0058] Before or after performing the above-mentioned key parameter extraction operation, the client key plugin performs a secure channel establishment and key negotiation process with the auditing device: the client key plugin initiates a TLS connection request to the auditing device's preset secure service port, and then both parties complete a standard TLS handshake. During the handshake process, the auditing device provides its digital certificate, the client key plugin verifies the validity of the certificate, and extracts the auditing device's public key from the certificate.

[0059] The client key plugin uses the obtained audit device public key to encrypt the extracted pre-master key, generating pre-master key ciphertext. Based on other extracted key parameters and the pre-master key ciphertext, a key packet is assembled and sent as application layer data to the audit device's secure service port via the established TLS secure channel (port 8443); [Master key]. Explain this feature in a short paragraph. The auditing device receives key packet data encrypted and transmitted via a TLS channel by the client plugin through its secure service port. It first performs TLS layer decryption to obtain a key packet containing the pre-master key ciphertext and TLS session metadata. The TLS session metadata may include client random numbers, server random numbers, and cipher suite identifiers. Then, it uses its own private key to perform asymmetric decryption of the pre-master key ciphertext to recover the original pre-master key. This provides the necessary foundational key material for subsequent key derivation and traffic decryption, ensuring the security and integrity of the key acquisition process.

[0060] In the above embodiments, a dedicated TLS channel is established between the client and the auditing device to achieve secure transmission of the key packet, avoiding the risk of the key being stolen or tampered with during transmission. Using the auditing device's private key to decrypt the pre-master key ensures that only authorized devices can obtain sensitive key information, improving the overall security of the system and providing a reliable key source for subsequent decryption operations.

[0061] The TLS session metadata may further include: a timestamp and a message authentication code. When generating a key packet, the client can assemble a key packet using a client-side random number, a server-side random number, a cipher suite identifier, the current timestamp, and the pre-master key ciphertext. Based on the key packet content, the client calculates the message authentication code and adds it to the key packet for subsequent integrity verification. The timestamp identifies the time the key packet was generated; the message authentication code is an authentication value used to verify the integrity and authenticity of the key packet.

[0062] Before decrypting the pre-master key ciphertext using the auditing device's private key, the auditing device can recalculate the message authentication value of the key packet and compare it with the message authentication code for verification. It also compares the timestamp with the current time to verify if it is within a preset valid time window. Specifically, the auditing device recalculates the message authentication value based on other fields in the key packet and compares it with the message authentication code carried in the packet to verify whether the data has been tampered with during transmission. Simultaneously, the auditing device parses the timestamp field, for example, a UNIX timestamp with a value of 1625097600 (corresponding to 2021-07-01 00:00:00 UTC), and compares it with the auditing device's current time. If the current time is 1625097615 (i.e., 2021-07-01 00:00:15 UTC), and the preset valid time window is ±300 seconds, then the timestamp falls within the valid window, and the verification passes. If the current time has exceeded 1625097900 (i.e., 2021-07-01 00:05:00 UTC), then the timestamp is invalid, and the verification fails. The auditing device will only continue processing the key packet if the message authentication code matches and the timestamp is within the valid window; if either verification fails, the packet is immediately discarded and the process terminates. That is, if the message authentication code verification passes and the timestamp verification is valid, the decryption operation of the pre-master key ciphertext continues; if the message authentication code verification fails or the timestamp verification is invalid, the key packet is discarded and the processing of the key packet is terminated.

[0063] In the above embodiments, by introducing a dual verification mechanism of timestamps and message authentication codes, replay attacks and data tampering threats are effectively resisted, enhancing the security and integrity of the key transmission process. Compared to existing technologies that may rely on only a single encryption method, this method improves the verification strength of the key packet, preventing expired or forged keys from being misused, thereby enhancing the system's anti-attack capability and audit accuracy.

[0064] It should be noted that TLS session metadata can also include a TLS session identifier. This identifier is used by the subsequent audit processing engine to accurately associate the decrypted plaintext traffic with the corresponding TLS session metadata, supporting session-level audit tracing, behavior analysis, and compliance report generation.

[0065] The following is a key package provided in an embodiment of the present invention: message KeyPackage { string session_id = 1; / / TLS session ID bytes client_random = 2; / / 32-byte random number bytes server_random = 3; CipherSuite cipher = 4; / / CipherSuite enumeration oneof encrypted_key { RSACipher rsa_key = 5; / / RSA encryption pre-master key SM2Cipher sm2_key = 6; / / SM2 encryption pre-master key } uint64 timestamp = 7; / / Timestamp (to prevent replay) bytes hmac = 8; / / HMAC-SHA256 signature } Step S120: Perform key derivation processing on the pre-master key based on the TLS session metadata to obtain the key parameters required to decrypt HTTPS traffic.

[0066] In this embodiment of the invention, the auditing device can determine the corresponding hash algorithm based on the cryptographic suite identifier, use the pre-master key as input key material, perform a key derivation function extraction operation based on the hash algorithm to obtain an intermediate key, and use the intermediate key, client random number and server random number as input to perform a tagged key derivation function expansion operation to obtain the key parameters.

[0067] Specifically, the auditing device can parse the cipher suite identifier (such as TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) in the TLS session metadata, extract the hash algorithm part (SHA384 in this case), and use it as the basis algorithm for subsequent key derivation.

[0068] Then, using the previously decrypted pre-master key as the key material, combined with the fixed label defined by the standard TLS (such as "extracted"), a key derivation function extraction operation based on the selected hash algorithm is performed. This step generates an intermediate key with a fixed length and stronger cryptographic strength, providing a secure entropy source for subsequent expansion.

[0069] The intermediate key obtained in the previous step is concatenated with the client's random number and the server's random number, and a specific tag (such as "client application traffic key" or "server application traffic key") is added. Then, a tagged key derivation function expansion operation is performed. This operation generates the key parameters required for actual encryption and decryption through multiple iterations of hashing.

[0070] The above embodiments implement a key derivation process fully compatible with the standard TLS protocol, ensuring that the key parameters generated by the auditing device are completely consistent with the keys actually used by both communicating parties, thereby supporting accurate decryption of encrypted traffic. By performing extraction and expansion operations step by step, the security and scalability of key derivation are improved, supporting adaptive processing of various cipher suites and enhancing the system's adaptability to different encryption environments.

[0071] It should be noted that key derivation function extraction is the process of extracting a fixed-length, cryptographically strong intermediate key from initial key material (such as a pre-master key). Its core principles are entropy concentration and randomization, ensuring uniform randomness in the output. In this invention, this operation typically uses a hash-based key derivation function (such as HKDF-Extract), taking the pre-master key as input and outputting a fixed-length intermediate key. Tag-based key derivation function expansion, on the other hand, uses the intermediate key as a basis, combined with specific tags (such as "client traffic key") and contextual information (such as client random numbers and server random numbers), to iteratively expand and generate one or more final key parameters (such as encryption keys, MAC keys, IVs, etc.). Its core principles are key separation and on-demand derivation, ensuring that keys for different purposes are independent of each other.

[0072] In this embodiment of the invention, the key parameter may include a client-side write key and a server-side write key. When decrypting the HTTPS traffic using the key parameter, the transmission direction of the HTTPS traffic can be identified. If the transmission direction is from the client to the server, the client-side write key is used for decryption; if the transmission direction is from the server to the client, the server-side write key is used for decryption.

[0073] Both the client-side write key and the server-side write key are directional session keys generated through key derivation function expansion operations. When decrypting HTTPS encrypted traffic using these key parameters, the transmission direction of the traffic must first be identified, and the corresponding write key must be selected accordingly for the decryption operation.

[0074] Regarding the directional distinction of key parameters, the client-write key refers to the symmetric key used in a TLS session to decrypt encrypted traffic sent from the client to the server (Client→Server). This key is used by the client to encrypt data during communication, and by the server to decrypt it. In this invention, the auditing device uses this key to decrypt client-to-client outbound traffic captured by the mirror. The server-write key refers to the symmetric key used in a TLS session to decrypt encrypted traffic sent from the server to the client (Server→Client). This key is used by the server to encrypt response data during communication, and by the client to decrypt it. In this invention, the auditing device uses this key to decrypt server-to-server outbound traffic captured by the mirror.

[0075] The auditing equipment determines the transmission direction of data packets by parsing the network layer and transport layer header information of the captured traffic: if the source IP / port matches the client and the destination IP / port matches the server, it is determined to be a client-to-server direction; if the source IP / port matches the server and the destination IP / port matches the client, it is determined to be a server-to-client direction.

[0076] Based on the transmission direction identification results, the auditing device can select a key according to the following rules: If the direction is client → server: use the client-written key for decryption; If the direction is server → client: use the server-written key for decryption; This selection mechanism ensures the consistency between the decryption key and the traffic encryption key, because bidirectional communication in the TLS protocol uses different write keys for encryption.

[0077] The above embodiments ensure the accuracy of decryption for bidirectional traffic by distinguishing between client-side and server-side write keys and selecting the appropriate key for decryption based on the traffic direction. Compared to existing technologies that may use only a single key or ignore directionality, this method strictly adheres to the TLS protocol specification, avoiding decryption failures or key misuse, and improving the decryption success rate and the integrity of audit content.

[0078] Step S130: When the HTTPS traffic of the mirror is obtained, the HTTPS traffic is decrypted using the key parameter to obtain plaintext application layer data.

[0079] In this embodiment of the invention, after key derivation, the auditing device can systematically establish and maintain an index structure, enabling the key parameters of each TLS session to be quickly retrieved and invoked based on their corresponding network session characteristics. Specifically, after key derivation, the auditing device establishes a multi-level index structure based on different dimensions of session characteristics: simultaneously associating key parameters with multiple identifier dimensions, including client random numbers, TLS session IDs, and 5-tuple information. The system maintains an independent hash index for each dimension and establishes cross-reference relationships between dimensions. When encrypted traffic arrives, the device attempts to query multiple identifier dimensions in parallel or sequentially, prioritizing protocol layer identifiers (such as client random numbers) and automatically downgrading to transport layer identifiers (such as 5-tuples) when they are unavailable. Each index entry is associated with a lifecycle marker, and expired data is periodically cleaned up by a background service. This mechanism, through redundant indexes and degradation strategies, ensures that traffic can be reliably mapped to the correct key parameters under different TLS versions, session states, and network environments.

[0080] In this embodiment of the invention, the auditing device determines the corresponding key parameters for the HTTPS traffic based on the correlation between the HTTPS traffic and the key parameters, and uses the determined key parameters to decrypt the HTTPS traffic. Specifically, the auditing device extracts session features (such as 5-tuples, TLS session IDs, or client random numbers) from the HTTPS traffic and performs a matching query in a pre-established key parameter index to assign a suitable client write key or server write key to the current traffic. Subsequently, the system uses the determined key parameters to call the corresponding symmetric decryption algorithm to decrypt the encrypted payload according to the traffic transmission direction (client to server or server to client), restoring the plaintext application layer data to support subsequent content auditing and security analysis.

[0081] By establishing and maintaining the association between HTTPS traffic and key parameters, a fast and accurate key matching mechanism is achieved. Compared to the potential low key lookup efficiency or mismatches in existing technologies, this method improves decryption performance under high-traffic environments and ensures that every encrypted traffic item can be correctly decrypted, supporting real-time or near real-time auditing requirements.

[0082] Step S140: Perform content auditing on the application layer data.

[0083] In this embodiment of the invention, a regular expression engine is used to match sensitive data in the application layer data. The sensitive data includes at least one of credit card number and ID card number. When sensitive data is matched and meets the preset blocking policy, based on the connection 5-tuple of the HTTPS traffic, TCP RST packets pointing to the client and server are forged and injected into the network to forcibly terminate the corresponding HTTPS connection. The connection 5-tuple includes source IP address, source port, destination IP address, destination port and protocol type.

[0084] In this embodiment of the invention, after decrypting HTTPS traffic, the auditing device submits the obtained application-layer plaintext data to a regular expression engine for sensitive content detection. This engine has pre-defined regular expression rules for various sensitive data patterns, including credit card numbers (such as a 16-digit number pattern following the Luhn algorithm) and ID card numbers (such as an 18-character pattern conforming to national standard encoding rules). The system performs a full scan of the decrypted HTTP request body, response body, and header fields. When a regular expression match matches a sensitive data pattern, a sensitive data leakage alarm is triggered.

[0085] If the matched sensitive data simultaneously meets the preset blocking policy (e.g., the data is transmitted to the Internet, the amount of sensitive data exceeds the threshold, or a high-risk sensitive type is matched), the auditing device will initiate an active blocking mechanism. Based on the TCP connection five-tuple information of the current HTTPS traffic—including source IP address, source port, destination IP address, destination port, and transport protocol type (TCP)—the device constructs two forged TCP RST (reset) packets. One RST packet, with the server IP and port as the source address, is sent to the client, notifying the client of an abnormal connection termination; the other RST packet, with the client IP and port as the source address, is sent to the server, notifying the server of an abnormal connection termination.

[0086] The auditing equipment injects these two forged RST packets into the network via bypass injection or switch cooperation. Upon receiving the RST packets, the client and server immediately tear down the corresponding TCP connections, thereby forcibly terminating the HTTPS session in the process of transmitting sensitive data. This blocking mechanism, after detecting the violation at the application layer, can achieve fast and proactive session interception at the transport layer, effectively preventing further leakage of sensitive data while maintaining zero interference with other network traffic.

[0087] In the above embodiments, a regular expression engine is introduced during the content auditing stage to perform high-precision sensitive data detection, and combined with an active blocking mechanism, the connection is terminated in real time by forging TCP RST packets when violations are detected. Compared with existing technologies that only perform audit logging without real-time blocking capabilities, this method achieves closed-loop security protection from detection to response, effectively preventing the leakage of sensitive data and improving the system's proactive defense capabilities and compliance control level.

[0088] In this embodiment of the invention, non-intrusive decryption and content auditing of mirrored HTTPS traffic are achieved by securely obtaining the session key from the client and deriving decryption parameters based on TLS metadata. This method effectively avoids single points of failure and certificate trust risks in man-in-the-middle proxy architectures, overcomes the ineffectiveness of traditional private key decryption schemes for forward secrecy algorithms (such as ECDHE), and avoids the system resource consumption and deployment complexity issues caused by client proxy modes. Thus, it achieves efficient, compliant, and bypass auditing of traffic encrypted with national cryptographic standards such as ECDHE, providing a complete and feasible technical path for encrypted traffic visualization and security compliance management while ensuring business continuity and protocol compatibility.

[0089] Figure 2 A block diagram of the HTTPS encrypted traffic auditing apparatus provided in this application is shown, the apparatus comprising: The receiving and processing module 210 is configured to receive and process a key packet from the client to obtain a pre-master key and TLS session metadata, wherein the session metadata includes: client random number, server random number and cipher suite identifier; The key derivation module 220 is configured to perform key derivation processing on the pre-master key based on the TLS session metadata to obtain the key parameters required to decrypt HTTPS traffic; The traffic decryption module 230 is configured to decrypt the HTTPS traffic using the key parameter when the mirrored HTTPS traffic is obtained, so as to obtain plaintext application layer data. The content auditing module 240 is configured to perform content auditing on the application layer data.

[0090] In an exemplary embodiment of this application, the receiving processing module 210 is configured to receive encrypted transmission data sent by the client through a Transport Layer Security (TLS) channel via a secure service port; decrypt the encrypted transmission data using the TLS layer to obtain a key packet, the key packet including: a pre-master key ciphertext and TLS session metadata; and decrypt the pre-master key ciphertext using the private key of the auditing device to obtain the pre-master key.

[0091] In an exemplary embodiment of this application, the TLS session metadata further includes: a timestamp and a message authentication code; before decrypting the pre-master key ciphertext using the private key of the auditing device, the receiving processing module 210 is further configured to compare and verify the recalculated message authentication value with the message authentication code; compare the timestamp with the current time to verify whether it is within a preset valid time window; if the message authentication code verification passes and the timestamp verification is valid, then the decryption operation of the pre-master key ciphertext continues; if the message authentication code verification fails or the timestamp verification is invalid, then the key packet is discarded and the processing flow of the key packet is terminated.

[0092] In an exemplary embodiment of this application, the key derivation module 220 is configured to: determine the corresponding hash algorithm based on the cryptographic suite identifier; use the pre-master key as input key material, perform a key derivation function extraction operation based on the hash algorithm to obtain an intermediate key; and use the intermediate key, client random number, and server random number as input, perform a labeled key derivation function expansion operation to obtain the key parameters.

[0093] In an exemplary embodiment of this application, the key parameters include a client-side write key and a server-side write key; the traffic decryption module 230 is configured to identify the transmission direction of the HTTPS traffic; if the transmission direction is from the client to the server, the client-side write key is used for decryption; if the transmission direction is from the server to the client, the server-side write key is used for decryption.

[0094] In an exemplary embodiment of this application, the traffic decryption module 230 is configured to: determine a corresponding key parameter for the HTTPS traffic based on the association between the HTTPS traffic and the key parameter; and decrypt the HTTPS traffic using the determined key parameter.

[0095] In an exemplary embodiment of this application, the content auditing module 240 is configured to use a regular expression engine to match sensitive data in the application layer data, the sensitive data including at least one of credit card number and ID card number; when sensitive data is matched and meets the preset blocking policy, based on the connection 5-tuple of the HTTPS traffic, TCP RST packets pointing to the client and server are forged and injected into the network to forcibly terminate the corresponding HTTPS connection; the connection 5-tuple includes source IP address, source port, destination IP address, destination port and protocol type.

[0096] In this embodiment of the invention, non-intrusive decryption and content auditing of mirrored HTTPS traffic are achieved by securely obtaining the session key from the client and deriving decryption parameters based on TLS metadata. This method effectively avoids single points of failure and certificate trust risks in man-in-the-middle proxy architectures, overcomes the ineffectiveness of traditional private key decryption schemes for forward secrecy algorithms (such as ECDHE), and avoids the system resource consumption and deployment complexity issues caused by client proxy modes. Thus, it achieves efficient, compliant, and bypass auditing of traffic encrypted with national cryptographic standards such as ECDHE, providing a complete and feasible technical path for encrypted traffic visualization and security compliance management while ensuring business continuity and protocol compatibility.

[0097] The following is for reference. Figure 3 To describe the electronic device 30 according to an embodiment of this application. Figure 3 A block diagram of the electronic device provided in this application is shown. Figure 3 The electronic device 30 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0098] like Figure 3 As shown, the electronic device 30 is presented in the form of a general-purpose computing device. The components of the electronic device 40 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, and a bus 340 connecting different system components (including storage unit 320 and processing unit 310).

[0099] The storage unit stores program code that can be executed by the processing unit 310, causing the processing unit 310 to perform the steps described in the exemplary method description section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 310 can perform actions such as... Figure 1 The steps shown.

[0100] Storage unit 320 may include readable media in the form of volatile storage units, such as random access memory (RAM) 3201 and / or cache memory 3202, and may further include read-only memory (ROM) 3203.

[0101] Storage unit 320 may also include a program / utility 3204 having a set (at least one) program module 3205, such program module 3205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0102] Bus 330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0103] Electronic device 30 can also communicate with one or more external devices 400 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 30, and / or with any device that enables electronic device 30 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 350. Input / output (I / O) interface 350 is connected to display unit 340. Furthermore, electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 360. As shown, network adapter 360 communicates with other modules of electronic device 30 via bus 330. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 30, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0104] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this application.

[0105] In an exemplary embodiment of this application, a computer-readable storage medium is also provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the methods described in the above method embodiments.

[0106] According to one embodiment of this application, a program product for implementing the methods in the above-described method embodiments is also provided. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of this invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0107] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0108] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0109] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0110] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as JAVA and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0111] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0112] Furthermore, although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0113] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0114] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the appended claims.

Claims

1. A method for auditing HTTPS encrypted traffic, characterized in that, The method includes: Receive and process key packets from the client to obtain a pre-master key and TLS session metadata, the session metadata including: client random number, server random number, and cipher suite identifier; Based on the TLS session metadata, the pre-master key is derivation processed to obtain the key parameters required to decrypt HTTPS traffic; When the HTTPS traffic of the mirror is obtained, the HTTPS traffic is decrypted using the key parameter to obtain plaintext application layer data; Perform content auditing on the application layer data.

2. The method according to claim 1, characterized in that, The process of receiving and processing key packets from the client to obtain the pre-master key and TLS session metadata includes: Receive encrypted transmission data sent by the client through a Transport Layer Security (TLS) channel via a secure service port; The encrypted transmission data is decrypted using the TLS layer to obtain a key packet, which includes: a pre-master key ciphertext and TLS session metadata; The pre-master key is obtained by decrypting the pre-master key ciphertext using the private key of the auditing device.

3. The method according to claim 2, characterized in that, The TLS session metadata also includes: timestamps and message authentication codes; Before decrypting the pre-master key ciphertext using the private key of the auditing device, the method further includes: The recalculated message authentication value is compared and verified with the message authentication code; The timestamp is compared with the current time to verify whether it is within a preset valid time window. If the message authentication code is verified and the timestamp is verified as valid, the decryption operation of the pre-master key ciphertext continues. If the message authentication code is verified as invalid or the timestamp is verified as invalid, the key packet is discarded and the processing flow of the key packet is terminated.

4. The method according to claim 1, characterized in that, The key derivation process performed on the pre-master key based on the TLS session metadata to obtain the key parameters required for decrypting HTTPS traffic includes: The corresponding hash algorithm is determined based on the cipher suite identifier; Using the pre-master key as input key material, a key derivation function extraction operation based on the hash algorithm is performed to obtain the intermediate key; Using the intermediate key, client random number, and server random number as input, a tagged key derivation function expansion operation is performed to obtain the key parameters.

5. The method according to claim 4, characterized in that, The key parameters include the client write key and the server write key; The process of decrypting the HTTPS traffic using the key parameters includes: Identify the transmission direction of the HTTPS traffic; If the transmission direction is from the client to the server, then the client write key is used for decryption; If the transmission direction is from the server to the client, then the server write key is used for decryption.

6. The method according to claim 1, characterized in that, When the HTTPS traffic of the mirror is obtained, the process of decrypting the HTTPS traffic using the key parameter includes: The corresponding key parameters are determined for the HTTPS traffic based on the correlation between the HTTPS traffic and the key parameters; The HTTPS traffic is decrypted using the defined key parameters.

7. The method according to claim 1, characterized in that, The content auditing of the application layer data includes: Use a regular expression engine to match sensitive data in the application layer data, where the sensitive data includes at least one of credit card numbers and ID card numbers; When sensitive data is matched and meets the preset blocking policy, based on the connection 5-tuple of the HTTPS traffic, TCP RST packets pointing to the client and server are forged and injected into the network to forcibly terminate the corresponding HTTPS connection. The connection quintuple includes the source IP address, source port, destination IP address, destination port, and protocol type.

8. An HTTPS encrypted traffic auditing device, characterized in that, The device includes: The receiving and processing module is configured to receive and process key packets from the client to obtain a pre-master key and TLS session metadata, wherein the session metadata includes: client random number, server random number and cipher suite identifier; The key derivation module is configured to perform key derivation processing on the pre-master key based on the TLS session metadata to obtain the key parameters required to decrypt HTTPS traffic; The traffic decryption module is configured to decrypt the HTTPS traffic using the key parameter when the mirrored HTTPS traffic is obtained, so as to obtain plaintext application layer data. The content auditing module is configured to perform content auditing on the application layer data.

9. An electronic device comprising a processor and a memory, the memory storing a computer program, wherein the processor executes the computer program to implement the steps of the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the processor of a computer, cause the computer to perform the method described in any one of claims 1 to 7.

Images